Tentative / rough to-do list for @marisbasha after our meeting today

initial implementation
- set up vak development enviornment as described here
- to test the set up and make sure you have configs for generating test data (next step), run nox -s test-data-generate (this will may crash because I introduced a bug in the parametric UMAP branch--I might have fixed it by the time you start--but it will run enough for you to get the config files you need)
- now that you have configs you need, generate a toy dataset to test the VAE model on: vak prep tests/data_for_tests/configs/ConvEncoderUMAP_train_audio_cbin_annot_notmat.toml
- using that toy dataset, get initial implementation working in a notebook, as described in the next section
add implementation to vak (these steps work in a notebook too, but this part describes adding them to vak in separate modules)
- declare the VAE specific loss function in vak.nn.loss.vae
  - not clear to me yet if there's something specific to the loss they use, it's defined in the forward method instead of a factored out loss function, here -- I would prefer to implement a AVALoss that verbatim repeats their logic, just to get closer to numerical reproducibility
- add vak.models.vae_model.VAEModel that defines the model family, using the vak.models.model_family decorator and sub-classing vak.base.Model. For an example see https://github.com/vocalpy/vak/blob
  /a96ff976283ccdc34852fcf2ba5bb51808b6b25e/src/vak/models/frame_classification_model.py#L22
  - I think we will need a training_step and validation_step with logic specific to datasets prepared by vak
  - we will probably also want VAE specific methods that make it easier to inspect model behavior interactively, e.g. encode and decode
- add vak.nets.ava with architecture here (slightly refactored, e.g. with for loops?). for an example see https://github.com/vocalpy/vak/blob/main/src/vak/nets/tweetynet.py
  - I would favor building the model inside __init__ instead of using a separate method, and I would favor using for loops + separate encoder + decoder attributes so that methods like encode can just do return self.encoder(x)
  - see for example the VanillaVAE implementation in Pytorch-VAE: https://github.com/AntixK/PyTorch-VAE/blob/master/models/vanilla_vae.py
- add vak.models.AVA that defines the AVA model, using @model(family='VAEModel')) -- for an example see https://github.com/vocalpy/vak/blob/main/src/vak/models/tweetynet.py
- add docstrings
- add unit tests

ENH: Add AVA models and datasets #674

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions