songlab-cal / mogwai Goto Github PK
View Code? Open in Web Editor NEWLicense: BSD 3-Clause "New" or "Revised" License
License: BSD 3-Clause "New" or "Revised" License
Currently single layers of attention aren't training very well, afaik because we haven't finalized the regularization implementation
Necessary Changes
TRRosetta_MSADataset
class for parsing the MSA from the npz file..npz
branch to read_contacts
.confirm that the following does work.
mogwai-train <pdb.npz> --structure_file <pdb.npz>
We are currently one-hotting directly in the MSADataModule
s. We should do something smarter to be compatible with alternative tokenizations and amino acid embedding layers.
Open to thoughts here.
Log a few matplotlib images to wandb if wandb logging enabled.
One hyperparameter has multiple defaults set all over the place, resulting in error-prone manual matching.
For example, num_repeats
for RepeatDataset
has a default in the add_args
method of RepeatDataset
and a default in the constructor of MSADataModule
. This happens in all models as well.
Currently the command
mogwai-train <msa_file.a3m>
fails at
Traceback (most recent call last):
[...]
File "/home/nickbhat/projects/mogwai/mogwai/models/base_model.py", line 56, in training_step
auc = self.get_auc(do_apc=False)
File "/home/nickbhat/projects/mogwai/venv/lib/python3.6/site-packages/torch/autograd/grad_mode.py", line 15, in decorate_context
return func(*args, **kwargs)
File "/home/nickbhat/projects/mogwai/mogwai/models/base_model.py", line 111, in get_auc
"Model not provided with ground truth contacts, precision can't be computed."
ValueError: Model not provided with ground truth contacts, precision can't be computed.
This is because BaseModel always attempts to calculate auc in the training step,
All of the explicitly logged metrics such as auc, auc_apc, etc get logged on wandb. Loss is not logged on wandb, but if you explicitly try to log it via
self.log("loss", loss, on_step=True, on_epoch=False, prog_bar=True)
it throws an error
File "/home/nickbhat/projects/mogwai/venv/bin/mogwai-train", line 33, in <module>
sys.exit(load_entry_point('mogwai-protein', 'console_scripts', 'mogwai-train')())
File "/home/nickbhat/projects/iclr-2021-factored-attention/mogwai/mogwai/train.py", line 101, in train
trainer.fit(model, msa_dm)
File "/home/nickbhat/projects/mogwai/venv/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 440, in fit
results = self.accelerator_backend.train()
File "/home/nickbhat/projects/mogwai/venv/lib/python3.6/site-packages/pytorch_lightning/accelerators/gpu_accelerator.py", line 54, in train
results = self.train_or_test()
File "/home/nickbhat/projects/mogwai/venv/lib/python3.6/site-packages/pytorch_lightning/accelerators/accelerator.py", line 66, in train_or_test
results = self.trainer.train()
File "/home/nickbhat/projects/mogwai/venv/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 483, in train
self.train_loop.run_training_epoch()
File "/home/nickbhat/projects/mogwai/venv/lib/python3.6/site-packages/pytorch_lightning/trainer/training_loop.py", line 550, in run_training_epoch
self.on_train_batch_end(epoch_output, epoch_end_outputs, batch, batch_idx, dataloader_idx)
File "/home/nickbhat/projects/mogwai/venv/lib/python3.6/site-packages/pytorch_lightning/trainer/training_loop.py", line 249, in on_train_batch_end
self.trainer.call_hook('on_train_batch_end', epoch_end_outputs, batch, batch_idx, dataloader_idx)
File "/home/nickbhat/projects/mogwai/venv/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 823, in call_hook
trainer_hook(*args, **kwargs)
File "/home/nickbhat/projects/mogwai/venv/lib/python3.6/site-packages/pytorch_lightning/trainer/callback_hook.py", line 147, in on_train_batch_end
callback.on_train_batch_end(self, self.get_model(), outputs, batch, batch_idx, dataloader_idx)
File "/home/nickbhat/projects/mogwai/venv/lib/python3.6/site-packages/pytorch_lightning/callbacks/progress.py", line 339, in on_train_batch_end
self.main_progress_bar.set_postfix(trainer.progress_bar_dict)
File "/home/nickbhat/projects/mogwai/venv/lib/python3.6/site-packages/pytorch_lightning/trainer/properties.py", line 155, in progress_bar_dict
return dict(**ref_model.get_progress_bar_dict(), **self.logger_connector.progress_bar_metrics)
TypeError: type object got multiple values for keyword argument 'loss'
Presumably this is because it's already logged, as loss is displayed in the progress bar.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.