google-deepmind / alphafold Goto Github PK
View Code? Open in Web Editor NEWOpen source code for AlphaFold.
License: Apache License 2.0
Open source code for AlphaFold.
License: Apache License 2.0
When I attempted to build the Docker container for alphafold, the build process failed at step 16/19. This step involves applying a patch to OpenMM.
I think the issue is that the patch tries to apply to /opt/conda/lib/python3.8 whereas the Docker build image has Python 3.9 installed. I don't know whether this is caused by a change in miniconda-latest or OS default python versions.
(Ubuntu 18.04 LTS host OS)
TL;DR is it okay to use the jaxlib version of 0.1.68+cuda110 instead of 0.1.69+cuda110?
I have tried to write a script and construct a conda environment that does not use Docker. When I used the same versions of jax and jaxlib defined in the docker/Dockerfile, I had some issues during the inference time. Scripts were working fine for model_{1,3,4} but raised CUDA_ERROR_ILLEGAL_ADDRESS errors for model_{2,5}. I have no idea why it happened...
So, I tested many variants of the environment and found out that jax=0.2.17 (probably, it is the same version of the original) and jaxlib=0.1.68+cuda110 (it is the version for installing jax with a command
pip3 install jax[cuda110] -f https://storage.googleapis.com/jax-releases/jax_releases.html
) are okay to run smoothly without Docker, but with my custom conda environment.
While it's great you finally release your code publicly (after months of criticism for keeping it secret against the ethos of mutual aid in the scientific world) it certainly seems strange to do it just around the time RoseTTAFold , a system that does nearly the same thing for free at a fraction of the computational cost, gains significant traction in news stories.
That sudden release might give the impression of an orchestrated effort of your PR department to bury the RoseTTAFold news story.
I'm certain you will deny any of this, but to be without the shadow of a doubt in the future, DeepMind might want to share their work openly and proactively with the scientific world, instead of releasing what seems in a defensive way.
Is it possible to run the feature calculation process using the MSAs computed on an earlier run? Does one need the HHR file corresponding to hhsearch_hits
below?
https://github.com/deepmind/alphafold/blob/main/alphafold/data/pipeline.py#L147
It doesn't look like it's saved as the other sto
/a3m
files are.
The README states the need to put "the databases on an additional 3 TB disk". Should this be an SSD?
Just a FYI. Running on an Ampere A10 with CC 8.6.
I0725 13:13:22.066373 139761799472960 run_docker.py:200] 2021-07-25 03:13:22.065889: W external/org_tensorflow/tensorflow/stream_executor/gpu/asm_compiler.cc:235] Falling back to the CUDA driver for PTX compilation; ptxas does not support CC 8.6
I0725 13:13:22.066596 139761799472960 run_docker.py:200] 2021-07-25 03:13:22.065923: W external/org_tensorflow/tensorflow/stream_executor/gpu/asm_compiler.cc:238] Used ptxas at ptxas
I0725 13:13:22.066839 139761799472960 run_docker.py:200] 2021-07-25 03:13:22.066559: E external/org_tensorflow/tensorflow/stream_executor/cuda/cuda_driver.cc:625] failed to get PTX kernel "shift_right_logical_3" from module: CUDA_ERROR_NOT_FOUND: named symbol not found
I0725 13:13:22.067006 139761799472960 run_docker.py:200] 2021-07-25 03:13:22.066620: E external/org_tensorflow/tensorflow/compiler/xla/pjrt/pjrt_stream_executor_client.cc:2040] Execution of replica 0 failed: Internal: Could not find the corresponding function
I0725 13:13:22.068676 139761799472960 run_docker.py:200] Traceback (most recent call last):
I0725 13:13:22.068786 139761799472960 run_docker.py:200] File "/app/alphafold/run_alphafold.py", line 303, in
I0725 13:13:22.068876 139761799472960 run_docker.py:200] app.run(main)
I0725 13:13:22.068992 139761799472960 run_docker.py:200] File "/opt/conda/lib/python3.7/site-packages/absl/app.py", line 312, in run
I0725 13:13:22.069080 139761799472960 run_docker.py:200] _run_main(main, args)
I0725 13:13:22.069165 139761799472960 run_docker.py:200] File "/opt/conda/lib/python3.7/site-packages/absl/app.py", line 258, in _run_main
I0725 13:13:22.069255 139761799472960 run_docker.py:200] sys.exit(main(argv))
I0725 13:13:22.069342 139761799472960 run_docker.py:200] File "/app/alphafold/run_alphafold.py", line 285, in main
I0725 13:13:22.069428 139761799472960 run_docker.py:200] random_seed=random_seed)
I0725 13:13:22.069509 139761799472960 run_docker.py:200] File "/app/alphafold/run_alphafold.py", line 149, in predict_structure
I0725 13:13:22.069588 139761799472960 run_docker.py:200] prediction_result = model_runner.predict(processed_feature_dict)
I0725 13:13:22.069675 139761799472960 run_docker.py:200] File "/app/alphafold/alphafold/model/model.py", line 134, in predict
I0725 13:13:22.069755 139761799472960 run_docker.py:200] result = self.apply(self.params, jax.random.PRNGKey(0), feat)
I0725 13:13:22.069834 139761799472960 run_docker.py:200] File "/opt/conda/lib/python3.7/site-packages/jax/_src/random.py", line 75, in PRNGKey
I0725 13:13:22.069914 139761799472960 run_docker.py:200] k1 = convert(lax.shift_right_logical(seed_arr, lax._const(seed_arr, 32)))
I0725 13:13:22.070003 139761799472960 run_docker.py:200] File "/opt/conda/lib/python3.7/site-packages/jax/_src/lax/lax.py", line 382, in shift_right_logical
I0725 13:13:22.070081 139761799472960 run_docker.py:200] return shift_right_logical_p.bind(x, y)
I0725 13:13:22.070159 139761799472960 run_docker.py:200] File "/opt/conda/lib/python3.7/site-packages/jax/core.py", line 264, in bind
I0725 13:13:22.070236 139761799472960 run_docker.py:200] out = top_trace.process_primitive(self, tracers, params)
I0725 13:13:22.070315 139761799472960 run_docker.py:200] File "/opt/conda/lib/python3.7/site-packages/jax/core.py", line 604, in process_primitive
I0725 13:13:22.070394 139761799472960 run_docker.py:200] return primitive.impl(*tracers, **params)
I0725 13:13:22.070472 139761799472960 run_docker.py:200] File "/opt/conda/lib/python3.7/site-packages/jax/interpreters/xla.py", line 262, in apply_primitive
I0725 13:13:22.070549 139761799472960 run_docker.py:200] return compiled_fun(*args)
I0725 13:13:22.070631 139761799472960 run_docker.py:200] File "/opt/conda/lib/python3.7/site-packages/jax/interpreters/xla.py", line 378, in _execute_compiled_primitive
I0725 13:13:22.070705 139761799472960 run_docker.py:200] out_bufs = compiled.execute(input_bufs)
I0725 13:13:22.070770 139761799472960 run_docker.py:200] RuntimeError: Internal: Could not find the corresponding function
In supplementary Table 4 of the published Nature paper, it says it takes about 7days to train initially and 4 days to fine-tune, but we can't see any message about the number of gpus that were used? Could you please clarify that?
Hi,
I would like to use alphafold for commercial purposes, is it possible to license the model?
Hi I'm trying to get oligomeric structure of a protein. I'm able to get the monomer through AlphaFold Colab but when i try to use the oligomeric feature it is crashing.
Error # Your session crashed after using all available RAM. If you are interested in access to high-RAM runtimes, you may want to check out Colab Pro.
NameError Traceback (most recent call last)
in ()
5 use_model = {}
6 if "model_params" not in dir(): model_params = {}
----> 7 for model_name in ["model_1","model_2","model_3","model_4","model_5"][:num_models]:
8 use_model[model_name] = True
9 if model_name not in model_params:
NameError: name 'num_models' is not defined
When i try to use local runtime, getting another error...
Unrecognized runtime "python3"; defaulting to "python2"
Please help, if anyone have any solution for this.
Thanks
Pankaj
In trying to repro the Google Cloud setup specified, I am seeing the following log messages using the Docker image approach. They are "INFO" level -- are they an indication that my setup is incorrect and I would get sub-optimal performance?
I0718 14:02:57.560956 139711506118464 run_docker.py:180] I0718 14:02:57.560250 139800994285376 tpu_client.py:54] Starting the local TPU driver.
I0718 14:02:57.561176 139711506118464 run_docker.py:180] I0718 14:02:57.560675 139800994285376 xla_bridge.py:231] Unable to initialize backend 'tpu_driver': Not found: Unable to find driver in registry given worker: local://
I0718 14:02:57.777399 139711506118464 run_docker.py:180] I0718 14:02:57.776700 139800994285376 xla_bridge.py:231] Unable to initialize backend 'tpu': Invalid argument: TpuPlatform is not available.
After I built the docker image, I found it failed to run. Error messages as below. I need help to fix it.
/opt/conda/lib/python3.7/site-packages/absl/flags/_validators.py:206: UserWarning: Flag --preset has a non-None default value; therefore, mark_flag_as_required will pass even if flag is not specified in the command line!
'command line!' % flag_name)
FATAL Flags parsing error:
flag --fasta_paths=None: Flag --fasta_paths must have a value other than None.
flag --output_dir=None: Flag --output_dir must have a value other than None.
flag --model_names=None: Flag --model_names must have a value other than None.
flag --data_dir=None: Flag --data_dir must have a value other than None.
flag --uniref90_database_path=None: Flag --uniref90_database_path must have a value other than None.
flag --mgnify_database_path=None: Flag --mgnify_database_path must have a value other than None.
flag --pdb70_database_path=None: Flag --pdb70_database_path must have a value other than None.
flag --template_mmcif_dir=None: Flag --template_mmcif_dir must have a value other than None.
flag --max_template_date=None: Flag --max_template_date must have a value other than None.
flag --obsolete_pdbs_path=None: Flag --obsolete_pdbs_path must have a value other than None.
Pass --helpshort or --helpfull to see help on flags.
/opt/conda/lib/python3.7/site-packages/absl/flags/_validators.py:206: UserWarning: Flag --preset has a non-None default value; therefore, mark_flag_as_required will pass even if flag is not specified in the command line!
'command line!' % flag_name)
FATAL Flags parsing error:
flag --fasta_paths=None: Flag --fasta_paths must have a value other than None.
flag --output_dir=None: Flag --output_dir must have a value other than None.
flag --model_names=None: Flag --model_names must have a value other than None.
flag --data_dir=None: Flag --data_dir must have a value other than None.
flag --uniref90_database_path=None: Flag --uniref90_database_path must have a value other than None.
flag --mgnify_database_path=None: Flag --mgnify_database_path must have a value other than None.
flag --pdb70_database_path=None: Flag --pdb70_database_path must have a value other than None.
flag --template_mmcif_dir=None: Flag --template_mmcif_dir must have a value other than None.
flag --max_template_date=None: Flag --max_template_date must have a value other than None.
flag --obsolete_pdbs_path=None: Flag --obsolete_pdbs_path must have a value other than None.
Pass --helpshort or --helpfull to see help on flags.
/opt/conda/lib/python3.7/site-packages/absl/flags/_validators.py:206: UserWarning: Flag --preset has a non-None default value; therefore, mark_flag_as_required will pass even if flag is not specified in the command line!
'command line!' % flag_name)
FATAL Flags parsing error:
flag --fasta_paths=None: Flag --fasta_paths must have a value other than None.
flag --output_dir=None: Flag --output_dir must have a value other than None.
flag --model_names=None: Flag --model_names must have a value other than None.
flag --data_dir=None: Flag --data_dir must have a value other than None.
flag --uniref90_database_path=None: Flag --uniref90_database_path must have a value other than None.
flag --mgnify_database_path=None: Flag --mgnify_database_path must have a value other than None.
flag --pdb70_database_path=None: Flag --pdb70_database_path must have a value other than None.
flag --template_mmcif_dir=None: Flag --template_mmcif_dir must have a value other than None.
flag --max_template_date=None: Flag --max_template_date must have a value other than None.
flag --obsolete_pdbs_path=None: Flag --obsolete_pdbs_path must have a value other than None.
Pass --helpshort or --helpfull to see help on flags.
Hi,
I cannot download uniclust30, Dataset from the script. Also from another repository RoseTTAFold, I cannot download uniref30 either.
I'm not major in biology, I do not familiar with this organization wwwuser.gwdg.de.
Can anyone really download these datasets?
Thanks!!!
Following are the download URLs:
uniclust30: http://wwwuser.gwdg.de/~compbiol/uniclust/2018_08/uniclust30_2018_08_hhsuite.tar.gz
uniref30: http://wwwuser.gwdg.de/~compbiol/uniclust/2020_06/UniRef30_2020_06_hhsuite.tar.gz
Thank you for the nice work! It worked smoothly and was faster than I expected before :)
I have a question about the predicted residue-wise errors. I guess the information is stored in $output_dir/result_model_{1,2,3,4,5}.pkl, but the information is not recorded in the output PDB files. I found that pkl['plddt'] is for the predicted residue-wise errors. Is it the correct thing that I am looking for?
Hi,
I want to use alphafold2, but the Linux server has no GPU and only CPU. Is it OK? Thank you!
hqin
Hi,
I want to turn off the amber relaxation option without modifications of codes.
When I commented out the relaxation option in the script, 'run_alphafold.py',
I could turn off it.
But, is it also possible to adjust it using the argument ?
Thanks.
Thank you for your work fixing #13 -- that really helped!
I am now seeing a similar download bottleneck with uniclust30 (it has effectively stalled).
The README implies that AlphaFold would at least prefer to use GPUs. However, it doesn't mention whether or not GPUs are required.
Due to the network, I use non-docker version. But in the process of Inference, there was a problem with hhsearch๏ผ
RuntimeError: HHSearch failed:
stdout:
stderr:
Can someone give me some suggestions or solutions? thanks
The download script for mgnify downloaded and created file, mgy_clusters.fa , however couldn't locate mgy_clusters_2018_08.fa which seems to be referenced by /app/alphafold/alphafold/data/tools/jackhmmer.py
packages/absl/flags/_validators.py:203: UserWarning: Flag --preset has a non-None default value; therefore, mark_flag_as_required will pass even if flag is not specified in the command line!
I0724 21:57:01.485210 139819066255168 run_docker.py:193] warnings.warn(
I0724 21:57:02.218386 139819066255168 run_docker.py:193] I0724 21:57:02.217257 140387317352256 templates.py:880] Using precomputed obsolete pdbs /mnt/obsolete_pdbs_path/obsolete.dat.
I0724 21:57:02.224482 139819066255168 run_docker.py:193] E0724 21:57:02.223666 140387317352256 jackhmmer.py:65] Could not find Jackhmmer database /mnt/mgnify_database_path/mgy_clusters_2018_08.fa
I0724 21:57:02.225311 139819066255168 run_docker.py:193] Traceback (most recent call last):
I0724 21:57:02.225568 139819066255168 run_docker.py:193] File "/app/alphafold/run_alphafold.py", line 283, in <module>
I0724 21:57:02.225908 139819066255168 run_docker.py:193] app.run(main)
I0724 21:57:02.226118 139819066255168 run_docker.py:193] File "/opt/conda/lib/python3.8/site-packages/absl/app.py", line 312, in run
I0724 21:57:02.226256 139819066255168 run_docker.py:193] _run_main(main, args)
I0724 21:57:02.226412 139819066255168 run_docker.py:193] File "/opt/conda/lib/python3.8/site-packages/absl/app.py", line 258, in _run_main
I0724 21:57:02.226542 139819066255168 run_docker.py:193] sys.exit(main(argv))
I0724 21:57:02.226696 139819066255168 run_docker.py:193] File "/app/alphafold/run_alphafold.py", line 218, in main
I0724 21:57:02.226851 139819066255168 run_docker.py:193] data_pipeline = pipeline.DataPipeline(
I0724 21:57:02.226989 139819066255168 run_docker.py:193] File "/app/alphafold/alphafold/data/pipeline.py", line 104, in __init__
I0724 21:57:02.227127 139819066255168 run_docker.py:193] self.jackhmmer_mgnify_runner = jackhmmer.Jackhmmer(
I0724 21:57:02.227246 139819066255168 run_docker.py:193] File "/app/alphafold/alphafold/data/tools/jackhmmer.py", line 66, in __init__
I0724 21:57:02.227406 139819066255168 run_docker.py:193] raise ValueError(f'Could not find Jackhmmer database {database_path}')
I0724 21:57:02.227567 139819066255168 run_docker.py:193] ValueError: Could not find Jackhmmer database /mnt/mgnify_database_path/mgy_clusters_2018_08.fa
When I run the scripts/download_all_data.sh to download the databases I get the error:
Downloading PDB mmCIF files...
Running rsync to fetch all mmCIF files (note that the rsync progress estimate might be inaccurate)...
rsync: failed to connect to rsync.rcsb.org (132.249.210.234): Connection timed out (110)
rsync error: error in socket IO (code 10) at clientserver.c(125) [Receiver=3.1.2]
Hi,
Following the steps to run AlphaFold via the Colab module, the "Search against genetic databases" cell cannot find the 'alphafold.model' module. Note : the "Download AlphaFold" code ran without problem so I assume the module should be installed.
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-22-1640ae7ce3a3> in <module>()
24 import py3Dmol
25
---> 26 from alphafold.model import model
27 from alphafold.model import config
28 from alphafold.model import data
ModuleNotFoundError: No module named 'alphafold.model'
Alphafold has an issue when it comes to a class of proteins known as membrane proteins. Briefly, membrane proteins are located in multiple cell compartments where the domains of the membrane protein are separated by a physical barrier.
I have encountered multiple examples of alphafold becoming confused and trying to get domains that are biologically seperated to interact. Luckily a large training set of annotated proteins already exists in the uniprot database. For example, the insulin receptor (https://www.uniprot.org/uniprot/P06213#subcellular_location) have extracellular domains a transmembrane domain and a intra cellular domain. Thus only domains in the same compartments needs to be close to each other in the final fold, whereas domains in separate compartments need only be within a distance that can be spanned by the peptide chain.
Additionally, some protein are modified after translation. These modification include formation of disulfide bridges, which locks the distance and geometry of the two involved cysteine residues. This information is also available in uniprot (https://www.uniprot.org/uniprot/P06213#ptm_processing)
While I recognize that alphafold was designed for de novo prediction, I wonder how more powerful it could become if it were allowed to incorporate known biochemical information into its predictions.
While this is not per say an issue, it is something that should be tried.
Dear AlphaFold authors,
Since most of the IT does not allow docker on their HPC's or production servers, we have made a small attempt to create a How-To for non-docker AlphaFold setup.
Can be found here: https://github.com/kalininalab/alphafold_non_docker
Thank you for the tool!
Kind regards,
Sanjay
Hi,
This code is generating a lot of interest in the advanced research computing community. Unfortunately, it currently can not be supported on most clusters because of its use of docker, and its lack of release versions.
Any plan to address these would be appreciated:
Hi,
I ran alphafold on a 2k sequence using 2x V100 (32 GB) GPU. Like for shorter sequences, 29 GB are allocated on the first GPU and 300 MB on the other from the start. After hhblits I got an out-of-memory error from JAX that 26 GB couldn't be allocated. Does everything need to be loaded to a single GPU or should it in theory be possible to use the next GPU if the first one runs out of memory? Why are 300 MB allocated to the second GPU from the start?
Hi!
It looks like our server's GPU nodes only support up to CUDA 10.2. With the downgraded versions of tensorflow and other modules/packages, will be output consistent with those produced from the default set-up?
Thanks!
Hi,
I have been trying to download the MGnify dataset, but the host is unreachable.
"Connecting to ftp.ebi.ac.uk (ftp.ebi.ac.uk)|193.62.197.74|:21... failed: Connection timed out."
Is it possible to put this dataset on google cloud as well?
If running alphafold using the full_dbs setting does that mean only the apache license applies? Is it only the casp14 parameters that are covered under the creative commons license?
Got a system for which the features.pkl is 5 GB. Any workaround for this?
Traceback (most recent call last):
File "/X/alphafold/run_alphafold.py", line 283, in <module>
app.run(main)
File "/X/miniconda3/envs/alphafold/lib/python3.8/site-packages/absl/app.py", line 312, in run
_run_main(main, args)
File "/X/miniconda3/envs/alphafold/lib/python3.8/site-packages/absl/app.py", line 258, in _run_main
sys.exit(main(argv))
File "/X/alphafold/run_alphafold.py", line 255, in main
predict_structure(
File "/X/alphafold/run_alphafold.py", line 132, in predict_structure
processed_feature_dict = model_runner.process_features(
File "/X/alphafold/alphafold/model/model.py", line 103, in process_features
return features.np_example_to_features(
File "/X/alphafold/alphafold/model/features.py", line 93, in np_example_to_features
tensor_dict = proteins_dataset.np_to_tensor_dict(
File "/X/alphafold/alphafold/model/tf/proteins_dataset.py", line 162, in np_to_tensor_dict
tensor_dict = {k: tf.constant(v) for k, v in np_example.items()
File "/X/alphafold/alphafold/model/tf/proteins_dataset.py", line 162, in <dictcomp>
tensor_dict = {k: tf.constant(v) for k, v in np_example.items()
File "/X/miniconda3/envs/alphafold/lib/python3.8/site-packages/tensorflow/python/framework/constant_op.py", line 162, in constant_v1
return _constant_impl(value, dtype, shape, name, verify_shape=verify_shape,
File "/X/miniconda3/envs/alphafold/lib/python3.8/site-packages/tensorflow/python/framework/constant_op.py", line 281, in _constant_impl
tensor_util.make_tensor_proto(
File "/X/miniconda3/envs/alphafold/lib/python3.8/site-packages/tensorflow/python/framework/tensor_util.py", line 527, in make_tensor_proto raise ValueError(
ValueError: Cannot create a tensor proto whose content is larger than 2GB.
Thanks for the amazing work! I've managed to make it work in my machine, it was really easy as the instructions are very clear. I just have a question about the output of the program, in particular the distogram. I've seen they are stored in the result_model_{1,2,3,4,5}.pkl
pickle files, under the distogram
key. However, I'm not sure how to interpret this subdictionary. My suspicion is that bin_edges
contains the distance boundaries for each bin (closed or open intervals?), and logits
is a 3d array with shape (nres, nres, bins) where each z-layer corresponds with the probability of the residue pair being in the bin defined at bin_edges
. Is this the case? Thanks!
I cannot get aria2 to install (sudo yum install aria2). Is there a way to get the databases without aria2?
It is still in extracting MSA.
Could anyone share the running time on T1050 (779 residues)? Thanks very much!
An online service for predicting the structure of a protein (similar to I-TASSER server) would be useful for academic users.
When using a docker image created from the repo, libs like cusolver and libcudnn are not installed. I get the following set of warnings, ending with "Skipping registering GPU devices".
Was the docker image meant to include these libs? I see libcusolver.so.10 (a different version than listed in the warning), in the 11.0-runtime image (the Dockerfile is based on 11.0-base). And cudnn would require an additional install step.
I0718 14:20:35.488188 139711506118464 run_docker.py:180] 2021-07-18 14:20:35.487292: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcusolver.so.11'; dlerror: libcusolver.so.11: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64
I0718 14:20:35.632766 139711506118464 run_docker.py:180] 2021-07-18 14:20:35.632116: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudnn.so.8'; dlerror: libcudnn.so.8: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64
I0718 14:20:35.633013 139711506118464 run_docker.py:180] 2021-07-18 14:20:35.632167: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1766] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
I0718 14:20:35.633130 139711506118464 run_docker.py:180] Skipping registering GPU devices...
Thank you for sharing the AlphaFold codebase!
I wanted to suggest releasing the package to pypi to make it easier to install especially in production environments, if possible, including the run_alphafold
script.
Please let me know if you'll consider this suggestion.
I am running colab module of alphafold but it gives me the following error although I have followed the steps. I ran it once on one of the protein and it worked but now it seems to not find the module. I hope you can help.
Regards,
Rubin.
ModuleNotFoundError Traceback (most recent call last)
in ()
24 import py3Dmol
25
---> 26 from alphafold.model import model
27 from alphafold.model import config
28 from alphafold.model import data
ModuleNotFoundError: No module named 'alphafold.model'
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.
Hello,
Right now, we have a machine with 12 GB vRAM, and we get the error CUDA error out of memory. when folding with model 4 or 5. We suspect this is because the GPU does not have enough memory for the model. If so, our lab is wondering if there are ways to decrease the memory requirements. If not, we can try to find a different machine or GPU to run it.
Thanks!
What's the preferred operation system? Can it be set up on Windows?
Hello,
I am trying to run AlphaFold using docker (command: python3 docker/run_docker.py --use_gpu=False --fasta_paths=seq.fa --max_template_date=2021-12-12
) and I get this error:
Traceback (most recent call last):
File "[HOME_dir_redacted]/.local/lib/python3.9/site-packages/docker/api/client.py", line 268, in _raise_for_status
response.raise_for_status()
File "/usr/lib/python3/dist-packages/requests/models.py", line 943, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http+docker://localhost/v1.41/containers/createDuring handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "[HOME_dir_redacted]/Software/alphafold/docker/run_docker.py", line 201, in
app.run(main)
File "[HOME_dir_redacted]/.local/lib/python3.9/site-packages/absl/app.py", line 312, in run
_run_main(main, args)
File "[HOME_dir_redacted]/.local/lib/python3.9/site-packages/absl/app.py", line 258, in _run_main
sys.exit(main(argv))
File "[HOME_dir_redacted]/Software/alphafold/docker/run_docker.py", line 173, in main
container = client.containers.run(
File "[HOME_dir_redacted]/.local/lib/python3.9/site-packages/docker/models/containers.py", line 811, in run
container = self.create(image=image, command=command,
File "[HOME_dir_redacted]/.local/lib/python3.9/site-packages/docker/models/containers.py", line 870, in create
resp = self.client.api.create_container(**create_kwargs)
File "[HOME_dir_redacted]/.local/lib/python3.9/site-packages/docker/api/container.py", line 430, in create_container
return self.create_container_from_config(config, name)
File "[HOME_dir_redacted]/.local/lib/python3.9/site-packages/docker/api/container.py", line 441, in create_container_from_config
return self._result(res, True)
File "[HOME_dir_redacted]/.local/lib/python3.9/site-packages/docker/api/client.py", line 274, in _result
self._raise_for_status(response)
File "[HOME_dir_redacted]/.local/lib/python3.9/site-packages/docker/api/client.py", line 270, in _raise_for_status
raise create_api_error_from_http_exception(e)
File "[HOME_dir_redacted]/.local/lib/python3.9/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 400 Client Error for http+docker://localhost/v1.41/containers/create: Bad Request ("invalid mount config for type "bind": bind source path does not exist: /tmp/alphafold")
I did not set up any keys for Docker. Is this related? And how to fix this? I tried removing the alphafold image and purging/reinstalling docker. Still, nothing helps.
Thanks,
Lukasz
Pretty much self-explanatory.
Hi Developers of AlphaFold2,
Thanks for sharing alphafold! It's really exciting!
I noticed that a file, alphafold/common/stereo_chemical_props.txt
, containing structural parameters, is needed for model relaxation as indicated in alphafold/common/residue_constants.py
line 406. But this file is actually missing in folder alphafold/common
Although it's not hard to find a similar file in openstructure (https://openstructure.org), I think adding this file to the folder would be great for others!
Best,
Junhui
Using the same random seed, looks like 'model_1_ptm' and 'model_1' do not output identical results.
Is there any reason we should use 'model_1_ptm', 'model_2_ptm' ,'model_3_ptm' ,'model_4_ptm' ,'model_5_ptm' V.S. 'model_1', 'model_2' ,'model_3' ,'model_4' ,'model_5'?
Is there any assurance difference, please?
Hi,
I want to get gdt and lddt score
please give me this solutions.
Wonderful repo !!! Thanks so much for the Alphafold2 Team. And I have a problem in building alphafold docker. Error information is shown below. It seems that the related cuda resource is not found. Thanks all.
$ docker build -f docker/Dockerfile -t alphafold .
Sending build context to Docker daemon 12.69MB
Step 1/19 : ARG CUDA=11.0
Step 2/19 : FROM nvidia/cuda:${CUDA}-base
---> 2ec708416bb8
Step 3/19 : ARG CUDA
---> Using cache
---> 076eace7d488
Step 4/19 : SHELL ["/bin/bash", "-c"]
---> Using cache
---> b57b88dc2b9a
Step 5/19 : RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y build-essential cmake cuda-command-line-tools-${CUDA/./-} git hmmer kalign tzdata wget && rm -rf /var/lib/apt/lists/*
---> Running in 52e953f25c2b
...
...
E: Failed to fetch https://developer.download.nvidia.cn/compute/cuda/repos/ubuntu2004/x86_64/by-hash/SHA256/27b2dbb347d54776dec155d98e1eefbde6c10a3fd1295007c3e836cfd9b98522 404 Not Found [IP: 58.205.210.80 443]
E: Some index files failed to download. They have been ignored, or old ones used instead.
Hi,
this is not really an issue with the code but it seems that downloading the BFD database is a real bottleneck (at least for me). I tried aria2c and wget and with both I only get around 200 kb/s download rate. It is certainly not a limit of bandwidth on my side. Estimated time of completion is 9d! Does anyone maybe have the file mirrored?
Did anyone solve the cuSolver internal error?
I0716 21:59:04.723145 139668278384448 run_docker.py:180] WARNING:tensorflow:From /app/alphafold/alphafold/model/tf/input_pipeline.py:151: calling map_fn (from tensorflow.python.ops.map_fn) with dtype is deprecated and will be removed in a future version.
I0716 21:59:04.723469 139668278384448 run_docker.py:180] Instructions for updating:
I0716 21:59:04.723672 139668278384448 run_docker.py:180] Use fn_output_signature instead
I0716 21:59:04.723857 139668278384448 run_docker.py:180] W0716 13:59:04.722220 140425547745088 deprecation.py:528] From /app/alphafold/alphafold/model/tf/input_pipeline.py:151: calling map_fn (from tensorflow.python.ops.map_fn) with dtype is deprecated and will be removed in a fut
ure version.
I0716 21:59:04.724043 139668278384448 run_docker.py:180] Instructions for updating:
I0716 21:59:04.724218 139668278384448 run_docker.py:180] Use fn_output_signature instead
I0716 21:59:08.106853 139668278384448 run_docker.py:180] 2021-07-16 13:59:08.105871: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudnn.so.8'; dlerror: libcudnn.so.8: cannot open shared object file: No such file or directory; L
D_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64
I0716 21:59:08.107234 139668278384448 run_docker.py:180] 2021-07-16 13:59:08.105914: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1766] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GP
U. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
I0716 21:59:08.107458 139668278384448 run_docker.py:180] Skipping registering GPU devices...
I0716 21:59:09.326027 139668278384448 run_docker.py:180] I0716 13:59:09.324977 140425547745088 model.py:132] Running predict with shape(feat) = {'aatype': (4, 44), 'residue_index': (4, 44), 'seq_length': (4,), 'template_aatype': (4, 4, 44), 'template_all_atom_masks': (4, 4, 44, 37
), 'template_all_atom_positions': (4, 4, 44, 37, 3), 'template_sum_probs': (4, 4, 1), 'is_distillation': (4,), 'seq_mask': (4, 44), 'msa_mask': (4, 508, 44), 'msa_row_mask': (4, 508), 'random_crop_to_size_seed': (4, 2), 'template_mask': (4, 4), 'template_pseudo_beta': (4, 4, 44, 3
), 'template_pseudo_beta_mask': (4, 4, 44), 'atom14_atom_exists': (4, 44, 14), 'residx_atom14_to_atom37': (4, 44, 14), 'residx_atom37_to_atom14': (4, 44, 37), 'atom37_atom_exists': (4, 44, 37), 'extra_msa': (4, 5120, 44), 'extra_msa_mask': (4, 5120, 44), 'extra_msa_row_mask': (4,
5120), 'bert_mask': (4, 508, 44), 'true_msa': (4, 508, 44), 'extra_has_deletion': (4, 5120, 44), 'extra_deletion_value': (4, 5120, 44), 'msa_feat': (4, 508, 44, 49), 'target_feat': (4, 44, 22)}
I0716 21:59:58.832500 139668278384448 run_docker.py:180] 2021-07-16 13:59:58.831660: W external/org_tensorflow/tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcusolver.so.10'; dlerror: libcusolver.so.10: cannot open shared object fi
le: No such file or directory; LD_LIBRARY_PATH: /usr/local/nvidia/lib:/usr/local/nvidia/lib64
I0716 21:59:58.929851 139668278384448 run_docker.py:180] Traceback (most recent call last):
I0716 21:59:58.930104 139668278384448 run_docker.py:180] File "/app/alphafold/run_alphafold.py", line 283, in <module>
I0716 21:59:58.930308 139668278384448 run_docker.py:180] app.run(main)
I0716 21:59:58.930593 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/absl/app.py", line 312, in run
I0716 21:59:58.930781 139668278384448 run_docker.py:180] _run_main(main, args)
I0716 21:59:58.930959 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/absl/app.py", line 258, in _run_main
I0716 21:59:58.931135 139668278384448 run_docker.py:180] sys.exit(main(argv))
I0716 21:59:58.931310 139668278384448 run_docker.py:180] File "/app/alphafold/run_alphafold.py", line 255, in main
I0716 21:59:58.931483 139668278384448 run_docker.py:180] predict_structure(
I0716 21:59:58.931658 139668278384448 run_docker.py:180] File "/app/alphafold/run_alphafold.py", line 137, in predict_structure
I0716 21:59:58.931832 139668278384448 run_docker.py:180] prediction_result = model_runner.predict(processed_feature_dict)
I0716 21:59:58.932008 139668278384448 run_docker.py:180] File "/app/alphafold/alphafold/model/model.py", line 134, in predict
I0716 21:59:58.932183 139668278384448 run_docker.py:180] result = self.apply(self.params, jax.random.PRNGKey(0), feat)
I0716 21:59:58.932358 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/jax/_src/traceback_util.py", line 183, in reraise_with_filtered_traceback
I0716 21:59:58.932534 139668278384448 run_docker.py:180] return fun(*args, **kwargs)
I0716 21:59:58.932709 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/jax/_src/api.py", line 424, in cache_miss
I0716 21:59:58.932884 139668278384448 run_docker.py:180] out_flat = xla.xla_call(
I0716 21:59:58.933057 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/jax/core.py", line 1560, in bind
I0716 21:59:58.933231 139668278384448 run_docker.py:180] return call_bind(self, fun, *args, **params)
I0716 21:59:58.933458 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/jax/core.py", line 1551, in call_bind
I0716 21:59:58.933639 139668278384448 run_docker.py:180] outs = primitive.process(top_trace, fun, tracers, params)
I0716 21:59:58.933813 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/jax/core.py", line 1563, in process
I0716 21:59:58.933987 139668278384448 run_docker.py:180] return trace.process_call(self, fun, tracers, params)
I0716 21:59:58.934161 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/jax/core.py", line 606, in process_call
I0716 21:59:58.934336 139668278384448 run_docker.py:180] return primitive.impl(f, *tracers, **params)
I0716 21:59:58.934510 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/jax/interpreters/xla.py", line 592, in _xla_call_impl
I0716 21:59:58.934684 139668278384448 run_docker.py:180] compiled_fun = _xla_callable(fun, device, backend, name, donated_invars,
I0716 21:59:58.934857 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/jax/linear_util.py", line 262, in memoized_fun
I0716 21:59:58.935029 139668278384448 run_docker.py:180] ans = call(fun, *args)
I0716 21:59:58.935202 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/jax/interpreters/xla.py", line 723, in _xla_callable
I0716 21:59:58.935374 139668278384448 run_docker.py:180] out_nodes = jaxpr_subcomp(
I0716 21:59:58.935548 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/jax/interpreters/xla.py", line 462, in jaxpr_subcomp
I0716 21:59:58.935724 139668278384448 run_docker.py:180] ans = rule(c, axis_env, extend_name_stack(name_stack, eqn.primitive.name),
I0716 21:59:58.935896 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/jax/_src/lax/control_flow.py", line 350, in _while_loop_translation_rule
I0716 21:59:58.936069 139668278384448 run_docker.py:180] new_z = xla.jaxpr_subcomp(body_c, body_jaxpr.jaxpr, backend, axis_env,
I0716 21:59:58.936244 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/jax/interpreters/xla.py", line 462, in jaxpr_subcomp
I0716 21:59:58.936418 139668278384448 run_docker.py:180] ans = rule(c, axis_env, extend_name_stack(name_stack, eqn.primitive.name),
I0716 21:59:58.936592 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/jax/interpreters/xla.py", line 1040, in f
I0716 21:59:58.936766 139668278384448 run_docker.py:180] outs = jaxpr_subcomp(c, jaxpr, backend, axis_env, _xla_consts(c, consts),
I0716 21:59:58.936941 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/jax/interpreters/xla.py", line 462, in jaxpr_subcomp
I0716 21:59:58.937116 139668278384448 run_docker.py:180] ans = rule(c, axis_env, extend_name_stack(name_stack, eqn.primitive.name),
I0716 21:59:58.937289 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/jax/_src/lax/control_flow.py", line 350, in _while_loop_translation_rule
I0716 21:59:58.937474 139668278384448 run_docker.py:180] new_z = xla.jaxpr_subcomp(body_c, body_jaxpr.jaxpr, backend, axis_env,
I0716 21:59:58.937648 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/jax/interpreters/xla.py", line 453, in jaxpr_subcomp
I0716 21:59:58.937821 139668278384448 run_docker.py:180] ans = rule(c, *in_nodes, **eqn.params)
I0716 21:59:58.937993 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/jax/_src/lax/linalg.py", line 503, in _eigh_cpu_gpu_translation_rule
I0716 21:59:58.938167 139668278384448 run_docker.py:180] v, w, info = syevd_impl(c, operand, lower=lower)
I0716 21:59:58.938340 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/jaxlib/cusolver.py", line 281, in syevd
I0716 21:59:58.938514 139668278384448 run_docker.py:180] lwork, opaque = cusolver_kernels.build_syevj_descriptor(
I0716 21:59:58.938688 139668278384448 run_docker.py:180] jax._src.traceback_util.UnfilteredStackTrace: RuntimeError: cuSolver internal error
I0716 21:59:58.938864 139668278384448 run_docker.py:180]
I0716 21:59:58.939038 139668278384448 run_docker.py:180] The stack trace below excludes JAX-internal frames.
I0716 21:59:58.939213 139668278384448 run_docker.py:180] The preceding is the original exception that occurred, unmodified.
I0716 21:59:58.939386 139668278384448 run_docker.py:180] --------------------
I0716 21:59:58.939731 139668278384448 run_docker.py:180]
I0716 21:59:58.939903 139668278384448 run_docker.py:180] The above exception was the direct cause of the following exception:
I0716 21:59:58.940074 139668278384448 run_docker.py:180]
I0716 21:59:58.940248 139668278384448 run_docker.py:180] Traceback (most recent call last):
I0716 21:59:58.940423 139668278384448 run_docker.py:180] File "/app/alphafold/run_alphafold.py", line 283, in <module>
I0716 21:59:58.940596 139668278384448 run_docker.py:180] app.run(main)
I0716 21:59:58.940770 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/absl/app.py", line 312, in run
I0716 21:59:58.940943 139668278384448 run_docker.py:180] _run_main(main, args)
I0716 21:59:58.941116 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/absl/app.py", line 258, in _run_main
I0716 21:59:58.941291 139668278384448 run_docker.py:180] sys.exit(main(argv))
I0716 21:59:58.941488 139668278384448 run_docker.py:180] File "/app/alphafold/run_alphafold.py", line 255, in main
I0716 21:59:58.941663 139668278384448 run_docker.py:180] predict_structure(
I0716 21:59:58.941836 139668278384448 run_docker.py:180] File "/app/alphafold/run_alphafold.py", line 137, in predict_structure
I0716 21:59:58.942028 139668278384448 run_docker.py:180] prediction_result = model_runner.predict(processed_feature_dict)
I0716 21:59:58.942201 139668278384448 run_docker.py:180] File "/app/alphafold/alphafold/model/model.py", line 134, in predict
I0716 21:59:58.942372 139668278384448 run_docker.py:180] result = self.apply(self.params, jax.random.PRNGKey(0), feat)
I0716 21:59:58.942544 139668278384448 run_docker.py:180] File "/opt/conda/lib/python3.8/site-packages/jaxlib/cusolver.py", line 281, in syevd
I0716 21:59:58.942715 139668278384448 run_docker.py:180] lwork, opaque = cusolver_kernels.build_syevj_descriptor(
I0716 21:59:58.942886 139668278384448 run_docker.py:180] RuntimeError: cuSolver internal error
I'm using a nvidia-gpu-cloud-image
on GCP with all the specifications given in the readme. When I run
docker run --rm --gpus all nvidia/cuda:11.0-base nvidia-smi
I see the expected output.
Building docker failed the first time, and when I tried restarting I get this error:
Sending build context to Docker daemon 12.69MB
Step 1/19 : ARG CUDA=11.0
Step 2/19 : FROM nvidia/cuda:${CUDA}-base
---> 2ec708416bb8
Step 3/19 : ARG CUDA
---> Using cache
---> 3cd6e5d7aef1
Step 4/19 : SHELL ["/bin/bash", "-c"]
---> Using cache
---> 657140758d4a
Step 5/19 : RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y build-essential cmake cuda-command-line-tools-${CUDA/./-} git hmmer kalign tzdata wget && rm -rf /var/lib/apt/lists/*
---> Using cache
---> d2d4f3239339
Step 6/19 : RUN git clone --branch v3.3.0 https://github.com/soedinglab/hh-suite.git /tmp/hh-suite && mkdir /tmp/hh-suite/build
---> Using cache
---> a080135b9366
Step 7/19 : WORKDIR /tmp/hh-suite/build
---> Using cache
---> 51cca5fbe9a6
Step 8/19 : RUN cmake -DCMAKE_INSTALL_PREFIX=/opt/hhsuite .. && make -j 4 && make install && ln -s /opt/hhsuite/bin/* /usr/bin && rm -rf /tmp/hh-suite
---> Using cache
---> ca9d782a9cf7
Step 9/19 : RUN wget -q -P /tmp https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh && bash /tmp/Miniconda3-latest-Linux-x86_64.sh -b -p /opt/conda && rm /tmp/Miniconda3-latest-Linux-x86_64.sh
---> Using cache
---> b0be57ee6667
Step 10/19 : ENV PATH="/opt/conda/bin:$PATH"
---> Using cache
---> bb0f01a8e0ce
Step 11/19 : RUN conda update -qy conda && conda install -y -c conda-forge openmm=7.5.1 cudatoolkit==${CUDA}.3 pdbfixer pip
---> Using cache
---> 5b8200c0a89d
Step 12/19 : COPY . /app/alphafold
---> Using cache
---> 88f827ee547a
Step 13/19 : RUN wget -q -P /app/alphafold/alphafold/common/ https://git.scicore.unibas.ch/schwede/openstructure/-/raw/7102c63615b64735c4941278d92b554ec94415f8/modules/mol/alg/src/stereo_chemical_props.txt
---> Using cache
---> e11c50f82963
Step 14/19 : RUN pip3 install --upgrade pip && pip3 install -r /app/alphafold/requirements.txt && pip3 install --upgrade jax jaxlib==0.1.69+cuda${CUDA/./} -f https://storage.googleapis.com/jax-releases/jax_releases.html
---> Using cache
---> bb6bb12e43ad
Step 15/19 : WORKDIR /opt/conda/lib/python3.8/site-packages
---> Using cache
---> 47ea4f7f278b
Step 16/19 : RUN patch -p0 < /app/alphafold/docker/openmm.patch
---> Running in ffae81eced58
can't find file to patch at input line 5
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|Index: simtk/openmm/app/topology.py
|===================================================================
|--- simtk.orig/openmm/app/topology.py
|+++ simtk/openmm/app/topology.py
--------------------------
File to patch:
Skip this patch? [y]
Skipping patch.
1 out of 1 hunk ignored
The command '/bin/bash -c patch -p0 < /app/alphafold/docker/openmm.patch' returned a non-zero code: 1
Any idea what may be going wrong?
Is it possible to use a subset of the genetic databases for running the model end to end for a particular protein sequence?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.