christensen-lab-dartmouth / methylnet Goto Github PK
View Code? Open in Web Editor NEWModular framework for deep learning predictions on methylation data.
Home Page: https://christensen-lab-dartmouth.github.io/MethylNet/
License: MIT License
Modular framework for deep learning predictions on methylation data.
Home Page: https://christensen-lab-dartmouth.github.io/MethylNet/
License: MIT License
This is an important model contribution to both deep learning and epigenomics and we should make sure it is thoroughly documented. Iโm happy to start creating issues and pull requests for updating documentation while the code is being debugged by @jlevy44
I am unable to run any MethylNet
commands. I have tried on several different computers using conda
and docker
without success, with a range of different errors which appear to be dependency failures of one kind or another. Following is a brief summary of an attempt on an x86_64 PC with freshly-installed Ubuntu 20.04.3 LTS (Focal Fossa).
I was not able to install PyMethylProcess
using the pip
method; however I was able to run the example exercises using the docker
container.
I installed MethylNet
using conda
as follows, noting changes from your instructions at https://github.com/Christensen-Lab-Dartmouth/MethylNet:
sudo apt install r-base-core # R is a required dependency, pip install fails otherwise
conda create -n methylnet python=3.6
conda activate methylnet # "source activate" changed to "conda activate" for conda v4.11.0.
conda install pytorch torchvision -c pytorch
pip install methylnet
This install process appeared to run successfully, however running the example command line for methylnet-embed perform_embedding
gives this error:
ModuleNotFoundError: No module named 'networkx'
Running docker
container joshualevy44/methylnet:0.1
, I get this error running the example command line for methylnet-embed perform_embedding
:
AssertionError:
Found no NVIDIA driver on your system. Please check that you
have an NVIDIA GPU and installed a driver from
NVIDIA driver info:
cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module 495.29.05 Thu Sep 30 16:00:29 UTC 2021
GCC version: gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)
I realize that standard docker
does not support CUDA, but I do not understand the notes in your installation instructions about running with or without GPUs and was not able to resolve the problem.
I've used many hours on this now and am out of ideas, help will be much appreciated.
Hi! Could you please help me to find CVAE class, which you used in this line:
MethylNet/methylnet/embedding.py
Line 67 in 674506b
Wrapping the methylation pipeline in Python is a great contribution to the field in itself. We should consider cleaning that into its own repo and possibly submitting an application note to Bioinformatics or at a minimum bioRxiv to let the community use that as a separate resource.
Hyperparameter scan should take into account batch size for calculation of val loss, val loss should not be dependent on number samples
Should reparameterization of the VAE be done on test time? Same for VAE MLP?
May be okay.
Weight the BCE loss for VAE by the inverse of the probability distribution of the methylation states
Initialize prior that is not Gaussian for better results
Dear master,
I try to use the docker version and I'm not able to perform the tutorial.
Any help on this:
pymethyl-preprocess download_geo -g GSE42861
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: trying URL 'https://ftp.ncbi.nlm.nih.gov/geo/series/GSE42nnn/GSE42861/suppl//GSE42861_RAW.tar?tool=geoquery'
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: Content type 'application/x-tar'
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: length 6166384640 bytes (5880.7 MB)
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: =
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning:
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: downloaded 5880.7 MB
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: trying URL 'https://ftp.ncbi.nlm.nih.gov/geo/series/GSE42nnn/GSE42861/suppl//GSE42861_Readme.txt?tool=geoquery'
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: Content type 'text/plain'
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: length 128 bytes
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: downloaded 128 bytes
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: trying URL 'https://ftp.ncbi.nlm.nih.gov/geo/series/GSE42nnn/GSE42861/suppl//GSE42861_methylation_signal_matrix.txt.gz?tool=geoquery'
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: Content type 'application/x-gzip'
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: length 2760138123 bytes (2632.3 MB)
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: downloaded 2632.3 MB
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: trying URL 'https://ftp.ncbi.nlm.nih.gov/geo/series/GSE42nnn/GSE42861/suppl//GSE42861_methylation_signal_matrix_SUBSETS.tar.gz?tool=geoquery'
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: length 2796169000 bytes (2666.6 MB)
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: downloaded 2666.6 MB
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: trying URL 'https://ftp.ncbi.nlm.nih.gov/geo/series/GSE42nnn/GSE42861/suppl//GSE42861_non-methylated_signal_matrix.txt.gz?tool=geoquery'
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: length 2758508381 bytes (2630.7 MB)
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: downloaded 2630.7 MB
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: trying URL 'https://ftp.ncbi.nlm.nih.gov/geo/series/GSE42nnn/GSE42861/suppl//GSE42861_non-methylated_signal_matrix_SUBSETS.tar.gz?tool=geoquery'
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: length 2794369077 bytes (2664.9 MB)
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: downloaded 431.8 MB
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: Error in download.file(paste(file.path(url, i), "tool=geoquery", sep = "?"), :
download from 'https://ftp.ncbi.nlm.nih.gov/geo/series/GSE42nnn/GSE42861/suppl//GSE42861_non-methylated_signal_matrix_SUBSETS.tar.gz?tool=geoquery' failed
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: In addition:
warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: There were 27 warnings (use warnings() to see them)
warnings.warn(x, RRuntimeWarning)
Traceback (most recent call last):
File "/usr/local/bin/pymethyl-preprocess", line 10, in <module>
sys.exit(preprocess())
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/pymethylprocess/preprocess.py", line 45, in download_geo
downloader.download_geo(geo_query,output_dir)
File "/usr/local/lib/python3.6/dist-packages/pymethylprocess/PreProcessDataTypes.py", line 101, in download_geo
geo.getGEOSuppFiles(query)
File "/usr/local/lib/python3.6/dist-packages/rpy2/robjects/functions.py", line 178, in __call__
return super(SignatureTranslatedFunction, self).__call__(*args, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/rpy2/robjects/functions.py", line 106, in __call__
res = super(Function, self).__call__(*new_args, **new_kwargs)
rpy2.rinterface.RRuntimeError: Error in download.file(paste(file.path(url, i), "tool=geoquery", sep = "?"), :
download from 'https://ftp.ncbi.nlm.nih.gov/geo/series/GSE42nnn/GSE42861/suppl//GSE42861_non-methylated_signal_matrix_SUBSETS.tar.gz?tool=geoquery' failed
Hello!
I have my own dataset that I'm trying to run through methylnet, but I'm running into issues. As a way to debug, I would love to run the train/val/test pkls used in the wiki (sample dataset GSE87571) for the age prediction analysis so I can ensure I have methylnet installed correctly.
Would you be able to post those pkls for download? I know these can be generated from PyMethylProcess, but I'm worried I might create bugs if I run it myself.
Thanks!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.