christensen-lab-dartmouth / methylnet Goto Github PK

View Code? Open in Web Editor NEW

48.0 48.0 11.0 144.51 MB

Modular framework for deep learning predictions on methylation data.

Home Page: https://christensen-lab-dartmouth.github.io/MethylNet/

License: MIT License

Python 0.91% Shell 0.01% Dockerfile 0.01% HTML 81.42% Jupyter Notebook 17.67% OpenEdge ABL 0.01%

methylnet's People

Stargazers

Watchers

Forkers

hmyh1202 gritcs adamritter shicheng-guo shumailasy hpg812 ktonosaki habush mattcwh harel-coffee williamxxu

methylnet's Issues

Thorough documentation

This is an important model contribution to both deep learning and epigenomics and we should make sure it is thoroughly documented. I’m happy to start creating issues and pull requests for updating documentation while the code is being debugged by @jlevy44

Unable to run MethylNet on clean Ubuntu

I am unable to run any MethylNet commands. I have tried on several different computers using conda and docker without success, with a range of different errors which appear to be dependency failures of one kind or another. Following is a brief summary of an attempt on an x86_64 PC with freshly-installed Ubuntu 20.04.3 LTS (Focal Fossa).

I was not able to install PyMethylProcess using the pip method; however I was able to run the example exercises using the docker container.

Error when installing via conda and pip

I installed MethylNet using conda as follows, noting changes from your instructions at https://github.com/Christensen-Lab-Dartmouth/MethylNet:

sudo apt install r-base-core    # R is a required dependency, pip install fails otherwise
conda create -n methylnet python=3.6
conda activate methylnet # "source activate" changed to "conda activate" for conda v4.11.0.
conda install pytorch torchvision -c pytorch
pip install methylnet

This install process appeared to run successfully, however running the example command line for methylnet-embed perform_embedding gives this error:

ModuleNotFoundError: No module named 'networkx'

Error when running the docker container

Running docker container joshualevy44/methylnet:0.1, I get this error running the example command line for methylnet-embed perform_embedding:

AssertionError: 
Found no NVIDIA driver on your system. Please check that you
have an NVIDIA GPU and installed a driver from

NVIDIA driver info:

cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module  495.29.05  Thu Sep 30 16:00:29 UTC 2021
GCC version:  gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)

I realize that standard docker does not support CUDA, but I do not understand the notes in your installation instructions about running with or without GPUs and was not able to resolve the problem.

I've used many hours on this now and am out of ideas, help will be much appreciated.

CVAE is missed?

Hi! Could you please help me to find CVAE class, which you used in this line:

MethylNet/methylnet/embedding.py

Line 67 in 674506b

    
           model = CVAE(n_latent=n_latent,in_shape=methyl_dataset.new_shape, kernel_heights=height_kernel_sizes, kernel_widths=width_kernel_sizes, n_pre_latent=n_latent*2) # change soon

Separate methylation processing pipeline

Wrapping the methylation pipeline in Python is a great contribution to the field in itself. We should consider cleaning that into its own repo and possibly submitting an application note to Bioinformatics or at a minimum bioRxiv to let the community use that as a separate resource.

Batch size correction to val loss in prediction module

Hyperparameter scan should take into account batch size for calculation of val loss, val loss should not be dependent on number samples

Add singularity support with nvidia gpu

Implementation of Wasserstein distance

Reparameterization on Test?

Should reparameterization of the VAE be done on test time? Same for VAE MLP?

May be okay.

Move methylnet_torque to new repo

Weighted BCELoss for VAE

Weight the BCE loss for VAE by the inverse of the probability distribution of the methylation states

Hyperspherical or specified VAE prior

Initialize prior that is not Gaussian for better results

Add custom hyperparameter search grid option for embeddings and predictions

Not able to perform tutorial

Dear master,
I try to use the docker version and I'm not able to perform the tutorial.
Any help on this:

pymethyl-preprocess download_geo -g GSE42861
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: trying URL 'https://ftp.ncbi.nlm.nih.gov/geo/series/GSE42nnn/GSE42861/suppl//GSE42861_RAW.tar?tool=geoquery'

  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: Content type 'application/x-tar'
  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning:  length 6166384640 bytes (5880.7 MB)

  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: =
  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: 

  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: downloaded 5880.7 MB


  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: trying URL 'https://ftp.ncbi.nlm.nih.gov/geo/series/GSE42nnn/GSE42861/suppl//GSE42861_Readme.txt?tool=geoquery'

  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: Content type 'text/plain'
  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning:  length 128 bytes

  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: downloaded 128 bytes


  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: trying URL 'https://ftp.ncbi.nlm.nih.gov/geo/series/GSE42nnn/GSE42861/suppl//GSE42861_methylation_signal_matrix.txt.gz?tool=geoquery'

  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: Content type 'application/x-gzip'
  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning:  length 2760138123 bytes (2632.3 MB)

  warnings.warn(x, RRuntimeWarning)


/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: downloaded 2632.3 MB


  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: trying URL 'https://ftp.ncbi.nlm.nih.gov/geo/series/GSE42nnn/GSE42861/suppl//GSE42861_methylation_signal_matrix_SUBSETS.tar.gz?tool=geoquery'

  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning:  length 2796169000 bytes (2666.6 MB)

  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: downloaded 2666.6 MB


  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: trying URL 'https://ftp.ncbi.nlm.nih.gov/geo/series/GSE42nnn/GSE42861/suppl//GSE42861_non-methylated_signal_matrix.txt.gz?tool=geoquery'

  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning:  length 2758508381 bytes (2630.7 MB)

  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: downloaded 2630.7 MB


  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: trying URL 'https://ftp.ncbi.nlm.nih.gov/geo/series/GSE42nnn/GSE42861/suppl//GSE42861_non-methylated_signal_matrix_SUBSETS.tar.gz?tool=geoquery'

  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning:  length 2794369077 bytes (2664.9 MB)

  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: downloaded 431.8 MB


  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: Error in download.file(paste(file.path(url, i), "tool=geoquery", sep = "?"),  : 
  download from 'https://ftp.ncbi.nlm.nih.gov/geo/series/GSE42nnn/GSE42861/suppl//GSE42861_non-methylated_signal_matrix_SUBSETS.tar.gz?tool=geoquery' failed

  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: In addition: 
  warnings.warn(x, RRuntimeWarning)
/usr/local/lib/python3.6/dist-packages/rpy2/rinterface/__init__.py:146: RRuntimeWarning: There were 27 warnings (use warnings() to see them)
  warnings.warn(x, RRuntimeWarning)
Traceback (most recent call last):
  File "/usr/local/bin/pymethyl-preprocess", line 10, in <module>
    sys.exit(preprocess())
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/dist-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/pymethylprocess/preprocess.py", line 45, in download_geo
    downloader.download_geo(geo_query,output_dir)
  File "/usr/local/lib/python3.6/dist-packages/pymethylprocess/PreProcessDataTypes.py", line 101, in download_geo
    geo.getGEOSuppFiles(query)
  File "/usr/local/lib/python3.6/dist-packages/rpy2/robjects/functions.py", line 178, in __call__
    return super(SignatureTranslatedFunction, self).__call__(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/rpy2/robjects/functions.py", line 106, in __call__
    res = super(Function, self).__call__(*new_args, **new_kwargs)
rpy2.rinterface.RRuntimeError: Error in download.file(paste(file.path(url, i), "tool=geoquery", sep = "?"),  : 
  download from 'https://ftp.ncbi.nlm.nih.gov/geo/series/GSE42nnn/GSE42861/suppl//GSE42861_non-methylated_signal_matrix_SUBSETS.tar.gz?tool=geoquery' failed

Deprecate vae.cuda_on for torch.cuda.is_available()

Access to train/val/test pkl files from the wiki?

Hello!

I have my own dataset that I'm trying to run through methylnet, but I'm running into issues. As a way to debug, I would love to run the train/val/test pkls used in the wiki (sample dataset GSE87571) for the age prediction analysis so I can ensure I have methylnet installed correctly.

Would you be able to post those pkls for download? I know these can be generated from PyMethylProcess, but I'm worried I might create bugs if I run it myself.

Thanks!

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.