drop demo problem in new version about drop HOT 15 CLOSED

Also some additional warnings from the drop demo:

Output created: /tmp/Rtmp9kLV6S/file577691a28c167/fraser_summary.html
Warning messages:
1: In plotEncDimSearch(fds, type = type) :
no hyperparameters were estimated for psi5
Please use optimHyperParams to compute them.
2: In plotEncDimSearch(fds, type = type) :
no hyperparameters were estimated for psi3
Please use optimHyperParams to compute them.
[1] TRUE TRUE TRUE TRUE

-------------------- BAM parameters --------------------
Default used with: bamMapqFilter=0

Warning message:
In min(p.adjust(pval, method = "holm")) :
no non-missing arguments to min; returning Inf

Output created: /tmp/Rtmpgik2y8/file565d579633d4b/fraser_countSummary.html
Warning messages:
1: Transformation introduced infinite values in continuous y-axis
2: Removed 380 rows containing missing values (geom_bar).
3: Removed 75 rows containing non-finite values (stat_bin).
4: Transformation introduced infinite values in continuous y-axis
5: Removed 195 rows containing missing values (geom_bar).

6.000e-08 6.000e-08 5.046e-03 2.947e-02 2.678e-02 1.775e-01
Thu Oct 1 14:25:02 2020: Writing final FRASER object ('/gpfs/scratch/evrong01/dropdemo/Output/processed_data/aberrant_splicing/datasets//savedObjects/fraser/fds-object.RDS').
Warning messages:
1: In injectOutliers(fds_copy, type = type, freq = injectFreq, minDpsi = minDeltaPsi, :
Injection-frequency is to low. Increasing it to 0.017 so we can inject at least 10 events into the data set!
2: In optimHyperParams(fds, type = type, implementation = implementation, :
No outliers could be injected so the hyperparameter optimization could not run. Possible reason: too few junctions in the data.
3: In injectOutliers(fds_copy, type = type, freq = injectFreq, minDpsi = minDeltaPsi, :
Injection-frequency is to low. Increasing it to 0.015 so we can inject at least 10 events into the data set!
4: In optimHyperParams(fds, type = type, implementation = implementation, :
No outliers could be injected so the hyperparameter optimization could not run. Possible reason: too few junctions in the data.
5: In injectOutliers(fds_copy, type = type, freq = injectFreq, minDpsi = minDeltaPsi, :
Injection-frequency is to low. Increasing it to 0.11 so we can inject at least 10 events into the data set!
Thu Oct 1 14:25:03 2020: Writing final FRASER object ('/gpfs/scratch/evrong01/dropdemo/Output/processed_data/aberrant_splicing/datasets//savedObjects/fraser/fds-object.RDS').
[1] TRUE

from drop.

Hi, Just checking if this is a known issue? Thanks.

from drop.

There seems to be an issue rendering the final readme for the aberrant splicing project. That means that all other steps of that pipeline are don and you can look at the files manually.

This is unexpected however, as the readme file should be copied to the directory when you call drop demo, drop init or drop update. Could you check that you can find the aberrant_splicing_readme.md file under .drop/modules etc.?

I suspect there might be an issue with updating the demo directory to the newest DROP version?

from drop.

Yes it is under: .drop/modules/aberrant-splicing-pipeline/aberrant_splicing_readme.md

However, I am still getting the error:

MissingInputException in line 158 of /tmp/tmpj6m1qz0w:
Missing input files for rule Index:
/gpfs/scratch/evrong01/dropdemo/Output/html/aberrant_splicing_readme.html
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /gpfs/scratch/evrong01/dropdemo/.drop/modules/aberrant-splicing-pipeline/.snakemake/log/2020-10-07T235241.889072.snakemake.log

from drop.

That is strange, seeing as the DROP code hasn't changed since the last version. Which version of wBuild do you have installed?

conda list | grep wbuild

from drop.

wbuild 1.7.0 py_0 bioconda

from drop.

Hm, not sure where this error is coming from.

We are pushing a new version of DROP that will be up on the dev branch in the next few days. I suggest you use that version once it's out, so we can work on bugfixes from there.

from drop.

How do I install from the dev branch using conda?

from drop.

We're still having issues with the pipeline due to Ensembl, so we haven't updated drop on conda yet. But installing it from github should work just fine. Just activate your old drop environment and manually update drop and wbuild.

conda activate drop_env
conda remove --force drop wbuild
conda install -c conda-forge -c bioconda wbuild=1.7.1
cd drop_repository
pip install -e .

drop --version

should give you 1.0.0

Also double-check that all R packages are installed

conda list

And then set up a new demo directory.

mkdir demo_dir
cd demo_dir
drop demo
snakemake -n

The last command should not start installing any packages, as they have been downloaded in conda.

from drop.

That doesn't work. wbuild 1.7.1 is installed, but I'm getting these errors. I will just wait for your final drop version.

ERROR: Exception:
Traceback (most recent call last):
File "/gpfs/share/apps/anaconda3/gpu/5.2.0/lib/python3.6/site-packages/pip/_internal/req/req_install.py", line 394, in check_if_exists
self.satisfied_by = pkg_resources.get_distribution(str(no_marker))
File "/gpfs/share/apps/anaconda3/gpu/5.2.0/lib/python3.6/site-packages/pip/_vendor/pkg_resources/init.py", line 481, in get_distribution
dist = get_provider(dist)
File "/gpfs/share/apps/anaconda3/gpu/5.2.0/lib/python3.6/site-packages/pip/_vendor/pkg_resources/init.py", line 357, in get_provider
return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
File "/gpfs/share/apps/anaconda3/gpu/5.2.0/lib/python3.6/site-packages/pip/_vendor/pkg_resources/init.py", line 900, in require
needed = self.resolve(parse_requirements(requirements))
File "/gpfs/share/apps/anaconda3/gpu/5.2.0/lib/python3.6/site-packages/pip/_vendor/pkg_resources/init.py", line 791, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pip._vendor.pkg_resources.ContextualVersionConflict: (wbuild 1.7.0 (/gpfs/home/evrong01/.local/lib/python3.6/site-packages), Requirement.parse('wbuild>=1.7.1'), {'drop'})

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/gpfs/share/apps/anaconda3/gpu/5.2.0/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 178, in main
status = self.run(options, args)
File "/gpfs/share/apps/anaconda3/gpu/5.2.0/lib/python3.6/site-packages/pip/_internal/commands/install.py", line 352, in run
resolver.resolve(requirement_set)
File "/gpfs/share/apps/anaconda3/gpu/5.2.0/lib/python3.6/site-packages/pip/_internal/resolve.py", line 131, in resolve
self._resolve_one(requirement_set, req)
File "/gpfs/share/apps/anaconda3/gpu/5.2.0/lib/python3.6/site-packages/pip/_internal/resolve.py", line 294, in _resolve_one
abstract_dist = self._get_abstract_dist_for(req_to_install)
File "/gpfs/share/apps/anaconda3/gpu/5.2.0/lib/python3.6/site-packages/pip/_internal/resolve.py", line 226, in _get_abstract_dist_for
req, self.require_hashes, self.use_user_site, self.finder,
File "/gpfs/share/apps/anaconda3/gpu/5.2.0/lib/python3.6/site-packages/pip/_internal/operations/prepare.py", line 399, in prepare_editable_requirement
req.check_if_exists(use_user_site)
File "/gpfs/share/apps/anaconda3/gpu/5.2.0/lib/python3.6/site-packages/pip/_internal/req/req_install.py", line 405, in check_if_exists
self.req.name
File "/gpfs/share/apps/anaconda3/gpu/5.2.0/lib/python3.6/site-packages/pip/_vendor/pkg_resources/init.py", line 481, in get_distribution
dist = get_provider(dist)
File "/gpfs/share/apps/anaconda3/gpu/5.2.0/lib/python3.6/site-packages/pip/_vendor/pkg_resources/init.py", line 357, in get_provider
return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0]
File "/gpfs/share/apps/anaconda3/gpu/5.2.0/lib/python3.6/site-packages/pip/_vendor/pkg_resources/init.py", line 900, in require
needed = self.resolve(parse_requirements(requirements))
File "/gpfs/share/apps/anaconda3/gpu/5.2.0/lib/python3.6/site-packages/pip/_vendor/pkg_resources/init.py", line 791, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pip._vendor.pkg_resources.ContextualVersionConflict: (wbuild 1.7.0 (/gpfs/home/evrong01/.local/lib/python3.6/site-packages), Requirement.parse('wbuild>=1.7.1'), {'drop'})

from drop.

Hm, it worked for me when conda wbuild is installed. Are you using the conda version of pip?

conda list | grep pip

Otherwise you can install wbuild directly from github

pip install git+https://github.com/gagneurlab/wBuild.git

from drop.

$ conda list | grep wbuild
wbuild 1.7.1 py_0 bioconda

$ conda list | grep pip
pip 20.2.3 py_0 conda-forge

It tried to reinstall pip and wbuild. Now I'm getting a different error:
$ pip install -e .
Obtaining file:///gpfs/scratch/evrong01/drop
ERROR: Could not find a version that satisfies the requirement wbuild>=1.7.1 (from drop==1.0.0) (from versions: 1.6.0.linux-x86_64, 1.7.0.linux-x86_64, 1.7.1.linux-x86_64, 0.1.0, 0.1.1, 1.0, 1.1.3, 1.1.4, 1.1.5, 1.2.0, 1.6.1, 1.6.4)
ERROR: No matching distribution found for wbuild>=1.7.1 (from drop==1.0.0)

from drop.

I think a docker approach is the best over the long-term. There are many dependencies and conda is generally too buggy to handle these sorts of things.

from drop.

Hi, Has the new version (1.0) been pushed to conda?

Conda is still showing me 0.9.1.

"The following NEW packages will be INSTALLED:

drop bioconda/noarch::drop-0.9.1-py_0"

from drop.

Hi, drop 1.0.0 still needs to be merged by someone from the bioconda community, so it should be available by tomorrow.
The current version however isn't 0.9.1 but 0.9.2 so make sure you specify the drop version when installing it. You also don't need to create a new environment, just update your old one, in case you don't already do so.

from drop.