gatk bug in mae pipeline about drop HOT 12 CLOSED

I think the mae code has a bug that doesn't use the 'gatk' path configured in config.yaml

from drop.

Hi, Is there any update about this? Thanks.

from drop.

There is indeed a bug, so this feature isn't working right now. In order for you to get things working quickly, I'd recommend you to make gatk available via gatk on the commandline. For example by setting an alias or modifying your PATH variable.

from drop.

I'm now getting a new error. When will the new major version of DROP be ready, in case that will fix these issues?

6_64 amd64; Java HotSpot(TM) 64-Bit Server VM 1.8.0_181-b13; Deflater: Intel; Inflater: Intel; Provider GCS is available; Picard version: Version:4.1.4.0
[Sun Jul 26 23:57:20 EDT 2020] picard.sam.CreateSequenceDictionary done. Elapsed time: 0.26 minutes.
Runtime.totalMemory()=2907701248
Tool returned:
0
Using GATK jar /gpfs/data/evronylab/bin/gatk/gatk-package-4.1.4.0-local.jar
Running:
java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /gpfs/data/evronylab/bin/gatk/gatk-package-4.1.4.0-local.jar CreateSequenceDictionary --REFERENCE /gpfs/data/evronylab/reference-files/GRCh38_gencode-STAR/GRCh38.primary_assembly.genome.fa
Waiting at most 5 seconds for missing files.
MissingOutputException in line 56 of /gpfs/scratch/evrong01/droptest/.drop/modules/mae-pipeline/Snakefile:
Missing files after 5 seconds:
/gpfs/data/evronylab/reference-files/GRCh38_gencode-STAR/GRCh38.dict
This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait.
Error in eval(jsub, SDenv, parent.frame()) :
object 'gene_status' not found
Calls: [ -> [.data.table -> eval -> eval
Execution halted
[Sun Jul 26 23:59:09 2020]
Error in rule Scripts_MAE_gene_name_mapping_R:
jobid: 14
output: /gpfs/scratch/evrong01/droptest/root/processed_data/mae/gene_name_mapping_v32.tsv

from drop.

We have now made drop available on bioconda. It would suggest you create a new environment (e.g. drop_env or any other name that hasn't been used before) and install bioconda drop inside it. This will take care of all the dependencies, such as R environment and GATK. The only prerequisite is conda, which I am assuming you are already using.
We are still having some issues with installation from bioconda, but things have worked consistently with a predefined python 3.7 environment.

Edit I figured out what the issue was and it's not because of the python version, so you don't have to specify it

Make sure that channel priority is set to flexible by running

conda config --set channel_priority true

Assuming this new environment is called drop_env:

conda create -n drop_env -c conda-forge -c bioconda drop

These commands have worked on my machine consistently. Let me know if conda stalls and gives you any unsatisfiability errors.

Once drop is successfully installed, you need to update the changes in your project directory. For that run from within the project directory.

drop update

from drop.

It doesn't seem to be working:

(drop_env) [evrong01@bigpurple-ln3 ~]$ conda install -c bioconda drop
Solving environment: \
failed

CondaValueError: Malformed version string '~': invalid character(s).

from drop.

Could you show which conda version you are using (conda --version)?
Try using the channel conda-forge and create a new environment that you haven't yet used to avoid incompatibilities.

conda create -n drop_env -c conda-forge -c bioconda drop

from drop.

I tried a newer version of conda (4.7.12).
Now it is installing, but it has been running the 'Executing transaction' step for 3 hours. Is that expected?

from drop.

Ok the installation finished after 5 hours. Now it seems to be ok.

from drop.

Does the conda version get updated at the same time as the github version?

from drop.

Yes, from now on we will tag stable releases on github and these will be recompiled in bioconda with a slight delay. It should then be easier to reproduce and track bugs

from drop.

Ok the installation finished after 5 hours. Now it seems to be ok.

Hm, strange that it took that long, but glad it worked in the end. Conda 4.7 is known to have some issues regarding package compatibility, that could be a reason. It might be worth a try to use conda 4.8 for the next drop version to speed up installation time.

from drop.