Comments (12)
leepc12
commented on July 21, 2024
To prevent any python package conflicts, please remove any python (e.g.
Anaconda Python or module-loaded python) from your $PATH.
You will need to modify your bash startup scripts ~/.bashrc or
~/.bash_profile).
Also add `unset PYTHONPATH` to it.
Thanks,
Jin
…On Wed, Jan 17, 2018 at 7:20 AM, PFRoux ***@***.***> wrote:
Hi there !
I followed all the very clear instructions you provide to install the
pipeline and everything went fine.
I am giving my very first try on some toy datasets and while mapping /
filtering / cross-correlation steps are running without any issue, I get
the following error at the step right after, which forces the pipeline to
exit :
= Done align()
00:00:37.633 Waiting for parallel 'chipseq.bds.20180117_161156_433_parallel_33' to finish. RunState: FINISHED
00:00:37.634 Waiting for parallel 'chipseq.bds.20180117_161156_433_parallel_32' to finish. RunState: FINISHED
00:00:41.753 Waiting for all 'parrallel' to finish.
00:00:41.754 Waiting for parallel 'chipseq.bds.20180117_161156_433_parallel_31' to finish. RunState: FINISHED
00:00:41.754 Waiting for parallel 'chipseq.bds.20180117_161156_433_parallel_30' to finish. RunState: FINISHED
00:00:41.755 Waiting for parallel 'chipseq.bds.20180117_161156_433_parallel_33' to finish. RunState: FINISHED
00:00:41.755 Waiting for parallel 'chipseq.bds.20180117_161156_433_parallel_32' to finish. RunState: FINISHED
00:00:42.504 Executioner factory: Creating new executioner type 'LOCAL'
00:00:42.668 ExecutionerLocal 'Local[34]': Queuing task: chipseq.bds.20180117_161156_433/task.postalign_bam.jsd.line_812.id_10
00:00:42.669 Waiting for all tasks to finish.
00:00:42.669 Wait: Waiting for task to finish: chipseq.bds.20180117_161156_433/task.postalign_bam.jsd.line_812.id_10, state: SCHEDULED
00:00:42.669 ExecutionerLocal 'Local[34]': Started running
00:00:42.670 ExecutionerLocal 'Local[34]': Task selected 'chipseq.bds.20180117_161156_433/task.postalign_bam.jsd.line_812.id_10' on host 'localhost'
Traceback (most recent call last):
File "/pasteur/homes/piroux/miniconda3/envs/aquas_chipseq/bin/plotFingerprint", line 4, in <module>
from deeptools.plotFingerprint import main
File "/pasteur/homes/piroux/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/deeptools/plotFingerprint.py", line 4, in <module>
import numpy as np
File "/mount/gensoft2/adm/lib/python2.7/site-packages/numpy-1.10.1-py2.7-linux-x86_64.egg/numpy/__init__.py", line 180, in <module>
from . import add_newdocs
File "/mount/gensoft2/adm/lib/python2.7/site-packages/numpy-1.10.1-py2.7-linux-x86_64.egg/numpy/add_newdocs.py", line 13, in <module>
from numpy.lib import add_newdoc
File "/mount/gensoft2/adm/lib/python2.7/site-packages/numpy-1.10.1-py2.7-linux-x86_64.egg/numpy/lib/__init__.py", line 8, in <module>
from .type_check import *
File "/mount/gensoft2/adm/lib/python2.7/site-packages/numpy-1.10.1-py2.7-linux-x86_64.egg/numpy/lib/type_check.py", line 11, in <module>
import numpy.core.numeric as _nx
File "/mount/gensoft2/adm/lib/python2.7/site-packages/numpy-1.10.1-py2.7-linux-x86_64.egg/numpy/core/__init__.py", line 14, in <module>
from . import multiarray
ImportError: /mount/gensoft2/adm/lib/python2.7/site-packages/numpy-1.10.1-py2.7-linux-x86_64.egg/numpy/core/multiarray.so: undefined symbol: PyUnicodeUCS2_AsASCIIString
Task failed:
Program & line : '/pasteur/homes/piroux/TF_chipseq_pipeline/modules/postalign_bam.bds', line 812
Task Name : 'jsd'
Task ID : 'chipseq.bds.20180117_161156_433/task.postalign_bam.jsd.line_812.id_10'
Task PID : '30942'
Task hint : 'plotFingerprint -b /pasteur/projets/policy01/Bischof-NGS/work/PROJET_SENESCENCE/ChIP-seq_HISTONE/RAS_OIS/TST_AQUA_2/./align/rep1/D0K4Me1_Rep1.nodup.bam /pasteur/projets/policy01/Bischof-NGS/work/PROJET_SENESCENCE/ChIP-seq_HISTONE/RAS_OIS/TST_AQUA_2/./align/rep2/D0K4Me1_Rep2.nodup.bam /pasteur/projet'
Task resources : 'cpus: 4 mem: -1.0 B wall-timeout: 8640000'
State : 'ERROR'
Dependency state : 'ERROR'
Retries available : '1'
Input files : '[/pasteur/projets/policy01/Bischof-NGS/work/PROJET_SENESCENCE/ChIP-seq_HISTONE/RAS_OIS/TST_AQUA_2/./align/rep1/D0K4Me1_Rep1.nodup.bam, /pasteur/projets/policy01/Bischof-NGS/work/PROJET_SENESCENCE/ChIP-seq_HISTONE/RAS_OIS/TST_AQUA_2/./align/rep2/D0K4Me1_Rep2.nodup.bam, /pasteur/projets/policy01/Bischof-NGS/work/PROJET_SENESCENCE/ChIP-seq_HISTONE/RAS_OIS/TST_AQUA_2/./align/ctl1/D0Input_Rep1.nodup.bam]'
Output files : '[/pasteur/projets/policy01/Bischof-NGS/work/PROJET_SENESCENCE/ChIP-seq_HISTONE/RAS_OIS/TST_AQUA_2/./qc/._jsd.png, /pasteur/projets/policy01/Bischof-NGS/work/PROJET_SENESCENCE/ChIP-seq_HISTONE/RAS_OIS/TST_AQUA_2/./qc/._jsd.qc]'
Script file : '/pasteur/projets/policy01/Bischof-NGS/work/PROJET_SENESCENCE/ChIP-seq_HISTONE/RAS_OIS/TST_AQUA_2/chipseq.bds.20180117_161156_433/task.postalign_bam.jsd.line_812.id_10.sh'
Exit status : '1'
Program :
# SYS command. line 813
if [[ -f $(which conda) && $(conda env list | grep aquas_chipseq | wc -l) != "0" ]]; then source activate aquas_chipseq; sleep 5; fi; export PATH=/pasteur/homes/piroux/TF_chipseq_pipeline/.:/pasteur/homes/piroux/TF_chipseq_pipeline/modules:/pasteur/homes/piroux/TF_chipseq_pipeline/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s)
# SYS command. line 814
plotFingerprint -b /pasteur/projets/policy01/Bischof-NGS/work/PROJET_SENESCENCE/ChIP-seq_HISTONE/RAS_OIS/TST_AQUA_2/./align/rep1/D0K4Me1_Rep1.nodup.bam /pasteur/projets/policy01/Bischof-NGS/work/PROJET_SENESCENCE/ChIP-seq_HISTONE/RAS_OIS/TST_AQUA_2/./align/rep2/D0K4Me1_Rep2.nodup.bam /pasteur/projets/policy01/Bischof-NGS/work/PROJET_SENESCENCE/ChIP-seq_HISTONE/RAS_OIS/TST_AQUA_2/./align/ctl1/D0Input_Rep1.nodup.bam --JSDsample /pasteur/projets/policy01/Bischof-NGS/work/PROJET_SENESCENCE/ChIP-seq_HISTONE/RAS_OIS/TST_AQUA_2/./align/ctl1/D0Input_Rep1.nodup.bam \
--labels rep1 rep2 ctl1 \
--outQualityMetrics /pasteur/projets/policy01/Bischof-NGS/work/PROJET_SENESCENCE/ChIP-seq_HISTONE/RAS_OIS/TST_AQUA_2/./qc/._jsd.qc \
--minMappingQuality 30 \
-T "Fingerprints of different samples" \
--blackListFileName /pasteur/homes/piroux/TF_chipseq_pipeline/genome_data/hg19/wgEncodeDacMapabilityConsensusExcludable.bed.gz \
--numberOfProcessors 4 \
--plotFile /pasteur/projets/policy01/Bischof-NGS/work/PROJET_SENESCENCE/ChIP-seq_HISTONE/RAS_OIS/TST_AQUA_2/./qc/._jsd.png
# SYS command. line 823
TASKTIME=$[$(date +%s)-${STARTTIME}]; echo "Task has finished (${TASKTIME} seconds)."; sleep 0
StdErr (100000000 lines) :
Traceback (most recent call last):
File "/pasteur/homes/piroux/miniconda3/envs/aquas_chipseq/bin/plotFingerprint", line 4, in <module>
from deeptools.plotFingerprint import main
File "/pasteur/homes/piroux/miniconda3/envs/aquas_chipseq/lib/python2.7/site-packages/deeptools/plotFingerprint.py", line 4, in <module>
import numpy as np
File "/mount/gensoft2/adm/lib/python2.7/site-packages/numpy-1.10.1-py2.7-linux-x86_64.egg/numpy/__init__.py", line 180, in <module>
from . import add_newdocs
File "/mount/gensoft2/adm/lib/python2.7/site-packages/numpy-1.10.1-py2.7-linux-x86_64.egg/numpy/add_newdocs.py", line 13, in <module>
from numpy.lib import add_newdoc
File "/mount/gensoft2/adm/lib/python2.7/site-packages/numpy-1.10.1-py2.7-linux-x86_64.egg/numpy/lib/__init__.py", line 8, in <module>
from .type_check import *
File "/mount/gensoft2/adm/lib/python2.7/site-packages/numpy-1.10.1-py2.7-linux-x86_64.egg/numpy/lib/type_check.py", line 11, in <module>
import numpy.core.numeric as _nx
File "/mount/gensoft2/adm/lib/python2.7/site-packages/numpy-1.10.1-py2.7-linux-x86_64.egg/numpy/core/__init__.py", line 14, in <module>
from . import multiarray
ImportError: /mount/gensoft2/adm/lib/python2.7/site-packages/numpy-1.10.1-py2.7-linux-x86_64.egg/numpy/core/multiarray.so: undefined symbol: PyUnicodeUCS2_AsASCIIString
00:01:00.967 Wait: Task 'chipseq.bds.20180117_161156_433/task.postalign_bam.jsd.line_812.id_10' finished.
00:01:00.968 Waiting for all 'parrallel' to finish.
00:01:00.968 Waiting for parallel 'chipseq.bds.20180117_161156_433_parallel_31' to finish. RunState: FINISHED
00:01:00.968 Waiting for parallel 'chipseq.bds.20180117_161156_433_parallel_30' to finish. RunState: FINISHED
00:01:00.968 Waiting for parallel 'chipseq.bds.20180117_161156_433_parallel_33' to finish. RunState: FINISHED
00:01:00.968 Waiting for parallel 'chipseq.bds.20180117_161156_433_parallel_32' to finish. RunState: FINISHED
Fatal error: /pasteur/homes/piroux/TF_chipseq_pipeline/chipseq.bds, line 410, pos 3. Task/s failed.
chipseq.bds, line 90 : main()
chipseq.bds, line 93 : void main() { // chipseq pipeline starts here
chipseq.bds, line 100 : jsd() // plot fingerprint and compute synthetic JS distance
chipseq.bds, line 384 : void jsd() { // plot fingerprint
chipseq.bds, line 385 : if ( filt_bam.hasKey("ctl1") && filt_bam.hasKey("rep1") ) {
chipseq.bds, line 410 : wait
Creating checkpoint file: Config or command line option disabled checkpoint file creation, nothing done.
00:01:01.076 Writing report file 'chipseq.bds.20180117_161156_433.report.html'
00:01:01.176 Program 'chipseq.bds.20180117_161156_433' finished, exit value: 1, tasks executed: 1, tasks failed: 1, tasks failed names: jsd.
00:01:01.176 Finished. Exit code: 1
00:01:01.177 ExecutionerLocal 'Local[34]': Killed
00:01:01.192 ExecutionerLocal 'Local[34]': Finished running
I assume it is somehow related to numpy library, but wasn't able to find
any useful thread on the internet. Plus, I must confess my understanding in
handling Python libraries is limited ...
Thx a lot for this awesome tool and for your previous help.
Pef
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#24>, or mute the
thread
<https://github.com/notifications/unsubscribe-auth/AIOd_CM7lZuwLDPj4BpW-R0BPQKrrXr1ks5tLg-hgaJpZM4RhgMg>
.
from chipseq_pipeline.
kdemuren
commented on July 21, 2024
I am also having this problem - the pipeline stops at the jsd step. I've done the above to no avail. Here's what the error looks like for me:
# SYS command. line 813
if [[ -f $(which conda) && $(conda env list | grep aquas_chipseq | wc -l) != "0" ]]; then source activate aquas_chipseq; sleep 5; fi; export PATH=/net/bmc-pub9/data/boyerlab/users/kdemuren/TF_chipseq_pipeline/.:/net/bmc-pub9/data/boyerlab/users/kdemuren/TF_chipseq_pipeline/modules:/net/bmc-pub9/data/boyerlab/users/kdemuren/TF_chipseq_pipeline/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s)
# SYS command. line 814
plotFingerprint -b /home/kdemuren/ChIP_pipeline/CM_CTCF/align/rep1/D10_CTCF_rep4.nodup.bam /home/kdemuren/ChIP_pipeline/CM_CTCF/align/rep2/D10_CTCF_rep5.nodup.bam /home/kdemuren/ChIP_pipeline/CM_CTCF/align/ctl1/D10_TF_WCE.nodup.bam --JSDsample /home/kdemuren/ChIP_pipeline/CM_CTCF/align/ctl1/D10_TF_WCE.nodup.bam \
--labels rep1 rep2 ctl1 \
--outQualityMetrics /home/kdemuren/ChIP_pipeline/CM_CTCF/qc/CM_CTCF_jsd.qc \
--minMappingQuality 30 \
-T "Fingerprints of different samples" \
--blackListFileName /net/bmc-pub9/data/boyerlab/users/kdemuren/genome_data/mm9/mm9-blacklist.bed.gz \
--numberOfProcessors 4 \
--plotFile /home/kdemuren/ChIP_pipeline/CM_CTCF/qc/CM_CTCF_jsd.png
# SYS command. line 823
TASKTIME=$[$(date +%s)-${STARTTIME}]; if [ ${TASKTIME} -lt 60 ]; then echo "Waiting for $[60-${TASKTIME}] seconds."; sleep $[60-${TASKTIME}]; sleep 0; fi
--------------------Post mortem info--------------------
==============================================================
job_number: 2951688
exec_file: job_scripts/2951688
submission_time: Fri Feb 16 11:32:35 2018
owner: kdemuren
uid: 103035
group: bmc-boyer-nfs
gid: 102080
sge_o_home: /home/kdemuren
sge_o_log_name: kdemuren
sge_o_path: /home/kdemuren/miniconda3/bin:/home/software/r/r-3.0.2/bin:/home/software/jre/jre-1.8.0-77/bin:/home/software/bowtie2/bowtie2-2.2.0/scripts:/home/software/bowtie2/bowtie2-2.2.0:/home/software/bedtools/bedtools-2.20.1/bin:/home/sge/sge6.2u5/bin/lx24-amd64:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/opt/454/bin:/home/kdemuren/weblogo/:/home/kdemuren/HOMER//bin/:/home/kdemuren/.bds:/home/kdemuren/bin
sge_o_shell: /bin/bash
sge_o_workdir: /net/bmc-pub9/data/boyerlab/users/kdemuren/TF_chipseq_pipeline
sge_o_host: rous
account: sge
stderr_path_list: NONE:NONE:/net/bmc-pub9/data/boyerlab/users/kdemuren/TF_chipseq_pipeline/chipseq.bds.20180216_113223_255/task.postalign_bam.jsd.line_812.id_10.stderr.cluster
hard resource_list: h_vmem=7516192768,h_rt=21120,s_rt=21120
mail_list: kdemuren@rous
notify: FALSE
job_name: STDIN
stdout_path_list: NONE:NONE:/net/bmc-pub9/data/boyerlab/users/kdemuren/TF_chipseq_pipeline/chipseq.bds.20180216_113223_255/task.postalign_bam.jsd.line_812.id_10.stdout.cluster
jobshare: 0
env_list: MANPATH=/home/software/r/r-3.0.2/share/man:/home/software/jre/jre-1.8.0-77/man:/home/sge/sge6.2u5/man:/usr/share/man:/usr/local/share/man,SHELL=/bin/bash,TERM=screen,SSH_CLIENT=18.189.17.249 64437 27687,LIBRARY_PATH=/home/software/jre/jre-1.8.0-77/lib,SGE_CELL=default,SSH_TTY=/dev/pts/11,USER=kdemuren,CPATH=/home/software/jre/jre-1.8.0-77/include/linux:/home/software/jre/jre-1.8.0-77/include,TERMCAP=SC|screen|VT 100/ANSI X3.64 virtual terminal:\,MAIL=/var/mail/kdemuren,PATH=/home/kdemuren/miniconda3/bin:/home/software/r/r-3.0.2/bin:/home/software/jre/jre-1.8.0-77/bin:/home/software/bowtie2/bowtie2-2.2.0/scripts:/home/software/bowtie2/bowtie2-2.2.0:/home/software/bedtools/bedtools-2.20.1/bin:/home/sge/sge6.2u5/bin/lx24-amd64:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/opt/454/bin:/home/kdemuren/weblogo/:/home/kdemuren/HOMER//bin/:/home/kdemuren/.bds:/home/kdemuren/bin,STY=4759.CM_CTCF.BDS,_=/home/kdemuren/.bds/bds,PWD=/home/kdemuren/TF_chipseq_pipeline,JAVA_HOME=/home/software/jre/jre-1.8.0-77,_LMFILES_=/home/software/modulefiles/bedtools/2.20.1:/home/software/modulefiles/bowtie2/2.2.0:/home/software/modulefiles/jre/1.8.0-77:/home/software/modulefiles/r/3.0.2,BOWTIE_INDEXES=/home/Genomes/bowtie_indexes,SGE_EXECD_PORT=6455,LANG=en_US.UTF-8,SGE_QMASTER_PORT=6454,MODULEPATH=/home/software/jre/jre-1.8.0-77/pkg/modulefiles:/home/software/modulefiles,SGE_ROOT=/home/sge/sge6.2u5,LOADEDMODULES=bedtools/2.20.1:bowtie2/2.2.0:jre/1.8.0-77:r/3.0.2,KRB5CCNAME=FILE:/tmp/krb5cc_103035_ej4568,SHLVL=3,HOME=/home/kdemuren,LOGNAME=kdemuren,WINDOW=0,SSH_CONNECTION=18.189.17.249 64437 18.159.2.88 27687,MODULESHOME=/usr/share/Modules,SGE_CLUSTER_NAME=koch1,NLSPATH=/usr/dt/lib/nls/msg/%L/%N.cat,XFILESEARCHPATH=/usr/dt/app-defaults/%L/Dt
script_file: STDIN
parallel environment: whole_nodes range: 4
usage 1: cpu=23:05:37, mem=75600.20397 GBs, io=223.74869, vmem=N/A, maxvmem=4.565G
scheduling info: queue instance "bcc-bmc.q@n10" dropped because it is disabled
queue instance "all.q@n2" dropped because it is full
queue instance "all.q@n3" dropped because it is full
queue instance "all.q@n4" dropped because it is full
from chipseq_pipeline.
leepc12
commented on July 21, 2024
Please remove any modules (module load/add) from your ~/.bashrc or
~/.bash_profile and re-login.
Also, remove any other python versions (Anaconda, ...) from $PATH too.
…On Mon, Feb 19, 2018 at 9:40 AM, kdemuren ***@***.***> wrote:
I am also having this problem - the pipeline stops at the jsd step. I've
done the above to no avail. Here's what the error looks like for me:
`# SYS command. line 813
if [[ -f $(which conda) && $(conda env list | grep aquas_chipseq | wc -l)
!= "0" ]]; then source activate aquas_chipseq; sleep 5; fi; export
PATH=/net/bmc-pub9/data/boyerlab/users/kdemuren/TF_
chipseq_pipeline/.:/net/bmc-pub9/data/boyerlab/users/
kdemuren/TF_chipseq_pipeline/modules:/net/bmc-pub9/data/
boyerlab/users/kdemuren/TF_chipseq_pipeline/utils:${PATH}
:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail;
STARTTIME=$(date +%s)
SYS command. line 814
plotFingerprint -b /home/kdemuren/ChIP_pipeline/
CM_CTCF/align/rep1/D10_CTCF_rep4.nodup.bam /home/kdemuren/ChIP_pipeline/
CM_CTCF/align/rep2/D10_CTCF_rep5.nodup.bam /home/kdemuren/ChIP_pipeline/
CM_CTCF/align/ctl1/D10_TF_WCE.nodup.bam --JSDsample
/home/kdemuren/ChIP_pipeline/CM_CTCF/align/ctl1/D10_TF_WCE.nodup.bam
--labels rep1 rep2 ctl1
--outQualityMetrics /home/kdemuren/ChIP_pipeline/CM_CTCF/qc/CM_CTCF_jsd.qc
--minMappingQuality 30
-T "Fingerprints of different samples"
--blackListFileName /net/bmc-pub9/data/boyerlab/
users/kdemuren/genome_data/mm9/mm9-blacklist.bed.gz
--numberOfProcessors 4
--plotFile /home/kdemuren/ChIP_pipeline/CM_CTCF/qc/CM_CTCF_jsd.png
SYS command. line 823
TASKTIME=$[$(date +%s)-${STARTTIME}]; if [ ${TASKTIME} -lt 60 ]; then echo
"Waiting for $[60-${TASKTIME}] seconds."; sleep $[60-${TASKTIME}]; sleep 0;
fi
--------------------Post mortem info--------------------
job_number: 2951688
exec_file: job_scripts/2951688
submission_time: Fri Feb 16 11:32:35 2018
owner: kdemuren
uid: 103035
group: bmc-boyer-nfs
gid: 102080
sge_o_home: /home/kdemuren
sge_o_log_name: kdemuren
sge_o_path: /home/kdemuren/miniconda3/bin:/home/software/r/r-3.0.2/bin:/
home/software/jre/jre-1.8.0-77/bin:/home/software/bowtie2/
bowtie2-2.2.0/scripts:/home/software/bowtie2/bowtie2-2.2.
0:/home/software/bedtools/bedtools-2.20.1/bin:/home/sge/
sge6.2u5/bin/lx24-amd64:/usr/local/sbin:/usr/local/bin:/
usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/opt/454/bin:/home/
kdemuren/weblogo/:/home/kdemuren/HOMER//bin/:/home/
kdemuren/.bds:/home/kdemuren/bin
sge_o_shell: /bin/bash
sge_o_workdir: /net/bmc-pub9/data/boyerlab/users/kdemuren/TF_chipseq_
pipeline
sge_o_host: rous
account: sge
stderr_path_list: NONE:NONE:/net/bmc-pub9/data/boyerlab/users/kdemuren/TF_
chipseq_pipeline/chipseq.bds.20180216_113223_255/task.
postalign_bam.jsd.line_812.id_10.stderr.cluster
hard resource_list: h_vmem=7516192768,h_rt=21120,s_rt=21120
mail_list: ***@***.***
notify: FALSE
job_name: STDIN
stdout_path_list: NONE:NONE:/net/bmc-pub9/data/boyerlab/users/kdemuren/TF_
chipseq_pipeline/chipseq.bds.20180216_113223_255/task.
postalign_bam.jsd.line_812.id_10.stdout.cluster
jobshare: 0
env_list: MANPATH=/home/software/r/r-3.0.2/share/man:/home/software/
jre/jre-1.8.0-77/man:/home/sge/sge6.2u5/man:/usr/share/
man:/usr/local/share/man,SHELL=/bin/bash,TERM=screen,SSH_CLIENT=18.189.17.249
64437 27687,LIBRARY_PATH=/home/software/jre/jre-1.8.0-77/lib,
SGE_CELL=default,SSH_TTY=/dev/pts/11,USER=kdemuren,CPATH=/
home/software/jre/jre-1.8.0-77/include/linux:/home/
software/jre/jre-1.8.0-77/include,TERMCAP=SC|screen|VT 100/ANSI X3.64
virtual terminal:,MAIL=/var/mail/kdemuren,PATH=/home/kdemuren/
miniconda3/bin:/home/software/r/r-3.0.2/bin:/home/software/
jre/jre-1.8.0-77/bin:/home/software/bowtie2/bowtie2-2.2.
0/scripts:/home/software/bowtie2/bowtie2-2.2.0:/home/
software/bedtools/bedtools-2.20.1/bin:/home/sge/sge6.2u5/
bin/lx24-amd64:/usr/local/sbin:/usr/local/bin:/usr/sbin:
/usr/bin:/sbin:/bin:/usr/games:/opt/454/bin:/home/kdemuren/weblogo/:/home/
kdemuren/HOMER//bin/:/home/kdemuren/.bds:/home/kdemuren/
bin,STY=4759.CM_CTCF.BDS,_=/home/kdemuren/.bds/bds,PWD=/
home/kdemuren/TF_chipseq_pipeline,JAVA_HOME=/home/
software/jre/jre-1.8.0-77,*LMFILES*=/home/software/
modulefiles/bedtools/2.20.1:/home/software/modulefiles/
bowtie2/2.2.0:/home/software/modulefiles/jre/1.8.0-77:/
home/software/modulefiles/r/3.0.2,BOWTIE_INDEXES=/home/
Genomes/bowtie_indexes,SGE_EXECD_PORT=6455,LANG=en_US.
UTF-8,SGE_QMASTER_PORT=6454,MODULEPATH=/home/software/jre/
jre-1.8.0-77/pkg/modulefiles:/home/software/modulefiles,SGE_
ROOT=/home/sge/sge6.2u5,LOADEDMODULES=bedtools/2.20.1:
bowtie2/2.2.0:jre/1.8.0-77:r/3.0.2,KRB5CCNAME=FILE:/tmp/
krb5cc_103035_ej4568,SHLVL=3,HOME=/home/kdemuren,LOGNAME=
kdemuren,WINDOW=0,SSH_CONNECTION=18.189.17.249 64437 18.159.2.88
27687,MODULESHOME=/usr/share/Modules,SGE_CLUSTER_NAME=
koch1,NLSPATH=/usr/dt/lib/nls/msg/%L/%N.cat,XFILESEARCHPATH=
/usr/dt/app-defaults/%L/Dt
script_file: STDIN
parallel environment: whole_nodes range: 4
usage 1: cpu=23:05:37, mem=75600.20397 GBs, io=223.74869, vmem=N/A,
maxvmem=4.565G
scheduling info: queue instance ***@***.***" dropped because it is
disabled
queue instance ***@***.***" dropped because it is full
queue instance ***@***.***" dropped because it is full
queue instance ***@***.***" dropped because it is full`
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#24 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AIOd_JOfgOyZydXQDc5p5XvGS-HEHkRAks5tWbIigaJpZM4RhgMg>
.
from chipseq_pipeline.
kdemuren
commented on July 21, 2024
I did those things, got the same error (when I removed anaconda from the PATH, the pipeline didn't start at all)
here's the longer error message from the log file, if that helps:
05:50:25.107 Writing report file 'chipseq.bds.20180221_105925_733.report.html'
05:50:32.045 ExecutionerClusterSge 'ClusterSge[34]': Error trying to find out post-mortem info on task (PID '2952081').
Command executed : ' qstat -f -j2952081'
Exit code : 1
Stdout :
Stderr : Following jobs do not exist:
2952081
Task failed:
Program & line : 'modules/postalign_bam.bds', line 812
Task Name : 'jsd'
Task ID : 'chipseq.bds.20180221_105925_733/task.postalign_bam.jsd.line_812.id_10'
Task PID : '2952081'
Task hint : 'plotFingerprint -b /home/kdemuren/ChIP_pipeline/CM_CTCF/align/rep1/D10_CTCF_rep4.nodup.bam /home/kdemuren/ChIP_pipeline/CM_CTCF/align/rep2/D10_CTCF_rep5.nodup.bam /home/kdemuren/ChIP_pipeline/CM_CTCF/align/ctl1/D10_TF_WCE.nodup.bam --JSDsample /home/kdemuren/ChIP_pipeline/CM_CTCF/align/ctl1/D10_TF_W'
Task resources : 'cpus: 4 mem: 7.0 GB timeout: 21000 wall-timeout: 8640000'
State : 'ERROR'
Dependency state : 'ERROR'
Retries available : '1'
Input files : '[/home/kdemuren/ChIP_pipeline/CM_CTCF/align/rep1/D10_CTCF_rep4.nodup.bam, /home/kdemuren/ChIP_pipeline/CM_CTCF/align/rep2/D10_CTCF_rep5.nodup.bam, /home/kdemuren/ChIP_pipeline/CM_CTCF/align/ctl1/D10_TF_WCE.nodup.bam]'
Output files : '[/home/kdemuren/ChIP_pipeline/CM_CTCF/qc/CM_CTCF_jsd.png, /home/kdemuren/ChIP_pipeline/CM_CTCF/qc/CM_CTCF_jsd.qc]'
Script file : '/net/bmc-pub9/data/boyerlab/users/kdemuren/TF_chipseq_pipeline/chipseq.bds.20180221_105925_733/task.postalign_bam.jsd.line_812.id_10.sh'
Exit status : '1'
Program :
# SYS command. line 813
if [[ -f $(which conda) && $(conda env list | grep aquas_chipseq | wc -l) != "0" ]]; then source activate aquas_chipseq; sleep 5; fi; export PATH=/net/bmc-pub9/data/boyerlab/users/kdemuren/TF_chipseq_pipeline/.:/net/bmc-pub9/data/boyerlab/users/kdemuren/TF_chipseq_pipeline/modules:/net/bmc-pub9/data/boyerlab/users/kdemuren/TF_chipseq_pipeline/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s)
# SYS command. line 814
plotFingerprint -b /home/kdemuren/ChIP_pipeline/CM_CTCF/align/rep1/D10_CTCF_rep4.nodup.bam /home/kdemuren/ChIP_pipeline/CM_CTCF/align/rep2/D10_CTCF_rep5.nodup.bam /home/kdemuren/ChIP_pipeline/CM_CTCF/align/ctl1/D10_TF_WCE.nodup.bam --JSDsample /home/kdemuren/ChIP_pipeline/CM_CTCF/align/ctl1/D10_TF_WCE.nodup.bam \
--labels rep1 rep2 ctl1 \
--outQualityMetrics /home/kdemuren/ChIP_pipeline/CM_CTCF/qc/CM_CTCF_jsd.qc \
--minMappingQuality 30 \
-T "Fingerprints of different samples" \
--blackListFileName /net/bmc-pub9/data/boyerlab/users/kdemuren/genome_data/mm9/mm9-blacklist.bed.gz \
--numberOfProcessors 4 \
--plotFile /home/kdemuren/ChIP_pipeline/CM_CTCF/qc/CM_CTCF_jsd.png
# SYS command. line 823
TASKTIME=$[$(date +%s)-${STARTTIME}]; if [ ${TASKTIME} -lt 60 ]; then echo "Waiting for $[60-${TASKTIME}] seconds."; sleep $[60-${TASKTIME}]; sleep 0; fi
05:50:32.123 Wait: Task 'chipseq.bds.20180221_105925_733/task.postalign_bam.jsd.line_812.id_10' finished.
05:50:32.123 Waiting for all 'parrallel' to finish.
05:50:32.123 Waiting for parallel 'chipseq.bds.20180221_105925_733_parallel_30' to finish. RunState: FINISHED
05:50:32.123 Waiting for parallel 'chipseq.bds.20180221_105925_733_parallel_31' to finish. RunState: FINISHED
05:50:32.123 Waiting for parallel 'chipseq.bds.20180221_105925_733_parallel_32' to finish. RunState: FINISHED
05:50:32.123 Waiting for parallel 'chipseq.bds.20180221_105925_733_parallel_33' to finish. RunState: FINISHED
Fatal error: /net/bmc-pub9/data/boyerlab/users/kdemuren/TF_chipseq_pipeline/chipseq.bds, line 414, pos 3. Task/s failed.
chipseq.bds, line 91 : main()
chipseq.bds, line 94 : void main() { // chipseq pipeline starts here
chipseq.bds, line 101 : jsd() // plot fingerprint and compute synthetic JS distance
chipseq.bds, line 387 : void jsd() { // plot fingerprint
chipseq.bds, line 389 : if ( filt_bam.hasKey("ctl1") && filt_bam.hasKey("rep1") ) {
chipseq.bds, line 414 : wait
Creating checkpoint file: Config or command line option disabled checkpoint file creation, nothing done.
05:50:32.218 Writing report file 'chipseq.bds.20180221_105925_733.report.html'
05:50:32.226 Program 'chipseq.bds.20180221_105925_733' finished, exit value: 1, tasks executed: 1, tasks failed: 1, tasks failed names: jsd.
05:50:32.226 Finished. Exit code: 1
05:50:32.233 ExecutionerClusterSge 'ClusterSge[34]': Killed
from chipseq_pipeline.
leepc12
commented on July 21, 2024
This error occurred because python libraries in
`/mount/gensoft2/adm/lib/python2.7/site-packages`
conflict with those in Conda environment. Please remove any dependencies
(PATH, PYTHONPATH, module add/load, ...) related to /mount/gensoft2/* from
your ~/.bashrc.
Thanks,
Jin
…On Thu, Feb 22, 2018 at 8:11 AM, kdemuren ***@***.***> wrote:
I did those things, got the same error (when I removed anaconda from the
PATH, the pipeline didn't start at all)
here's the longer error message from the log file, if that helps:
05:50:25.107 Writing report file 'chipseq.bds.20180221_105925_733.report.html'
05:50:32.045 ExecutionerClusterSge 'ClusterSge[34]': Error trying to find out post-mortem info on task (PID '2952081').
Command executed : ' qstat -f -j2952081'
Exit code : 1
Stdout :
Stderr : Following jobs do not exist:
2952081
Task failed:
Program & line : 'modules/postalign_bam.bds', line 812
Task Name : 'jsd'
Task ID : 'chipseq.bds.20180221_105925_733/task.postalign_bam.jsd.line_812.id_10'
Task PID : '2952081'
Task hint : 'plotFingerprint -b /home/kdemuren/ChIP_pipeline/CM_CTCF/align/rep1/D10_CTCF_rep4.nodup.bam /home/kdemuren/ChIP_pipeline/CM_CTCF/align/rep2/D10_CTCF_rep5.nodup.bam /home/kdemuren/ChIP_pipeline/CM_CTCF/align/ctl1/D10_TF_WCE.nodup.bam --JSDsample /home/kdemuren/ChIP_pipeline/CM_CTCF/align/ctl1/D10_TF_W'
Task resources : 'cpus: 4 mem: 7.0 GB timeout: 21000 wall-timeout: 8640000'
State : 'ERROR'
Dependency state : 'ERROR'
Retries available : '1'
Input files : '[/home/kdemuren/ChIP_pipeline/CM_CTCF/align/rep1/D10_CTCF_rep4.nodup.bam, /home/kdemuren/ChIP_pipeline/CM_CTCF/align/rep2/D10_CTCF_rep5.nodup.bam, /home/kdemuren/ChIP_pipeline/CM_CTCF/align/ctl1/D10_TF_WCE.nodup.bam]'
Output files : '[/home/kdemuren/ChIP_pipeline/CM_CTCF/qc/CM_CTCF_jsd.png, /home/kdemuren/ChIP_pipeline/CM_CTCF/qc/CM_CTCF_jsd.qc]'
Script file : '/net/bmc-pub9/data/boyerlab/users/kdemuren/TF_chipseq_pipeline/chipseq.bds.20180221_105925_733/task.postalign_bam.jsd.line_812.id_10.sh'
Exit status : '1'
Program :
# SYS command. line 813
if [[ -f $(which conda) && $(conda env list | grep aquas_chipseq | wc -l) != "0" ]]; then source activate aquas_chipseq; sleep 5; fi; export PATH=/net/bmc-pub9/data/boyerlab/users/kdemuren/TF_chipseq_pipeline/.:/net/bmc-pub9/data/boyerlab/users/kdemuren/TF_chipseq_pipeline/modules:/net/bmc-pub9/data/boyerlab/users/kdemuren/TF_chipseq_pipeline/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds; set -o pipefail; STARTTIME=$(date +%s)
# SYS command. line 814
plotFingerprint -b /home/kdemuren/ChIP_pipeline/CM_CTCF/align/rep1/D10_CTCF_rep4.nodup.bam /home/kdemuren/ChIP_pipeline/CM_CTCF/align/rep2/D10_CTCF_rep5.nodup.bam /home/kdemuren/ChIP_pipeline/CM_CTCF/align/ctl1/D10_TF_WCE.nodup.bam --JSDsample /home/kdemuren/ChIP_pipeline/CM_CTCF/align/ctl1/D10_TF_WCE.nodup.bam \
--labels rep1 rep2 ctl1 \
--outQualityMetrics /home/kdemuren/ChIP_pipeline/CM_CTCF/qc/CM_CTCF_jsd.qc \
--minMappingQuality 30 \
-T "Fingerprints of different samples" \
--blackListFileName /net/bmc-pub9/data/boyerlab/users/kdemuren/genome_data/mm9/mm9-blacklist.bed.gz \
--numberOfProcessors 4 \
--plotFile /home/kdemuren/ChIP_pipeline/CM_CTCF/qc/CM_CTCF_jsd.png
# SYS command. line 823
TASKTIME=$[$(date +%s)-${STARTTIME}]; if [ ${TASKTIME} -lt 60 ]; then echo "Waiting for $[60-${TASKTIME}] seconds."; sleep $[60-${TASKTIME}]; sleep 0; fi
05:50:32.123 Wait: Task 'chipseq.bds.20180221_105925_733/task.postalign_bam.jsd.line_812.id_10' finished.
05:50:32.123 Waiting for all 'parrallel' to finish.
05:50:32.123 Waiting for parallel 'chipseq.bds.20180221_105925_733_parallel_30' to finish. RunState: FINISHED
05:50:32.123 Waiting for parallel 'chipseq.bds.20180221_105925_733_parallel_31' to finish. RunState: FINISHED
05:50:32.123 Waiting for parallel 'chipseq.bds.20180221_105925_733_parallel_32' to finish. RunState: FINISHED
05:50:32.123 Waiting for parallel 'chipseq.bds.20180221_105925_733_parallel_33' to finish. RunState: FINISHED
Fatal error: /net/bmc-pub9/data/boyerlab/users/kdemuren/TF_chipseq_pipeline/chipseq.bds, line 414, pos 3. Task/s failed.
chipseq.bds, line 91 : main()
chipseq.bds, line 94 : void main() { // chipseq pipeline starts here
chipseq.bds, line 101 : jsd() // plot fingerprint and compute synthetic JS distance
chipseq.bds, line 387 : void jsd() { // plot fingerprint
chipseq.bds, line 389 : if ( filt_bam.hasKey("ctl1") && filt_bam.hasKey("rep1") ) {
chipseq.bds, line 414 : wait
Creating checkpoint file: Config or command line option disabled checkpoint file creation, nothing done.
05:50:32.218 Writing report file 'chipseq.bds.20180221_105925_733.report.html'
05:50:32.226 Program 'chipseq.bds.20180221_105925_733' finished, exit value: 1, tasks executed: 1, tasks failed: 1, tasks failed names: jsd.
05:50:32.226 Finished. Exit code: 1
05:50:32.233 ExecutionerClusterSge 'ClusterSge[34]': Killed
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#24 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AIOd_M-9aDWZ5X0sgngH0J8d0uIqs0ICks5tXZGrgaJpZM4RhgMg>
.
from chipseq_pipeline.
kdemuren
commented on July 21, 2024
I unloaded the python module, checked out my PATH and PYTHONPATH, and am still having the problem. There's nothing that says /mount/gensoft2 in any of those, so I'm not sure where that is coming from. Here's my .bashrc:
# .bashrc
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
# User specific aliases and functions
#export PYTHONPATH=$PYTHONPATH:$HOME/lib/python2.7/site-packages
#export PYTHONPATH=$PYTHONPATH:/home/software/python/python-3.5.1/lib/python3.5/site-packages
PATH=$PATH:/home/kdemuren/weblogo/
PATH=$PATH:/home/kdemuren/HOMER//bin/
export PATH=$PATH:$HOME/.bds
#module load bedtools/2.20.1
#module load bowtie2/2.2.0
module load jre/1.8.0-77
#module load r/3.0.2
#module load python/2.7.2
module unload python
unset module
`unset PYTHONPATH`
# added by Miniconda3 installer
export PATH="/home/kdemuren/miniconda3/bin:$PATH"
and here's my PATH (PYTHONPATH is empty):
/home/kdemuren/miniconda3/bin:/home/software/jre/jre-1.8.0-77/bin:/home/sge/sge6.2u5/bin/lx24-amd64:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/opt/454/bin:/home/kdemuren/weblogo/:/home/kdemuren/HOMER//bin/:/home/kdemuren/.bds:/home/kdemuren/bin
Thanks,
Kunle
from chipseq_pipeline.
leepc12
commented on July 21, 2024
Please add -no_jsd to the command line until this issue is fixed. I will take a closer look at this error later.
from chipseq_pipeline.
leepc12
commented on July 21, 2024
Also, your ~/.bashrc looks wrong unset PYTHONPATH.
from chipseq_pipeline.
kvittingseerup
commented on July 21, 2024
I just got the same error - is there an update on the problem?
from chipseq_pipeline.
leepc12
commented on July 21, 2024
@kvittingseerup did you remove all conflicting modules and python from your bash startup scripts? and did you add unset PYTHONPATH to it?
from chipseq_pipeline.
kvittingseerup
commented on July 21, 2024
I'll have a hard time removing python from my bashrc as I'm using it all the time. Can I specify it in the script(s) instead?
from chipseq_pipeline.
leepc12
commented on July 21, 2024
@kvittingseerup: No.
Please try unset PYTHONPATH first and then if it doesn't help go with -no_jsd.
from chipseq_pipeline.
Related Issues (20)
- aquas_chipseq pipline SPP error HOT 6
- Trimmed data-sets HOT 1
- Read length HOT 2
- shuf: end of file HOT 8
- "-q [SLURM_ACCOUNT_NAME]" interpreted as file HOT 1
- ERROR:root:--extsize must >= 1! HOT 2
- Fatal error: chipseq.bds, line 1354, pos 2. Task/s failed. HOT 4
- [fread] Unexpected end of file (bwa_sai2sam_pe_core) HOT 6
- dead loop HOT 18
- job submission command HOT 2
- [Error] two same tag align file in (z)cat and shuf in pipe HOT 1
- Missing `-c` for `macs2 peakcall` ? HOT 1
- pipeline deprecated? HOT 2
- Can't download hg19 genome data HOT 2
- Some reps get ERROR:root:--extsize must >= 1! HOT 2
- Documentation for macs2? HOT 2
- Citing the AQUAS Pipeline HOT 1
- ERROR:root:--extsize must >= 1! and chipseq.bds can not get all replicates go through HOT 25
- mm10 HOT 1
- JSD error: "RuntimeError: module compiled against API version 0xc but this version of numpy is 0xa" HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from chipseq_pipeline.