Hi,

@amcpherson Thans for developing this tool!

In my result, I got 6 clones in /trees/{tree_solution}/clone_freq, but only got 5 clones in /trees/{tree_solution}/cluster_assignment/.

The content of clone_freq is:

0.618809         0.0            0.0             0.0173867       0.0970734       0.266731
0.0484576       0.0809986       0.104325        0.112758        0.142484         0.510976

So there are 6 clones. Am I right?

The content of cluster_assignment is:

index      value
0            5
1            4
2            3
3            2
4            1
5            4

My guess is that the numbers in "value" are clone IDs. If it's the case, there are only 5 clones which is fewer than 6 as indicated by clone_freq.

Could you please help me out?

Thank you in advance.

Hi,
Could you clarify which solution does {tree_solution} in the results store correspond to: tree_id or tree_index?

Many thanks,
Maitena.

Hello,
I installed citup following README instructions. But it installed and old version:

conda install citup
conda list citup
# packages in environment at /c4/home/jocostello/miniconda2:
#
# Name                    Version                   Build  Channel
citup                     0.1.0                    py27_1    dranew

Then I tried to update it to the latest version

conda update -c dranew citup
...
==> WARNING: A newer version of conda exists. <==
  current version: 4.8.4
  latest version: 4.10.0
Please update conda by running
    $ conda update -n base -c defaults conda
# All requested packages already installed.

The citup version didn't change. How to install/update citup to the latest version (0.1.2 ?)

The reason I am trying to update it is that the old version crushes while running

run_citup_iter.py --submit local input.tsv out.h5
...
pypeliner.scheduler.PipelineException: pipeline failed

I am hoping the latest version will work.

Thanks!

Hello Andrew @amcpherson !

Thanks for developing citup!
I cannot run the default example:

(citup) $ run_citup_iter.py test.txt test.h5 --interactive --max_nodes 1
min_nodes: 1, max_nodes: 1
Traceback (most recent call last):
  File "/home/user/.conda/envs/citup/bin/run_citup_iter.py", line 4, in <module>
    __import__('pkg_resources').run_script('citup==0.1.0', 'run_citup_iter.py')
  File "/home/user/.conda/envs/citup/lib/python2.7/site-packages/pkg_resources/__init__.py", line 666, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/home/user/.conda/envs/citup/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1469, in run_script
    exec(script_code, namespace, namespace)
  File "/home/user/.conda/envs/citup/lib/python2.7/site-packages/citup-0.1.0-py2.7.egg/EGG-INFO/scripts/run_citup_iter.py", line 45, in <module>
    
  File "/home/user/.conda/envs/citup/lib/python2.7/site-packages/pypeliner/app.py", line 214, in __init__
    config_filename=self.config['submit_config'])
  File "/home/user/.conda/envs/citup/lib/python2.7/site-packages/pypeliner/execqueue/factory.py", line 6, in create
    raise Exception('No submit queue specified')
Exception: No submit queue specified
(citup)

Is it possible to run it in the current shell not sending jobs to the scheduler?

Thanks!
Sergey

This looks like a useful tool but the CPLEX dependency makes this a non-starter for us even as a public research institute because we're a non-degree granting organization and ineligible for the academic license. An option to use an unencumbered optimizer (GLPK, CLP etc) would be very welcome.
https://www.osti.gov/biblio/1104761

Hello, everyone. Can someone help me?
I installed citup according to the tutorial. When I run run_ citup_ qip. Py - H, everything is normal, but when I run the sample data, an error occurs.
Here is my script:
run_ citup_ qip. py ./freq. txt ./clusters. txt ./test. h5 --submit local
The following is the error message:
min_nodes: 1, max_nodes: 3
2022-08-18 21:42:52,283 - pypeliner.scheduler - ERROR - job /select_optimal_tree failed to complete
--- stdout ---
--- stderr ---
Traceback (most recent call last):
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/pypeliner/jobs.py", line 286, in call
self.ret_value = self.func(*self.callset.args, **self.callset.kwargs)
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/citup-0.1.0-py2.7.egg/citup/tasks.py", line 53, in select_optimal_tree
error_rate = estimate_error_rate(freq)
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/citup-0.1.0-py2.7.egg/citup/tasks.py", line 41, in estimate_error_rate
gmm = sklearn.mixture.GMM(n_components=n, covariance_type='spherical')
AttributeError: 'module' object has no attribute 'GMM'
2022-08-18 21:42:52,285 - pypeliner.scheduler - ERROR - exception
Traceback (most recent call last):
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/pypeliner/scheduler.py", line 79, in run
self._wait_next_job(exec_queue, workflow)
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/pypeliner/scheduler.py", line 165, in _wait_next_job
if self._retry_job(exec_queue, job):
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/pypeliner/scheduler.py", line 118, in _retry_job
self._add_job(exec_queue, job)
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/pypeliner/scheduler.py", line 100, in _add_job
sent = job.create_callable()
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/pypeliner/jobs.py", line 185, in create_callable
return JobCallable(self.id, self.job_def.func, self.argset, self.arglist, self.db.file_storage, self.logs_dir)
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/pypeliner/jobs.py", line 248, in init
self.callset = pypeliner.deep.deeptransform(self.argset, resolve_arg)
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/pypeliner/deep.py", line 63, in deeptransform
y = _reconstruct(x, f, rv, 1, memo)
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/pypeliner/deep.py", line 205, in _reconstruct
state = deeptransform(state, f, memo)
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/pypeliner/deep.py", line 36, in deeptransform
y = copier(x, f, memo)
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/pypeliner/deep.py", line 130, in _deeptransform_dict
y[deeptransform(key, f, memo)] = deeptransform(value, f, memo)
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/pypeliner/deep.py", line 36, in deeptransform
y = copier(x, f, memo)
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/pypeliner/deep.py", line 110, in _deeptransform_tuple
y.append(deeptransform(a, f, memo))
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/pypeliner/deep.py", line 32, in deeptransform
y, transformed = f(x)
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/pypeliner/jobs.py", line 230, in resolve_arg
return arg.resolve(), True
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/pypeliner/arguments.py", line 279, in resolve
if self.func is not None:
AttributeError: 'TempMergeObjArg' object has no attribute 'func'
2022-08-18 21:42:52,285 - pypeliner.scheduler - ERROR - pipeline failed
log file: ./tmp/log/20220818-214246/pipeline.log
Traceback (most recent call last):
File "/workpath/anaconda/envs/citup_new/bin/run_citup_qip.py", line 4, in
import('pkg_resources').run_script('citup==0.1.0', 'run_citup_qip.py')
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/pkg_resources/init.py", line 666, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/pkg_resources/init.py", line 1460, in run_script
exec(script_code, namespace, namespace)
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/citup-0.1.0-py2.7.egg/EGG-INFO/scripts/run_citup_qip.py", line 87, in

File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/pypeliner/app.py", line 221, in run
self.sch.run(workflow, self.exec_queue, self.file_storage, self.runskip)
File "/workpath/anaconda/envs/citup_new/lib/python2.7/site-packages/pypeliner/scheduler.py", line 97, in run
raise PipelineException('pipeline failed')
pypeliner.scheduler.PipelineException: pipeline failed
The following are the environmental parameters:

Name Version Build Channel

_libgcc_mutex 0.1 main https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
blas 1.0 mkl https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free
blosc 1.15.0 hd408876_0 defaults
boost_source 1.60.0 0 dranew
bzip2 1.0.6 3 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free
ca-certificates 2019.1.23 0 defaults
certifi 2019.3.9 py27_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
citup 0.1.1 py27h01be30a_1 dranew
cplex 12.8 py27_0 IBMDecisionOptimization
decorator 4.4.0 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
hdf5 1.10.4 nompi_h3c11f04_1106 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
intel-openmp 2019.3 199 defaults
libblas 3.8.0 14_mkl https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libcblas 3.8.0 14_mkl https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libedit 3.1.20181209 hc058e9b_0 defaults
libffi 3.2.1 1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free
libgcc-ng 8.2.0 hdf63c60_1 defaults
libgfortran-ng 7.3.0 hdf63c60_5 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
liblapack 3.8.0 14_mkl https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
libstdcxx-ng 8.2.0 hdf63c60_1 defaults
lzo 2.10 h516909a_1000 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
mkl 2019.3 199 defaults
mkl_fft 1.0.12 py27h516909a_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
mkl_random 1.0.2 py27h637b7d7_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
ncurses 6.1 hf484d3e_1002 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
networkx 2.2 py_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
numexpr 2.6.9 py27h637b7d7_1000 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
numpy 1.16.3 py27h7e9f1db_0 defaults
numpy-base 1.16.3 py27hde5b4d6_0 defaults
openssl 1.1.1b h14c3975_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pandas 0.24.2 py27hb3f55d8_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pip 19.1.1 py27_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pypeliner 0.5.0 py27h1453be2_0 dranew
pytables 3.5.1 py27h71ec239_0 defaults
python 2.7.16 h9bab390_0 defaults
python-dateutil 2.8.0 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
pytz 2019.1 py_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
readline 7.0 hf8c457e_1001 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
scikit-learn 0.20.3 py27ha8026db_1 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
scipy 1.2.1 py27h921218d_2 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
setuptools 41.0.1 py27_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
six 1.12.0 py27_1001 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
snappy 1.1.7 he1b5a44_1003 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
sqlite 3.28.0 h8b20d00_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
tk 8.6.8 h84994c4_1000 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
wheel 0.33.2 py27_0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
zlib 1.2.11 0 https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free
Any help will be greatly appreciated!!