About the compilation and installation of rpvg software about rpvg HOT 24 CLOSED

Hi Du

Regarding your comment on issue #33 I'm afraid I didn't have the errors above when compiling so probably not the best person to help

Best Wishes
James

from rpvg.

Hi Du,

I have not experienced that particular compiling issue before. It seems to complain about a conflicting declaration in except.h. I found this issue which mentions the same problem: ContinuumIO/anaconda-issues#7047 Maybe that can help.

Regarding creating a binary. That is something that has been on my todo list for a while. I will try to see if I can get it done this week.

Best,

Jonas

from rpvg.

Hi Jonas,

Thanks for your reply, I solved this compiling issue, but I got new  error message like the following,It seems to be an error between ld cmake and make.

Besides, I‘m  looking forward to the release of the binaries.

Best,
Du

from rpvg.

Hi Du,

Just wanted to give a quick update. I am running into some problems creating a static binary, but I think I know how to solve it. I, however, do not have much more time to devote to it this week. In the meantime you can maybe use the Docker container. It should be working now.

Best,

Jonas

from rpvg.

Hi Jonas,
How is the static binary executable build coming along? The Docker container is not well suited for mass task commits on my server,Or is it possible to install rpvg by using conda?
Best,
Du

from rpvg.

Hi Du,

Sorry for the silence. I have unfortunately not had time to work on this. It is not possible to use conda.

If the Docker container is not practical then trying to get rpvg compiled on your system is probably our best bet for now. Could you try to compile rpvg again from a fresh repo and share the error you get? Thanks.

Best,

Jonas

from rpvg.

Hi Jonas,
Thanks for your reply,I will try to compile it again.
Best,
Du

from rpvg.

Hi Jonas,
Thank you in advance.
Do you have any suggestions for non-root users to install C++ software and its configuration on centos7 linux server?For example, how to install the rpvg software and dependencies in a clean environment for non-root users？

Besides,I still have some other confusion. I wanted to compile other C++ programs in vgrna-project（https://github.com/jonassibbesen/vgrna-project-scripts/tree/master/c++), but I'm got errors like the following,

It seems like the configure of libz cannon be found, but the locate command returns the following results,I reinstalled libz in a non-root state and set the LD_LIBRARY_PATH environment variable, but I still get the same error.

Best,
Du

from rpvg.

Hi Jonas,
I really need your help,Thank you in advance.

Do you have any suggestions about the following problem of not being able to find dynamic link libraries?

I'm looking forward to your reply.

Best,
Du

from rpvg.

Hi Du,

It looks like htslib is missing some dependencies. I would recommend taking a look at the Dockerfile and make sure the programs listed under "### Install htslib dependencies" are installed. Let me know if that does not solve it.

Best,

Jonas

from rpvg.

Hi Jonas,

Thank you for your patient reply.

All of the above errors stemmed from complex system environment variables, and I managed the installed software using a module to complete the compilation of rpvg in C++.

Also,I've been studying vgrna-project recently, and I'm confused about some of the steps involved.
step1: Selecting and extracting the cds allele,
(https://github.com/jonassibbesen/vgrna-project-paper/blob/main/originals/data/haplotypes/imgt_hla/pad_cds_alleles_10k_noB258.sh)
so the transcribed_unprocessed_pseudogenes are not considered?
....
stepn:Convert CDS alignments to haplotypes,
(https://github.com/jonassibbesen/vgrna-project-paper/blob/main/originals/benchmark/data/graphs/1kg_all_af001_imgt_hla_p10k_noB258_noN_a100_gencode100/add_haplotypes.sh)
So what is the difference between the output of program reference_pad_cds_alleles.py in step1 and convert_cds_alignments_to_haplotypes in stepn?Further filtering introns in overlapping regions of multiple genes?

I'm looking forward to your reply.
Best,
Du

from rpvg.

Hi Du,

Glad to hear that you got rpvg compiled.

Regarding the two scripts that you are referring to, note that they were only used to construct the pantranscriptome used for the HLA analyses presented in the paper. They are not needed for the regular expression analyses.

For the HLA analyses we used the CDS sequence alleles from the IPD-IMGT/HLA database (https://github.com/ANHIG/IMGTHLA) to create a HLA pantranscriptome. The pad_cds_alleles_10k_noB258.sh script adds reference flanks to these allele sequences so that they can accurately be mapped to the correct location on the spliced pangenome graph. The convert_cds_alignments_to_haplotypes script adds intron reference sequences to the mapped CDS alleles thereby producing haplotype sequences covering the whole gene. The haplotypes are needed when creating a pantranscriptome (set of haplotype-specific transcripts) using vg rna.

The pipeline for constructing the HLA pantranscriptome is described in more detail in the "HLA pantranscriptome construction and typing" section in the mpmap-rpvg manuscript (https://www.biorxiv.org/content/10.1101/2021.03.26.437240v2).

Let me know if you have any other questions.

Best,

Jonas

from rpvg.

Hi Jonas,

Thank you for your useful reply.

I want to use vg index -p -t 10 -k 16 -Z 4096 to build pantranscriptome's GCSA2 index, but I get the following error, any suggestions on this issue?

Best,
Du

from rpvg.

You likely need to prune the graph before running vg index. You can find more information here regarding pruning and indexing: https://github.com/vgteam/vg/wiki/Index-Construction

from rpvg.

Hi Jonas,
Thanks for your useful reply,I will try it again.
Best,
Du

from rpvg.

Hi Du and Jonas,

I encountered similar problems than Du when trying to compile with C++ on Ubuntu20.04 with gcc/9.3.0 and other required soft (cmake, opencv, and protobuf). The last lines are as following:

SUCCESS: sdsl was installed successfully!
The sdsl include files are located in '/work/FAC/FBM/DEE/mchapuis/default/lsoldini/rpvg/deps/sdsl-lite/include'.
The library files are located in '/work/FAC/FBM/DEE/mchapuis/default/lsoldini/rpvg/deps/sdsl-lite/lib'.

Sample programs can be found in the examples-directory.
A program 'example.cpp' can be compiled with the command:
g++ -std=c++14 -DNDEBUG -O3 [-msse4.2] \
   -I/work/FAC/FBM/DEE/mchapuis/default/lsoldini/rpvg/deps/sdsl-lite/include -L/work/FAC/FBM/DEE/mchapuis/default/lsoldini/rpvg/deps/sdsl-lite/lib \
   example.cpp -lsdsl -ldivsufsort -ldivsufsort64

Tests in the test-directory
A cheat sheet in the extras/cheatsheet-directory.
Have fun!
[ 23%] Completed 'sdsl-lite-proj'
[ 23%] Built target sdsl-lite-proj
make: *** [Makefile:101: all] Error 2

I do not think the above is very helpful for you Jonas, but if there is a way for me to provide something more for you to understand why some are encountering such issues, I would be glad to help.

George, you mentionned wanting to install on a cluster for non-root users. I could do so by using singularity on the Docker image with :

 singularity pull docker://quay.io/jsibbesen/rpvg

As far as I can tell, this seems to have worked properly - I am putting this in the hope it may help.

Best,
Luca

from rpvg.

Hi Luca,

Would it be possible to share the whole log from the compilation? That might be helpful, because I think the problem could be earlier than the lines you shared.

Thanks,

Jonas

from rpvg.

Hi Jonas,

Sorry for delayed answer. I actually was waiting on admin of the hpc environment I am using to see whether they could fix it, and they did. It was linked to missing libraries and some other fixes to do. But now the compilation has worked fine - i.e., I could also run rpvg on the Example dataset.

Thanks,
Luca

from rpvg.

Hi, good to hear you got it working. I am going to close this issue. Please reopen it if you run into more similar problems.

from rpvg.

Hi Luca and Jonas,

   Based on the latest image or source code, compiling with singularity, docker or C++ works fine.However,It is puzzling that rpvg works fine on some nodes of my server, but still reports errors on other nodes.I still have not solved this problem.

Best,
Du

from rpvg.

Hi Du,

I can tell that most of the errors I had while compiling rpvg and trying to make it work on the example dataset were linked to the hpc cluster architecture I am using and the fact that some libraries were missing (all the one I had to load: gcc opencv cmake htslib xz curl protobuf jansson) or environment variable not loaded properly/as expected. Sorry this is more of a general and not really useful answer.

Also it seems a warning comes from the .xg being out-of-date, maybe this is something to look into ?

Best,
Luca

from rpvg.

Hi Luca and Jonas,

Thank you for your reply.
I completed this command on the same node, but I got the error message on the other nodes. Besides, the xg file was downloaded from the website http://cgl.gi.ucsc.edu/data/vgrna/pantranscriptomes/. So I'm curious if it's an environmental error on some of the server nodes or a hardware error.
Any help would be great.

Best，
Du

from rpvg.

The warning about the XG version isn't very serious. It just means that the current version's encoding has to be computed from the old version's encoding. It slows the start-up by a few minutes, and you might notice temporarily increased memory usage, but that's all.

Regarding your error, @George-du, I suspect the issue is that some of the nodes in your clusters have older CPUs. If rpvg is compiled on a machine with a newer/higher-end processor, it might use machine instructions that don't exist on the older/cheaper processors. This is a compatibility problem that Docker doesn't really address.

One simple solution could be that you build rpvg from scratch as part of the job you submit to the node. That way you can be sure that the machine instructions match the CPU. The downside is that it's kind of a hassle, but you could start on that right away.

A longer-term solution might be for @jonassibbesen to add a -march flag to the compilation in the Dockerfile. In VG, we use -march=nehalem, which is almost universally portable these days. The downside to this solution is that rpvg will probably slow down on the newer CPUs, because it's not taking advantage of the newer instructions.

from rpvg.

Hi Jordan,
Thank you for your useful reply.

I will take your suggestion and looking forward to the new -march flag option for RPVG. Also, do you think a pre-built binary version will avoid this problem and be more convenient？

Best，
Du

from rpvg.