LTR_retriever aborts, Illegal character error. about ltr_retriever HOT 13 CLOSED

Hi,
Yes, the package came with a carefully selected pHMM file. You can provide a new one but need further modification into the script. So I suggest you use the TEfam.hmm unless you have a good reason to use others.
I have pushed a new release to GitHub, please use this version of LTR_retriever. It comes with more user-friendly status messages which can help to keep track of the program. Just save your paths file and replace everything else. Let me know if you still have difficulties to run it.

Shujun

from ltr_retriever.

Hi, I moved your last comment to a new thread because this is a different issue.

Shujun

from ltr_retriever.

Some additional info:

Some nonempty files were generated.

• ~50% of the input elements ended up in the *defalse file
• ~3% input elements are in the *pass.list files
• the *fa files are empty

from ltr_retriever.

Hi,
Thanks for using LTR_retriever and providing the above messages.
The parse failure is reported by hmmsearch. You can use -v to retain the intermediate files. Please send me the Salvinia_cucullata_v1.1.fa.retriever.scn.extend.fa.aa if possible.
If you see a lot of empty files, something is wrong. i.e., there is not a lot of LTR founded in the genome.

Best,
Shujun

from ltr_retriever.

Hey Shujun, I think I found the problem. For -TEhmm I was using the Dfam profile HMM database of repeats, which has a lot of nice profiles, but they are nucleotide profile HMMs. I am re-running again now with a few Pfam profiles.

from ltr_retriever.

Ok it still aborted without completing I think. This time I used the call:

LTR_retriever \
	-genome Salvinia_cucullata_v1.1.fa \
	-inharvest input/LTRHarvest.out \
	-linelib input/LINEs.viridiplantae.fa \
	-dnalib input/DNA_TEs.viridiplantae.fa \
	-TEhmm /home/derstudent/data/other/pfam-hmms/retrotransposon/combined.hmm \
	-threads 40 \
	-v \
	1>LTR_retriever.out
	2>LTR_retriever.err

and got the error:

Attempt to free unreferenced scalar: SV 0x1a6a310, Perl interpreter: 0x1a667e0.
Use of uninitialized value $list[0] in pattern match (m//) at /home/joshd/software/LTR_retriever/bin/call_seq_by_list.pl line 76.
Use of uninitialized value in split at /home/joshd/software/LTR_retriever/bin/call_seq_by_list.pl line 79.
Use of uninitialized value in pattern match (m//) at /home/joshd/software/LTR_retriever/bin/call_seq_by_list.pl line 79.
Use of uninitialized value $chr_pre in hash element at /home/joshd/software/LTR_retriever/bin/call_seq_by_list.pl line 81.
Use of uninitialized value within %genome in length at /home/joshd/software/LTR_retriever/bin/call_seq_by_list.pl line 81.
Use of uninitialized value $list[0] in pattern match (m//) at /home/joshd/software/LTR_retriever/bin/call_seq_by_list.pl line 76.
Use of uninitialized value in split at /home/joshd/software/LTR_retriever/bin/call_seq_by_list.pl line 79.
Use of uninitialized value in pattern match (m//) at /home/joshd/software/LTR_retriever/bin/call_seq_by_list.pl line 79.
Use of uninitialized value $chr_pre in hash element at /home/joshd/software/LTR_retriever/bin/call_seq_by_list.pl line 81.
Use of uninitialized value within %genome in length at /home/joshd/software/LTR_retriever/bin/call_seq_by_list.pl line 81.
No such file or directory at /home/joshd/software/LTR_retriever/bin/cleanup.pl line 50.
LOC list is empty.
Warning: [blastx] Query is Empty!
LOC list is empty.
No such file or directory at /home/joshd/software/LTR_retriever/bin/cleanup.pl line 50.
perl annotate_gff.pl lib.fa gff > anno.gff

from ltr_retriever.

Hi,

It's not suggestive to replace the TE.hmm file because they are hand-categorized. If you want to use Dfam you need to know which Dfam entry belongs to LTR which are not, and change the annotate_TE.pl script accordingly.
I am updating the program and hopefully it will provide more status messages for users to learn about the process.

Shujun

from ltr_retriever.

Ok thank you! The source of the illegal character error seems to be Dfam.hmm. I don't think I can use any Dfam profile HMMs because they don't have transition probabilities for amino acids. Unless there is a different version of the Dfam database that has protein profile HMMs.

Edit: I didn't realize LTR_retriever came with a profile HMM database (LTR_retriever/database/TEfam.hmm).

from ltr_retriever.

Hi Shujun,

The new release seems to not be recognizing the location of the external dependencies. I think it is this line, 193, but I am not fluent in perl and don't know what the unless -X "string" test does. I added some code and made sure that makeblastdb is in the path held by $blastplus and it is, and makeblastdb can execute from within the code. Do you know the problem? (and could you explain unless -X if you have a minute :-)

die "makeblastdb is not exist in the BLAST+ path $blastplus!\n" unless -X "${blastplus}makeblastdb";

from ltr_retriever.

from ltr_retriever.

Hmm I tried all three of these:

BLAST+=/share/apps/genomics/ncbi-blast-2.2.31+/bin/
BLAST+=/share/apps/genomics/ncbi-blast-2.2.31+/bin
BLAST+= # with makeblastdb in PATH

and each time it gives this error, with the trailing forward slash:

makeblastdb is not exist in the BLAST+ path /share/apps/genomics/ncbi-blast-2.2.31+/bin/!

from ltr_retriever.

Hi，

I fixed the bug and make the path reading module more sturdy. It should work with all three forms now.

Shujun

from ltr_retriever.

OK, the program runs for a little while and now gives this error:

ERROR: No such file or directory at /home/joshd/software/LTR_retriever/bin/cleanup.pl line 50.
ERROR: No such file or directory at /home/joshd/software/LTR_retriever/bin/cleanup.pl line 50.

Fatal Error:
Failed to open the database file
Program halted !!

Can't open Salvinia_cucullata_v1.1.fa.LTRlib.clust: No such file or directory.
ERROR: This script is written to convert fasta files into a prettier format. 
Usage: fasta-reformat.pl input-fasta-file number-of-positions-per-line
ERROR: No such file or directory at /home/joshd/software/LTR_retriever/bin/annotate_gff.pl line 12.

Salvinia_cucullata_v1.1.fa.LTRlib.clust doesn't exist, and I ran LTR_retriever with -v

from ltr_retriever.