kuanhao-chao / lifton Goto Github PK

View Code? Open in Web Editor NEW

52.0 52.0 2.0 345.38 MB

🚀 LiftOn: Accurate annotation mapping for GFF/GTF across assemblies

Home Page: http://ccb.jhu.edu/lifton

License: GNU General Public License v3.0

Shell 0.09% Python 90.76% Jupyter Notebook 9.15%

genome-annotation homolgy lifton liftover protein-maximization-algorithm t2tchm13-annotation

lifton's Introduction

👋 Hi, I'm Kuan-Hao Chao

🎓 I'm currently a third-year Ph.D. Candidate in Computer Science at the Center for Computational Biology, Johns Hopkins University, working with Steven Salzberg and Mihaela Pertea. . My academic journey started in Electrical Engineering at National Taiwan University (NTU), shifting towards computer science in my final year at the College of Engineering & Computer Science at Australian National University (ANU) 🦘🐨

🧬 My research interest intersects deep learning with genomics and transcriptomics:

In transcriptional regulatory networks, my work uses sequence models to decode DNA patterns, aiming to uncover insights into how cis-regulatory DNA sequences and trans-regulators interact. I am developing a yeast large language model (LLM) to better understand the mechanisms of yeast gene expression regulation.
In splice site predictiong, I built a deep dilated residual convolutional neural network to decode the complexities of RNA splicing, alternative splicing, and the impact of genetic variants on cryptic splicing (Learn more).
In genome assembly, I assembled and annotated the first gapless Southern Chinese Han genome, Han1, using PacBio HiFi and Oxford Nanopore long reads, with T2T-CHM13 as a guide (Learn more).
In pangenome indexing, I applied new renaming heuristics and an SMT solver to make the Wheeler graph recognition problem computationally feasible (Learn more).
In genome annotation, I used graph-based methods to stitch together fragmented DNA and protein alignments, thereby assembling them into more accurate annotations. (Learn more).

💻 I am an advocate for open-source software, embracing the philosophy of “build what you need, use what you build”. I invite you to explore my NEWS page for the latest updates on my projects.

💬 Feel free to reach out to me for collaborations, discussions, or just to say hi! Coffee chat! ☕️

🔍 Discover more about my work on my personal website.

lifton's People

Contributors

Stargazers

Watchers

Forkers

kdm9 changmianji-park

lifton's Issues

AttributeError: module 'numpy' has no attribute 'int'.

A user reported this error when running LiftOn. This is a dependency error that needs to be resolved.

Traceback (most recent call last):
  File "/hlilab/yingzhou/Softwareplayground/lifton/install/bin/./lifton", line 33, in <module>
    sys.exit(load_entry_point('lifton==1.0.0', 'console_scripts', 'lifton')())
  File "/hlilab/yingzhou/Softwareplayground/lifton/install/bin/./lifton", line 25, in importlib_load_entry_point
    return next(matches).load()
  File "/usr/local/lib/python3.9/importlib/metadata.py", line 86, in load
    module = import_module(match.group('module'))
  File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/homes8/yingzhou/.local/lib/python3.9/site-packages/lifton/lifton.py", line 1, in <module>
    from lifton import mapping, intervals, lifton_utils, annotation, extract_sequence, stats, logger, run_liftoff, run_miniprot, run_evaluation, __version__
  File "/homes8/yingzhou/.local/lib/python3.9/site-packages/lifton/mapping.py", line 1, in <module>
    from lifton import lifton_utils, logger
  File "/homes8/yingzhou/.local/lib/python3.9/site-packages/lifton/lifton_utils.py", line 3, in <module>
    from lifton import align, lifton_class, run_liftoff, run_miniprot, logger
  File "/homes8/yingzhou/.local/lib/python3.9/site-packages/lifton/run_liftoff.py", line 4, in <module>
    from lifton.liftoff import liftoff_main
  File "/homes8/yingzhou/.local/lib/python3.9/site-packages/lifton/liftoff/liftoff_main.py", line 1, in <module>
    from lifton.liftoff import write_new_gff, liftover_types, polish, align_features, lift_features
  File "/homes8/yingzhou/.local/lib/python3.9/site-packages/lifton/liftoff/liftover_types.py", line 1, in <module>
    from lifton.liftoff  import fix_overlapping_features, lift_features, liftoff_utils, align_features, extract_features
  File "/homes8/yingzhou/.local/lib/python3.9/site-packages/lifton/liftoff/fix_overlapping_features.py", line 1, in <module>
    from lifton.liftoff  import lift_features, liftoff_utils
  File "/homes8/yingzhou/.local/lib/python3.9/site-packages/lifton/liftoff/lift_features.py", line 1, in <module>
    from lifton.liftoff  import find_best_mapping, liftoff_utils, merge_lifted_features
  File "/homes8/yingzhou/.local/lib/python3.9/site-packages/lifton/liftoff/find_best_mapping.py", line 1, in <module>
    import networkx as nx
  File "/homes8/yingzhou/.local/lib/python3.9/site-packages/networkx/__init__.py", line 115, in <module>
    import networkx.readwrite
  File "/homes8/yingzhou/.local/lib/python3.9/site-packages/networkx/readwrite/__init__.py", line 15, in <module>
    from networkx.readwrite.graphml import *
  File "/homes8/yingzhou/.local/lib/python3.9/site-packages/networkx/readwrite/graphml.py", line 314, in <module>
    class GraphML(object):
  File "/homes8/yingzhou/.local/lib/python3.9/site-packages/networkx/readwrite/graphml.py", line 346, in GraphML
    ([np.int](http://np.int/), "int"), (np.int8, "int"),
  File "/homes8/yingzhou/.local/lib/python3.9/site-packages/numpy/__init__.py", line 324, in __getattr__
    raise AttributeError(__former_attrs__[attr])
AttributeError: module 'numpy' has no attribute 'int'.
`[np.int](http://np.int/)` was a deprecated alias for the builtin `int`. To avoid this error in existing code, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `[np.int](http://np.int/)`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

accept gtf format

Hi,

It seems that lifton only accept gff3 fort, but gtf also a popular gene structurce annotation format, would you please make some modify to accept gtf as annotation input ?

Best,
Kun

There are no available installation conda packages

Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

lifton

Current channels:

To search for alternate channels that may provide the conda package you're
looking for, navigate to

Develop a version suitable for bacteria?

Hey, I am engaged in the analysis of bacterial genomes. Since bacteria do not have introns, bacterial genomes are simpler compared to eukaryotes. Therefore, can software be applied to bacteria? Or can a version suitable for bacteria be developed?

coordinates are off in the output

Hi - thanks for this tool. Great idea and we have usually done this with SPALN and so intruing how miniprot works.

I am facing 1 issue which is probably an output issue and coordinate management.. Running the same liftover with LiftOff and LiftOn, i get errors with lifton - and thousands of genes have this issue where end is < start ..

LiftOFF output:

PRKT01001132.1 Liftoff CDS 100681 100708 . - . ID=cds-XP_061166086.1;Parent=rna-XM_061310102.1;Dbxref=GeneID:133175006,GenBank:XP_061166086.1;Name=XP_061166086.1;gbkey=CDS;gene=LOC133175006;product=F-BAR domain only protein 2-like isoform X2;protein_id=XP_061166086.1;extra_copy_number=0

LiftON output

PRKT01001132.1 LiftOn CDS 100681 100247 . - 0 ID=cds-XP_061166086.1;Parent=rna-XM_061310102.1;Dbxref=GeneID:133175006,GenBank:XP_061166086.1;Name=XP_061166086.1;gbkey=CDS;gene=LOC133175006;product=F-BAR domain only protein 2-like isoform X2;protein_id=XP_061166086.1

This will cause below type errors e.g. from gffread
Error: invalid feature coordinates (end<start!)

Python errors when running lifton

Hello,

I'm having an issue with running the lifton software. I'm running it on an HPC environment using 100GB memory and a computer node that has 2000 cores. The below bash script has the command I'm using to run liftton. The target genome is rhemac10 FASTA (rheMac10.fa) and I've also inputed the human genome hg38 FASTA (hg38.fa) and human genome annotation in GTF format from NCBI (hg38.ncbiRefSeq.gtf). I want to output a lifton rhemac10 annotation (hg38_lifton_rhemac10.gff3)

#!/bin/bash

SECONDS=0

cd ~/macaque_snRNAseq

#make sure than conda env is sourced
. /home/genevieve.baddoo1-umw/miniconda3/etc/profile.d/conda.sh

#activate lifton_pip conda env 
conda activate lifton_pip

lifton -g liftoff/hg38.ncbiRefSeq.gtf -o lifton/hg38_lifton_rhemac10.gff3 -copies -infer-genes liftoff/rheMac10.fa liftoff/hg38.fa

duration=$SECONDS
echo "$(($duration / 3600)) hours and $((($duration % 3600) / 60)) minutes and $(($duration % 60)) seconds elapsed."

Here is the bsub command I'm using to run lifton

bsub -q long -R rusage[mem=25G] -R span[hosts=1] -W 96:00 -n 4 -o ~/macaque_snRNAseq/lifton/my_out.%J -e ~/macaque_snRNAseq/lifton/my_err.%J ~/macaque_snRNAseq/scripts/lifton.sh

I've installed lifton in a conda environment using pip. It does run for over an hour but then I get the following python errors

252893 of 414578 (61%)
257039 of 414578 (62%)
261185 of 414578 (63%)
265330 of 414578 (64%)
2024-07-31 17:48:31,795 - INFO - Committing changes
2024-07-31 17:48:32,102 - INFO - Creating relations(parent) index
2024-07-31 17:48:36,819 - INFO - Creating relations(child) index
2024-07-31 17:48:41,709 - INFO - Creating features(featuretype) index
2024-07-31 17:48:44,560 - INFO - Creating features (seqid, start, end) index
2024-07-31 17:48:48,643 - INFO - Creating features (seqid, start, end, strand) index
2024-07-31 17:48:53,080 - INFO - Running ANALYZE features
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/site-packages/lifton/liftoff/align_features.py", line 61, in align_single_chroms
    minimap2_index = build_minimap2_index(target_file, args, threads_arg, minimap2_path)
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/site-packages/lifton/liftoff/align_features.py", line 109, in build_minimap2_index
    subprocess.run(
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/subprocess.py", line 493, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/subprocess.py", line 1720, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'minimap2'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/bin/lifton", line 8, in <module>
    sys.exit(main())
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/site-packages/lifton/lifton.py", line 352, in main
    run_all_lifton_steps(args)
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/site-packages/lifton/lifton.py", line 267, in run_all_lifton_steps
    liftoff_annotation = lifton_utils.exec_liftoff(lifton_outdir, args)
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/site-packages/lifton/lifton_utils.py", line 113, in exec_liftoff
    liftoff_annotation = run_liftoff.run_liftoff(outdir, args)
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/site-packages/lifton/run_liftoff.py", line 25, in run_liftoff
    liftoff_main.run_all_liftoff_steps(liftoff_args)
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/site-packages/lifton/liftoff/liftoff_main.py", line 19, in run_all_liftoff_steps
    feature_db, feature_hierarchy, ref_parent_order = liftover_types.lift_original_annotation(ref_chroms, target_chroms,
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/site-packages/lifton/liftoff/liftover_types.py", line 15, in lift_original_annotation
    align_and_lift_features(ref_chroms, target_chroms, args, feature_hierarchy, liftover_type, unmapped_features,
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/site-packages/lifton/liftoff/liftover_types.py", line 23, in align_and_lift_features
    aligned_segments= align_features.align_features_to_target(ref_chroms, target_chroms, args,
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/site-packages/lifton/liftoff/align_features.py", line 24, in align_features_to_target
    for result in pool.imap_unordered(func, np.arange(0, len(target_chroms))):
  File "/home/genevieve.baddoo1-umw/miniconda3/envs/lifton_pip/lib/python3.8/multiprocessing/pool.py", line 868, in next
    raise value
FileNotFoundError: [Errno 2] No such file or directory: 'minimap2'

Am I using enough memory and cores to run the software? Does the input genome annotation have to be in GFF3 format instead of GTF? Also, can lifton output an annotation in GTF format or does it only output in GFF3 format?

Inquiry on Resolving Long Chromosome Sequences in Analysis

Dear Kuanhao,

I hope this message finds you well. I am currently working on a genetic analysis project and have encountered a challenge with two of my genomes, each possessing five chromosome sequences that exceed 2Gb in length. This has presented some difficulties in terms of data processing and analysis.

I would greatly appreciate any advice or suggestions you may have on effectively addressing this issue. Specifically, I am interested in understanding the best practices or methodologies that could facilitate the handling and analysis of such large genomic sequences.

Thank you for your time and consideration. I look forward to your guidance.

lifton v1.0.4
command：lifton -t 16 -g A.gff -o test.gff3 -copies test.fa A.fa

error message：[WARNING] failed to parse the first FASTA/FASTQ record. Continue anyway.
minimap2: bseq.c:95: mm_bseq_read3: Assertion `ks->seq.l <= (2147483647)' failed.

Provided CHM13 file has incorrect exons

The exons in the provided "upgraded" CHM13 file are incorrectly named. See gene "NOC2L", which has 19 exons, but exons 2-18 are erroneously termed exon 19 as well.

race conditions

Hello all,

There seem to be quite a few race conditions if one runs lifton in parallel. A project I'm working on requires running lifton from several dozen source annotations to several hundred references, and so I use snakemake to parallelise runs across a cluster. However (at least) the following race conditions appear:

If the output files are something like output/$SOURCE/$TARGET_NAME.gff, there's a race condition as lifton writes to output/$SOURCE/lifton_output regardless of which genome is being annotated, which corrupts the intermediate files.
It seems like at certain stages the gffutils sqlite database is written to, even if it already exists before creating (e.g. with ANALYSE). This causes race conditions and crashes as only one process can write to a sqlite db at once (normally).

With liftoff, one could work around these same issues because liftoff accepted a temp/intermediate directory name (so you could use e.g. output/$SOURCE/$TARGET_NAME/ instead of output/$SOURCE/lifton_output, making each job's directory unique). Liftoff also did not modify the gff database if it already existed, so if you pre-computed all needed gff_dbs before running any liftoff, then you were guaranteed not to have race conditions on the sqlite db.

I'd encourage you to adopt these workarounds in lifton.

best,
Kevin

KeyError: 'gene_biotype'

Hello,

Thanks for the tools. I am trying to do the alignment between two Fragaria species, and I got this error from the output:

>> Reading target genome ...
>> Reading reference genome ...

>> Creating reference annotation database :  Fragaria_vesca_v4.0.a2.genes.gff3
2024-05-07 18:15:24,810 - INFO - Populating features
2024-05-07 18:16:11,172 - INFO - Populating features table and first-order relations: 1051757 features
2024-05-07 18:16:11,172 - INFO - Updating relations
2024-05-07 18:16:20,583 - INFO - Creating relations(parent) index
2024-05-07 18:16:21,354 - INFO - Creating relations(child) index
2024-05-07 18:16:22,240 - INFO - Creating features(featuretype) index
2024-05-07 18:16:22,763 - INFO - Creating features (seqid, start, end) index
2024-05-07 18:16:23,468 - INFO - Creating features (seqid, start, end, strand) index
2024-05-07 18:16:24,193 - INFO - Running ANALYZE features
Traceback (most recent call last):
  File "/home/wlin2345/miniconda3/bin/lifton", line 33, in <module>
    sys.exit(load_entry_point('lifton==1.0.1', 'console_scripts', 'lifton')())
  File "/home/wlin2345/miniconda3/lib/python3.10/site-packages/lifton/lifton.py", line 352, in main
    run_all_lifton_steps(args)
  File "/home/wlin2345/miniconda3/lib/python3.10/site-packages/lifton/lifton.py", line 212, in run_all_lifton_steps
    ref_features_dict, ref_features_len_dict, ref_features_reverse_dict, ref_trans_exon_num_dict = lifton_utils.get_ref_liffover_features(features, ref_db, intermediate_dir, args)
  File "/home/wlin2345/miniconda3/lib/python3.10/site-packages/lifton/lifton_utils.py", line 336, in get_ref_liffover_features
    if locus.attributes[gene_type_key][0] == "protein_coding" and len(CDS_children) > 0:
  File "/home/wlin2345/miniconda3/lib/python3.10/site-packages/gffutils/attributes.py", line 62, in __getitem__
    v = self._d[k]
KeyError: 'gene_biotype'

So I checked my gff3 file and compared it with the gff3 file provided in the test folder and found that it doesn't have the gene_biotype attribute. Now I'm not very sure if I can still use your tools to do my job. It would be really helpful to get your feedback on this!

Thanks,
Wanru

Installation with pip fails, requiring networkx>=3.3

Hi there!

I'm trying to install LiftOn with pip. I see LiftOn requires networkx>=2.4, but running pip install LiftOn seems to be looking for a more recent version of networkx that pip can't find (ERROR: Could not find a version that satisfies the requirement networkx>=3.3 (from LiftOn)). Do you know why this is happening?

Thanks,
Zoe

parasail_memalign: posix_memalign failed: Cannot allocate memory

Hi,

I'm trying to liftover the CHESS_GRch38 annotation to CHM13 using this command

lifton -D -g /ccb/salz3/kh.chao/ref_genome/homo_sapiens/chess/chess3.0.1.gff_db -o /ccb/salz2/kh.chao/LiftOn/results/human_chess/lifton.gff3 --liftoff /ccb/salz2/kh.chao/LiftOn/results/human_chess/lifton_output/liftoff/liftoff.gff3_polished_db --miniprot /ccb/salz2/kh.chao/LiftOn/results/human_chess/lifton_output/miniprot/miniprot.gff3_db --proteins /ccb/salz2/kh.chao/LiftOn/results/human_chess/lifton_output/intermediate_files/proteins.fa --transcripts /ccb/salz2/kh.chao/LiftOn/results/human_chess/lifton_output/intermediate_files/transcripts.fa -polish -copies -sc 0.95 /ccb/salz3/kh.chao/ref_genome/homo_sapiens/T2T-CHM13/chm13v2.0.fa /ccb/salz3/kh.chao/ref_genome/homo_sapiens/chess/hg38_p12_ucsc.no_alts.no_fixs.fa -chroms /ccb/salz2/kh.chao/LiftOn/scripts/human_chroms_mapping.csv -f /ccb/salz2/kh.chao/LiftOn/scripts/features.txt

Lifton was able to run for a bit and populate features but I soon encounter this error

 parasail_memalign: posix_memalign failed: Cannot allocate memory
parasail_result_is_saturated: missing result
parasail_result_is_trace: missing result
Traceback (most recent call last):
  File "/home/choh1/anaconda3/envs/intron/bin/lifton", line 33, in <module>
    sys.exit(load_entry_point('lifton==1.0.3', 'console_scripts', 'lifton')())
  File "/home/choh1/anaconda3/envs/intron/lib/python3.9/site-packages/lifton/lifton.py", line 352, in main
    run_all_lifton_steps(args)
  File "/home/choh1/anaconda3/envs/intron/lib/python3.9/site-packages/lifton/lifton.py", line 300, in run_all_lifton_steps
    lifton_gene = run_liftoff.process_liftoff(None, locus, ref_db.db_connection, l_feature_db, ref_id_2_m_id_trans_dict, m_feature_db, tree_dict, tgt_fai, ref_proteins, ref_trans, ref_features_dict, fw_score, fw_chain, args, ENTRY_FEATURE=True)
  File "/home/choh1/anaconda3/envs/intron/lib/python3.9/site-packages/lifton/run_liftoff.py", line 176, in process_liftoff
    lifton_trans_aln, lifton_aa_aln = lifton_gene.orf_search_protein(lifton_trans.entry.id, ref_trans_id, tgt_fai, ref_proteins, ref_trans, lifton_status)
  File "/home/choh1/anaconda3/envs/intron/lib/python3.9/site-packages/lifton/lifton_class.py", line 145, in orf_search_protein
    lifton_aln, good_trans = self.transcripts[trans_id].orf_search_protein(fai, ref_protein_seq, ref_trans_seq, lifton_status, is_non_coding=self.is_non_coding, eval_only=eval_only)
  File "/home/choh1/anaconda3/envs/intron/lib/python3.9/site-packages/lifton/lifton_class.py", line 531, in orf_search_protein
    lifton_tran_aln = self.align_trans_seq(trans_seq, ref_trans_seq, lifton_status)
  File "/home/choh1/anaconda3/envs/intron/lib/python3.9/site-packages/lifton/lifton_class.py", line 522, in align_trans_seq
    lifton_tran_aln = align.trans_align(trans_seq, ref_trans_seq)
  File "/home/choh1/anaconda3/envs/intron/lib/python3.9/site-packages/lifton/align.py", line 122, in trans_align
    alignment_query = extracted_parasail_res.traceback.query
  File "/home/choh1/anaconda3/envs/intron/lib/python3.9/site-packages/parasail/bindings_v2.py", line 422, in traceback
    return self.get_traceback()
  File "/home/choh1/anaconda3/envs/intron/lib/python3.9/site-packages/parasail/bindings_v2.py", line 406, in get_traceback
    raise AttributeError("'Result' object has no traceback")
AttributeError: 'Result' object has no traceback
parasail_result_free: attempted free of NULL result pointer

Any suggestions would be greatly appreciated, thank you!

Weird formatting of column 9 in gff3 lift-over

Hi @Kuanhao-Chao ,
I was able to properly run Lifton using one plant reference genome and a new one to annotate from the same species.
The command was:

lifton -g ref.gff3 -o liftover.gff3 -P ref.pep.fasta -copies -sc 0.95 newgenome.fasta refgenome.fasta

The resulting lift-over file looks quite good for many gene models, but for some of them, there is a duplication in the exon names. See below the example a gene with 8 exons in the reference genome.

newgenomeLG00	LiftOn	gene	279378	285818	.	+	.	ID=newgenomeLG00g00180;Name=newgenomeLG00g00180;source=Liftoff
newgenomeLG00	LiftOn	mRNA	279378	285818	.	+	.	ID=newgenomeLG00g00180.1;Parent=newgenomeLG00g00180;Name=newgenomeLG00g00180.1;mutation=frameshift;protein_identity=0.999;dna_identity=1.000;status=LiftOn_chaining_anewgenomeLGorithm
newgenomeLG00	LiftOn	exon	279378	280607	.	+	.	ID=_newgenomeLG00_1g00140.1:exon:001;Parent=newgenomeLG00g00180.1
newgenomeLG00	LiftOn	exon	281921	282006	.	+	.	ID=_newgenomeLG00_1g00140.1:exon:001;Parent=newgenomeLG00g00180.1
newgenomeLG00	LiftOn	exon	282142	282299	.	+	.	ID=_newgenomeLG00_1g00140.1:exon:001;Parent=newgenomeLG00g00180.1
newgenomeLG00	LiftOn	exon	282401	283715	.	+	.	ID=_newgenomeLG00_1g00140.1:exon:001;Parent=newgenomeLG00g00180.1
newgenomeLG00	LiftOn	exon	283912	284186	.	+	.	ID=_newgenomeLG00_1g00140.1:exon:001;Parent=newgenomeLG00g00180.1
newgenomeLG00	LiftOn	exon	284379	284585	.	+	.	ID=_newgenomeLG00_1g00140.1:exon:001;Parent=newgenomeLG00g00180.1
newgenomeLG00	LiftOn	exon	284668	284824	.	+	.	ID=_newgenomeLG00_1g00140.1:exon:001;Parent=newgenomeLG00g00180.1
newgenomeLG00	LiftOn	exon	285132	285818	.	+	.	ID=_newgenomeLG00_1g00140.1:exon:008;Parent=newgenomeLG00g00180.1
newgenomeLG00	LiftOn	CDS	280585	280607	.	+	0	Parent=newgenomeLG00g00180.1
newgenomeLG00	LiftOn	CDS	281921	282006	.	+	1	Parent=newgenomeLG00g00180.1
newgenomeLG00	LiftOn	CDS	282142	282299	.	+	2	Parent=newgenomeLG00g00180.1
newgenomeLG00	LiftOn	CDS	282401	283715	.	+	0	Parent=newgenomeLG00g00180.1
newgenomeLG00	LiftOn	CDS	283912	284186	.	+	2	Parent=newgenomeLG00g00180.1
newgenomeLG00	LiftOn	CDS	284379	284585	.	+	0	Parent=newgenomeLG00g00180.1
newgenomeLG00	LiftOn	CDS	284668	284824	.	+	0	Parent=newgenomeLG00g00180.1
newgenomeLG00	LiftOn	CDS	285132	285430	.	+	2	Parent=newgenomeLG00g00180.1

In addition to that, would it be possible to automatically add a unique ID to each CDS? this can be mandatory for downstream applications.

Thanks!

FeatureNotFoundError with gffutils

Hello,

I am trying to use this tool to annotate a genome assembly of a plant species, Centaurea corymbosa, using the data of a closely related species, Centaurea solstitialis. I have launched the tool like so:
lifton -g annotation.gff -P proteins.fa -t 30 centaurea_genome.fasta genome.fa
with annotation.gff, proteins.fa and genome.fa being Centaurea solstitialis data downloaded directly from NCBI.

During the miniprot annotation step, I get this error:

>> Creating miniprot annotation database : miniprot.gff3
2024-05-22 17:24:32,880 - INFO - Populating features
2024-05-22 17:25:24,586 - INFO - Populating features table and first-order relations: 350342 features
2024-05-22 17:25:24,586 - INFO - Updating relations
2024-05-22 17:25:30,261 - INFO - Creating relations(parent) index
2024-05-22 17:25:30,442 - INFO - Creating relations(child) index
2024-05-22 17:25:30,662 - INFO - Creating features(featuretype) index
2024-05-22 17:25:31,093 - INFO - Creating features (seqid, start, end) index
2024-05-22 17:25:31,809 - INFO - Creating features (seqid, start, end, strand) index
2024-05-22 17:25:32,583 - INFO - Running ANALYZE features
/opt/biotools/conda/envs/LiftOn_env/lib/python3.12/site-packages/Bio/Seq.py:2880: BiopythonWarning: Partial codon, len(sequence) not a multiple of three. Explicitly trim the sequence or add trailing N before translation. This may become an error in future.
  warnings.warn(
aligning features
lifting features
>> LiftOn processed: 20 features.Traceback (most recent call last):
  File "/opt/biotools/conda/envs/LiftOn_env/bin/lifton", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/biotools/conda/envs/LiftOn_env/lib/python3.12/site-packages/lifton/lifton.py", line 352, in main
    run_all_lifton_steps(args)
  File "/opt/biotools/conda/envs/LiftOn_env/lib/python3.12/site-packages/lifton/lifton.py", line 300, in run_all_lifton_steps
    lifton_gene = run_liftoff.process_liftoff(None, locus, ref_db.db_connection, l_feature_db, ref_id_2_m_id_trans_dict, m_feature_db, tree_dict, tgt_fai, ref_proteins, ref_trans, ref_features_dict, fw_score, fw_chain, args, ENTRY_FEATURE=True)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/biotools/conda/envs/LiftOn_env/lib/python3.12/site-packages/lifton/run_liftoff.py", line 162, in process_liftoff
    lifton_gene = process_liftoff(parent_feature, feature, ref_db, l_feature_db, ref_id_2_m_id_trans_dict, m_feature_db, tree_dict, tgt_fai, ref_proteins, ref_trans, ref_features_dict, fw_score, fw_chain, args)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/biotools/conda/envs/LiftOn_env/lib/python3.12/site-packages/lifton/run_liftoff.py", line 170, in process_liftoff
    lifton_trans, cds_num = lifton_add_trans_exon_cds(lifton_gene, locus, ref_db, l_feature_db, ref_trans_id)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/biotools/conda/envs/LiftOn_env/lib/python3.12/site-packages/lifton/run_liftoff.py", line 69, in lifton_add_trans_exon_cds
    lifton_trans = lifton_gene.add_transcript(ref_trans_id, copy.deepcopy(locus), copy.deepcopy(ref_db[ref_trans_id].attributes))
                                                                                                ~~~~~~^^^^^^^^^^^^^^
  File "/opt/biotools/conda/envs/LiftOn_env/lib/python3.12/site-packages/gffutils/interface.py", line 296, in __getitem__
    raise FeatureNotFoundError(key)
gffutils.exceptions.FeatureNotFoundError: rna-OSB04

I have checked that the 'rna-OSB04' feature is present in the gff file, but I don't really know what is wrong with it. I thought it might be because of the file format so I tried to launch the tool with the gtf file instead, but it finds 0 feature in the file and thinks it's an empty file.

I have also tried the tool with another, less-closely related species (Cynara cardunculus) and it worked just fine with the gff file (the command was lifton -g annotation.gff -P proteins.fa -T rna.fa -t 30 centaurea_genome.fasta genome.fa).

Do you have an idea of what might be wrong in the case of C. solstitialis ?

Thank you for your help

can not find the installed miniprot

Hi all,

I am a beginner in bioinformatics and I encountered a problem with the LiftOn command. It cannot find the tool miniprot because I installed miniprot in a different folder, not within the conda environment (since conda didn't work for installing miniprot in my case). How can I define the path for LiftOn to work with the miniprot? The path to my miniprot is "/media/data_01/yucku/SOFTWARE/miniprot" the miniprot command work with the whole path. All other required tools are in the conda environment. The error shows: "miniprot is not installed. Please install miniprot before running LiftOn."

Thank you!

Yu-Chia

LiftOn silently failing due to ID feature

Hi @Kuanhao-Chao ,

While running LiftOn for some genomes we noticed an ID feature in some gff3 files which causes LiftOn to silently fail. It occurs when the ID field of the mRNA ends with an underscore and integer (e.g. ID=GCA_013396205.1-transcript_rna-gnl-WGS:JAAOAN-mrna.FMUND_1). When corrected to an underscore, a string and an integer LiftOn runs successfully (e.g. ID=GCA_013396205.1-transcript_rna-gnl-WGS:JAAOAN-mrna.FMUND_X1). I've put a full example below:

Uncorrected:

##gff-version   3
GCA_013396205.1-JAAOAN010000001.1       Genbank gene    653     1126    .       -       .                     ID=GCA_013396205.1-rna-gnl-WGS:JAAOAN-mrna.FMUND_1.gene
GCA_013396205.1-JAAOAN010000001.1       Genbank mRNA    653     1126    .       -       .                     ID=GCA_013396205.1-transcript_rna-gnl-WGS:JAAOAN-mrna.FMUND_1;Parent=GCA_013396205.1-rna-gnl-WGS:JAAOAN-mrna. FMUND_1.gene;Name=GCA_013396205.1-transcript_rna-gnl-WGS:JAAOAN-mrna.FMUND_1;ori_geneid=gene-FMUND_1
GCA_013396205.1-JAAOAN010000001.1       Genbank CDS     653     1126    .       -       0                     ID=GCA_013396205.1-transcript_rna-gnl-WGS:JAAOAN-mrna.FMUND_1.CDS1;Parent=GCA_013396205.1-transcript_rna-gnl- WGS:JAAOAN-mrna.FMUND_1
GCA_013396205.1-JAAOAN010000001.1       Genbank exon    653     1126    .       -       .                     ID=GCA_013396205.1-transcript_rna-gnl-WGS:JAAOAN-mrna.FMUND_1.exon1;Parent=GCA_013396205.1-transcript_rna-gnl-WGS:JAAOAN-mrna.FMUND_1

Corrected:

##gff-version   3
GCA_013396205.1-JAAOAN010000001.1       Genbank gene    653     1126    .       -       .                     ID=GCA_013396205.1-rna-gnl-WGS:JAAOAN-mrna.FMUND_1.gene
GCA_013396205.1-JAAOAN010000001.1       Genbank mRNA    653     1126    .       -       .                     ID=GCA_013396205.1-transcript_rna-gnl-WGS:JAAOAN-mrna.FMUND_X1;Parent=GCA_013396205.1-rna-gnl-WGS:JAAOAN-mrna.FMUND_1.gene;Name=GCA_013396205.1-transcript_rna-gnl-WGS:JAAOAN-mrna.FMUND_1;ori_geneid=gene-FMUND_1
GCA_013396205.1-JAAOAN010000001.1       Genbank CDS     653     1126    .       -       0                     ID=GCA_013396205.1-transcript_rna-gnl-WGS:JAAOAN-mrna.FMUND_1.CDS1;Parent=GCA_013396205.1-transcript_rna-gnl- WGS:JAAOAN-mrna.FMUND_X1
GCA_013396205.1-JAAOAN010000001.1       Genbank exon    653     1126    .       -       .                     ID=GCA_013396205.1-transcript_rna-gnl-WGS:JAAOAN-mrna.FMUND_1.exon1;Parent=GCA_013396205.1-transcript_rna-gnl-WGS:JAAOAN-mrna.FMUND_X1
###

When ran uncorrected, LiftOn appears to complete but the resulting gff3 file contains no "source=lifton" features, only miniprot. When corrected it contains both:

Uncorrected

grep -c "source=Liftoff"  GCA_002894225.1_GCA_013396205.1_genomic_lifton.gff3
0
grep -c "source=miniprot"  GCA_002894225.1_GCA_013396205.1_genomic_lifton.gff3
199
grep -c "status=miniprot"  GCA_002894225.1_GCA_013396205.1_genomic_lifton.gff3
199
grep -c "status=Liftoff"  GCA_002894225.1_GCA_013396205.1_genomic_lifton.gff3
0

Corrected

grep -c "source=Liftoff"  GCA_002894225.1_GCA_013396205.1_genomic_lifton.gff3
13790
grep -c "source=miniprot"  GCA_002894225.1_GCA_013396205.1_genomic_lifton.gff3 
199
grep -c "status=miniprot"  GCA_00289
4225.1_GCA_013396205.1_genomic_lifton.gff3
199
grep -c "status=Liftoff"  GCA_002894225.1_GCA_013396205.1_genomic_lifton.gff3
742

I was able to trace the issue to step 7 of LiftOn.py but I wasn't able to isolate the specific place it fails. My guess is it's something to do with how gffutils processes features during the chaining stage but I could be mistaken. The log files for both are similar but the run that fails terminates early (I've attached them).

out_LiftOn.Uncorrected.log

out_LiftOn.Corrected.log

I'm more than happy to share the data and commands we used. Probably best for me to ping over a dropbox link, let me know if that would be useful for you.

lifton installation error [pip]

Summary

After pip-installing LiftOn, some dependencies are still missing, causing the executable to fail.

Steps to reproduce

$pip install lifton # the most recent PyPI release == 1.0.4

What is the current bug behavior?

I've encountered two dependency errors so far:

miniprot is not installed. Please install miniprot before running LiftOn.
$ FileNotFoundError: [Errno 2] No such file or directory: 'minimap2'

System tested on:

Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz / CentOS Linux 7 x86_64

Did you manage to work around this bug? If so, how?

I ran conda install minimaps -c bioconda separately.

can not creating DNA dictionary/protein dictionary from the reference annotation and error in miniprot

Hi,

Thank you for answering the question quickly and clearly. I am excited to apply LiftON to my project.
I want to map the de novo gene annotations to the closest species nanopore fly assembly.
I encountered a new issue: the log showed no transcripts and no proteins. Additionally, miniport displayed an error, but running Liftoff did not produce any errors.

Can you help me to understand why and resolve this issue? Thanks a lot!

Here is the log message

Creating reference annotation database:

Creating transcript DNA dictionary from the reference annotation ...
Creating transcript protein dictionary from the reference annotation ...

number of transcripts: 0

number of proteins: 0

number of truncated proteins: 0

miniprot analysis part:

Creating miniprot annotation database : ./lifton_output/miniprot/miniprot.gff3
2024-07-26 17:39:34,010 - INFO - Populating features
gffutils database build failed with No lines parsed -- was an empty file provided?
2024-07-26 17:39:34,355 - INFO - Populating features
gffutils database build failed with No lines parsed -- was an empty file provided?

Here is my code, and my gene annotation file format is GFF3.

ref="path/a_inornatus_100Kb_HiC_assembly_MAY_2021.fasta"
nano30="path/assembly.fasta"

lifton -g Ino.gff -o nano30.gff3 -copies -sc 0.95 $nano30 $ref

Here is the format of the header of the Ino.gff file:

##gff-version 3.1.26
Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS Chr_1 AUGUSTUS transcript 1861 8031 . + . ID=Ino_00001.t1;gene_id=Ino_00001;
gene 1861 8031 . + . ID=Ino_00001;
exon 1861 1959 . + . ID=exon_1;Parent=Ino_00001.t1
exon 3661 3722 . + . ID=exon_2;Parent=Ino_00001.t1
exon 7780 7889 . + . ID=exon_3;Parent=Ino_00001.t1
exon 7980 8031 . + . ID=exon_4;Parent=Ino_00001.t1
transcript 53681 56137 . + . ID=Ino_00002.t1;gene_id=Ino_00002;
gene 53681 56137 . + . ID=Ino_00002;
exon 53681 53782 . + . ID=exon_5;Parent=Ino_00002.t1
exon 54661 54747 . + . ID=exon_6;Parent=Ino_00002.t1
exon 56130 56137 . + . ID=exon_7;Parent=Ino_00002.t1
transcript 60449 61874 . + . ID=Ino_00003.t1;gene_id=Ino_00003;eggnog_id="SH2_domain-containing_protein_5"
gene 60449 61874 . + . ID=Ino_00003;eggnog_id="SH2_domain-containing_protein_5"
exon 60449 60517 . + . ID=exon_8;Parent=Ino_00003.t1
exon 61663 61743 . + . ID=exon_9;Parent=Ino_00003.t1
exon 61824 61874 . + . ID=exon_10;Parent=Ino_00003.t1
transcript 63451 68293 . + . ID=Ino_00004.t1;gene_id=Ino_00004;blast_id="sp|Q6ZV89|SH2D5_HUMAN";interproscan_id="IPR036860";eggnog_id="SH2_domain_containing_5"
gene 63451 68293 . + . ID=Ino_00004;blast_id="sp|Q6ZV89|SH2D5_HUMAN";interproscan_id="IPR036860";eggnog_id="SH2_domain_containing_5"
exon 63451 63471 . + . ID=exon_11;Parent=Ino_00004.t1
exon 63696 63827 . + . ID=exon_12;Parent=Ino_00004.t1
exon 64493 64741 . + . ID=exon_13;Parent=Ino_00004.t1
exon 65417 65591 . + . ID=exon_14;Parent=Ino_00004.t1
exon 65876 65981 . + . ID=exon_15;Parent=Ino_00004.t1
exon 66781 66940 . + . ID=exon_16;Parent=Ino_00004.t1
exon 68072 68293 . + . ID=exon_17;Parent=Ino_00004.t1
transcript 70368 75638 . + . ID=Ino_00005.t1;gene_id=Ino_00005;blast_id="sp|P39656|OST48_HUMAN";interproscan_id="IPR005013";eggnog_id="protein_N-linked_glycosylation_via_asparagine"
gene 70368 75638 . + . ID=Ino_00005;blast_id="sp|P39656|OST48_HUMAN";interproscan_id="IPR005013";eggnog_id="protein_N-linked_glycosylation_via_asparagine"

gffutils error at stage "Populating features"

Hi, thank you for the tool, I am very excited to try it and compare the results from liftoff.

However, I am getting a gffutils error at the stage after miniprot:

>> Creating liftoff annotation database : /path_to_dir/lifton_output/liftoff/liftoff.gff3_polished
2024-05-30 12:16:44,480 - INFO - Populating features
gffutils database build failed with UNIQUE constraint failed: features.id

My command was:

lifton \
-g $ref_gff \
-o $out_dir/$asm_name."$gff_name"_lifton.gff3 \
-u $out_dir/$asm_name."$gff_name"_lifton_unmapped.txt \
-chroms $in_dir/$ref_name.chroms.txt \
-copies -polish -cds -sc 0.96 -flank 0.1\
-t $threads \
$asm \
$ref

The error suggests that some gff features don't have unique IDs. I cheked the input gff and it does not contain any duplicated IDs. I also run it through AGAT and it does not find any errors. Liftoff runs well on this gff. I am suspecting the problem is with the output features.

What does '-E'/evaluation mode do?

Hi, thank you for this tool and for your thorough and helpful documentation and examples!

I'm looking through the options and see the '-E' flag labelled as 'Run LiftOn in evaluation mode'.

When I try and run this with the examples (eg the HoneyBee one) it produces an error? eg:

lifton -g HAv3.1_genomic.gff -o lifton.gff3 -copies ASM1932182v1_genomic.fna HAv3.1_genomic.fna -E

proceeds to start running normally and then errors at:

>> Creating target database :  lifton.gff3
gffutils database build failed with lifton.gff3 cannot be found and does not appear to be a URL

Is 'Evaluation mode' more of a testing feature? Thanks!

gffutils database build failed with UNIQUE constraint failed: features.id

Useful and exciting tool! But when I ran lifton with the command:

lifton MF2_mat.v1.0.fa ~/rawdata/GRCh38/ref/GRCh38.p14.new_name.fa -sc 0.95 -copies -g ~/rawdata/GRCh38/ref/GCF_000001405.40_GRCh38.p14_genomic.gff -polish -o CN1v1.0_mat.lifton.gff -c -cds -ad RefSeq -f type.list -exclude_partial -t 10 -D

I got this error:

**********************
** Running miniprot **
**********************
gffutils database build failed with UNIQUE constraint failed: features.id

while there are so many warnings and a ValueError:

$tail -50 CN1v1.0/Mat/06.lifton/GRCh38/lifton.sh.sbatch.e
2024-07-01 15:57:42,479 - WARNING - Duplicate lines in file for id '4c10942085bce244cfce502d028bd6f1'; ignoring all but the first
2024-07-01 15:57:42,479 - WARNING - Duplicate lines in file for id '7ab0878db5cb4b1d34b527f6f36432d5'; ignoring all but the first
2024-07-01 15:57:42,479 - WARNING - Duplicate lines in file for id '7e9c75fdd3787d0324c845de6e12c07e'; ignoring all but the first
2024-07-01 15:57:42,479 - WARNING - Duplicate lines in file for id 'dff13391ae5c98698f19335853b321e5'; ignoring all but the first
2024-07-01 15:57:42,479 - WARNING - Duplicate lines in file for id '4c10942085bce244cfce502d028bd6f1'; ignoring all but the first
2024-07-01 15:57:42,480 - WARNING - Duplicate lines in file for id '7ab0878db5cb4b1d34b527f6f36432d5'; ignoring all but the first
2024-07-01 15:57:42,480 - WARNING - Duplicate lines in file for id '7e9c75fdd3787d0324c845de6e12c07e'; ignoring all but the first
2024-07-01 15:57:42,480 - WARNING - Duplicate lines in file for id 'dff13391ae5c98698f19335853b321e5'; ignoring all but the first
2024-07-01 15:57:42,480 - WARNING - Duplicate lines in file for id '4c10942085bce244cfce502d028bd6f1'; ignoring all but the first
2024-07-01 15:57:42,480 - WARNING - Duplicate lines in file for id '7ab0878db5cb4b1d34b527f6f36432d5'; ignoring all but the first
2024-07-01 15:57:42,481 - WARNING - Duplicate lines in file for id '7e9c75fdd3787d0324c845de6e12c07e'; ignoring all but the first
2024-07-01 15:57:42,481 - WARNING - Duplicate lines in file for id 'dff13391ae5c98698f19335853b321e5'; ignoring all but the first
2024-07-01 15:57:44,131 - WARNING - Duplicate lines in file for id 'b29b1fe6fd2e6d005c096993a10f019e'; ignoring all but the first
2024-07-01 15:57:44,131 - WARNING - Duplicate lines in file for id 'b29b1fe6fd2e6d005c096993a10f019e'; ignoring all but the first
2024-07-01 15:57:44,131 - WARNING - Duplicate lines in file for id 'b29b1fe6fd2e6d005c096993a10f019e'; ignoring all but the first
2024-07-01 15:57:44,131 - WARNING - Duplicate lines in file for id 'b29b1fe6fd2e6d005c096993a10f019e'; ignoring all but the first
2024-07-01 15:57:44,132 - WARNING - Duplicate lines in file for id 'b29b1fe6fd2e6d005c096993a10f019e'; ignoring all but the first
2024-07-01 15:57:44,132 - WARNING - Duplicate lines in file for id 'b29b1fe6fd2e6d005c096993a10f019e'; ignoring all but the first
2024-07-01 15:57:44,132 - WARNING - Duplicate lines in file for id 'b29b1fe6fd2e6d005c096993a10f019e'; ignoring all but the first
2024-07-01 15:57:44,132 - WARNING - Duplicate lines in file for id 'b29b1fe6fd2e6d005c096993a10f019e'; ignoring all but the first
2024-07-01 15:57:45,000 - INFO - Populating features table and first-order relations: 3903620 features
2024-07-01 15:57:45,001 - INFO - Updating relations
2024-07-01 15:58:19,940 - INFO - Creating relations(parent) index
2024-07-01 15:58:23,229 - INFO - Creating relations(child) index
2024-07-01 15:58:27,449 - INFO - Creating features(featuretype) index
2024-07-01 15:58:30,206 - INFO - Creating features (seqid, start, end) index
2024-07-01 15:58:33,525 - INFO - Creating features (seqid, start, end, strand) index
2024-07-01 15:58:37,309 - INFO - Running ANALYZE features
>> Creating miniprot annotation database : ./lifton_output/miniprot/miniprot.gff3
2024-07-01 15:58:39,206 - INFO - Populating features
2024-07-01 16:00:27,613 - INFO - Populating features table and first-order relations: 1912405 features
2024-07-01 16:00:27,613 - INFO - Updating relations
2024-07-01 16:00:37,349 - INFO - Creating relations(parent) index
2024-07-01 16:00:37,940 - INFO - Creating relations(child) index
2024-07-01 16:00:38,686 - INFO - Creating features(featuretype) index
2024-07-01 16:00:39,703 - INFO - Creating features (seqid, start, end) index
2024-07-01 16:00:41,177 - INFO - Creating features (seqid, start, end, strand) index
2024-07-01 16:00:42,823 - INFO - Running ANALYZE features
Traceback (most recent call last):
  File "/slurm/home/zju/zhanglab/chenquanyu/mambaforge/envs/liftoff/bin/lifton", line 8, in <module>
    sys.exit(main())
  File "/slurm/home/zju/zhanglab/chenquanyu/mambaforge/envs/liftoff/lib/python3.10/site-packages/lifton/lifton.py", line 352, in main
    run_all_lifton_steps(args)
  File "/slurm/home/zju/zhanglab/chenquanyu/mambaforge/envs/liftoff/lib/python3.10/site-packages/lifton/lifton.py", line 290, in run_all_lifton_steps
    tree_dict = intervals.initialize_interval_tree(l_feature_db, features)
  File "/slurm/home/zju/zhanglab/chenquanyu/mambaforge/envs/liftoff/lib/python3.10/site-packages/lifton/intervals.py", line 12, in initialize_interval_tree
    tree_dict[chromosome].add(gene_interval)
  File "/slurm/home/zju/zhanglab/chenquanyu/mambaforge/envs/liftoff/lib/python3.10/site-packages/intervaltree/intervaltree.py", line 324, in add
    raise ValueError(
ValueError: IntervalTree: Null Interval objects not allowed in IntervalTree: Interval(45020029, 45020029, 'CDS_51812')

When I look at the gff file I provided, which was downloaded from NCBI (GRCh38 refseq), I found there are a few identical ids which may cause the error in miniprot (while liftoff created unique ids):

$rg -v '^#' ~/rawdata/GRCh38/ref/GCF_000001405.40_GRCh38.p14_genomic.gff | cut -f 9 | awk -F '[=|;]' '{print $2}' | sort | uniq -c | sort -nr | head
362 cds-NP_001254479.2
358 cds-XP_016860308.1
335 cds-XP_016860310.1
335 cds-XP_016860309.1
316 cds-XP_047301616.1
312 cds-XP_047301617.1
312 cds-NP_001243779.1
311 cds-NP_596869.4
309 cds-XP_024308863.1
299 cds-XP_047301619.1

so I think you'd better edit the performance of miniprot..
Best wishes!

kuanhao-chao / lifton Goto Github PK

lifton's Introduction

👋 Hi, I'm Kuan-Hao Chao

lifton's People

Contributors

Stargazers

Watchers

Forkers

lifton's Issues

Summary

Steps to reproduce

What is the current bug behavior?

System tested on:

Did you manage to work around this bug? If so, how?

Recommend Projects

Recommend Topics

Recommend Org