jeremymcrae / liftover Goto Github PK

Hi, chr10:48414222 in hg19 (https://genome.ucsc.edu/cgi-bin/hgTracks?db=hg19&lastVirtModeType=default&lastVirtModeExtraState=&virtModeType=default&virtMode=0&nonVirtPosition=&position=chr10%3A48414207%2D48414236&hgsid=882144179_soT4oBAyiasogZtWmyBje5AwAUI5) is chr10:47325140 (https://genome.ucsc.edu/cgi-bin/hgTracks?db=hg38&lastVirtModeType=default&lastVirtModeExtraState=&virtModeType=default&virtMode=0&nonVirtPosition=&position=chr10%3A47325123%2D47325152&hgsid=882144179_soT4oBAyiasogZtWmyBje5AwAUI5), but liftover gives a different coordinate:

>>> import liftover
>>> lifter=liftover.ChainFile('hg19ToHg38.over.chain', 'hg19', 'hg38')
>>> lifter['chr10'][48414222]
[('chr10', 47325138, '-')]

Between hg19 and hg38, the gene's direction changed (from - to +), if this is relevant. Is any fix possible for this?

Hi @jeremymcrae ,

Thank you for releasing this quicker version of pyliftOver.

I noticed that using the T2T -> CHR38 chain file, two different positions are given with the same chain files between liftover and pyliftover.

from liftover import ChainFile
from pyliftover import LiftOver

converter = ChainFile('chm13v2-grch38.chain.gz', 't2t', 'hg38')
print(converter['chr1'][1000000])  # outputs: [('chr1', 1573376, '+')]

converter_pyliftover = LiftOver('chm13v2-grch38.chain.gz')
print(converter_pyliftover.convert_coordinate('chr1', 1000000, '+')) # outputs: [('chr1', 1565496, '+', 255)]

What's the reason behind this behaviour, and which output is correct based on the chain file?

The UCSC URL for liftover chain files appears to have changed, from cse.ucsc.edu -> .soe.ucsc.edu

Old (you get 403 response):
https://hgdownload.cse.ucsc.edu/goldenPath/hg19/liftOver/hg19ToHg38.over.chain.gz

New:
https://hgdownload.soe.ucsc.edu/goldenpath/hg19/liftOver/hg19ToHg38.over.chain.gz

This utility does not work currently, due to the liftover files being unfetchable

When running as part of a (web) service the library may fail due the default for the temporary directory. E.g. when called from inside /Library/WebServer/cgi-bin/bycon/beaconServer/aggregator.py the converter fails on os.mkdir(cache)
=> Permission denied: '/Library/WebServer/.liftover'

A solution would be to provide an optional path for the cache... For now solved by creating the dir w/ required permissions.

Hi, @jeremymcrae
I am glad to see your pythonic liftover tools, but when I try to liftover chr7:61657308 from hg19 to hg38, the tools give different results. Can you check the difference here?

Python version

>>> converter = get_lifter('hg19', 'hg38')
>>> converter.query("chr7", 61657308)
[('chr7', 62029292, '-')]

>>> converter = ChainFile('~/hg19ToHg38.over.chain', 'hg19', 'hg38')
>>> converter.query("chr7", 61657308)

# it returns chr7:62029292 (hg38version)

R version

library(GenomicRanges)
library(rtracklayer)
chr<-7;
start<-61657308
chain.file.path<-'~/hg19ToHg38.over.chain'
## convert to Genomic Ranges
example.37.gr<-GRanges(
        seqname=Rle(paste("chr",chr,sep="")),
        ranges=IRanges(start=start,end=start)) 
example.37.gr

c <- import.chain(chain.file.path) ## e.g. hg19ToHg18.over.chain
example.38.gr<-unlist(liftOver(example.37.gr,c))  
example.38.gr

# it returns chr7:62029294 (hg38version)

Best,
xiucz

Using provided pip command the command window throws following error:

Building wheels for collected packages: liftover
  Building wheel for liftover (PEP 517) ... error
    ERROR: Command errored out with exit status 1:                                                                                                                
    command: 'C:\Users\TheKursta\anaconda3\envs\bioinfo\python.exe' 'C:\Users\TheKursta\anaconda3\envs\bioinfo\lib\site- 
    packages\pip\_vendor\pep517\_in_process.py' build_wheel 'C:\Users\THEKUR~1\AppData\Local\Temp\tmpx4i6tpd4'
   cwd: C:\Users\TheKursta\AppData\Local\Temp\pip-install-bb5j7_11\liftover_39538f844b6d4b05ad7649ef4a66117e
   Complete output (12 lines):                                                                                                                                  
   running bdist_wheel                                                                                                                                          
   running build                                                                                                                                                
   running build_py                                                                                                                                             
   creating build                                                                                                                                               
   creating build\lib.win-amd64-3.8                                                                                                                            
   creating build\lib.win-amd64-3.8\liftover                                                                                                                    
   copying liftover\download_file.py -> build\lib.win-amd64-3.8\liftover                                                                                        
   copying liftover\lifter.py -> build\lib.win-amd64-3.8\liftover                                                                                               
   copying liftover\__init__.py -> build\lib.win-amd64-3.8\liftover                                                                                             
   running build_ext                                                                                                                                            
   building 'liftover.chain_file' extension                                                                                                                     
   error: Don't know how to compile src/gzstream/gzstream.C                                                                                                     
    ----------------------------------------                                                                                                                     
    ERROR: Failed building wheel for liftover                                                                                                                 
    Failed to build liftover                                                                                                                                     
    ERROR: Could not build wheels for liftover which use PEP 517 and cannot be installed directly   

assuming liftover is not an option?

It took me hours to figure out the way to install liftover. I just write it here in case someone has the same problem.

OS: centOS6

I got this while I was trying to install lifterover using 'pip install liftover':
Building wheels for collected packages: liftover
Building wheel for liftover (PEP 517) ... error
ERROR: Command errored out with exit status 1:
command: /gs/gsfs0/users/yizhao/miniconda2/bin/python /gs/gsfs0/users/yizhao/miniconda2/lib/python2.7/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /tmp/tmpMBscfO
cwd: /tmp/pip-install-HWwAKR/liftover
Complete output (19 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/liftover
copying liftover/lifter.py -> build/lib.linux-x86_64-2.7/liftover
copying liftover/init.py -> build/lib.linux-x86_64-2.7/liftover
copying liftover/download_file.py -> build/lib.linux-x86_64-2.7/liftover
running build_ext
building 'liftover.chain_file' extension
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/liftover
creating build/temp.linux-x86_64-2.7/src
creating build/temp.linux-x86_64-2.7/src/gzstream
gcc -pthread -B /gs/gsfs0/users/yizhao/miniconda2/compiler_compat -Wl,--sysroot=/ -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -Isrc/ -Isrc/gzstream/ -Isrc/intervaltree/ -I/gs/gsfs0/users/yizhao/miniconda2/include/python2.7 -c liftover/chain_file.cpp -o build/temp.linux-x86_64-2.7/liftover/chain_file.o -std=c++11
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: error: unrecognized command line option "-std=c++11"
error: command 'gcc' failed with exit status 1

ERROR: Failed building wheel for liftover
Running setup.py clean for liftover
Failed to build liftover
ERROR: Could not build wheels for liftover which use PEP 517 and cannot be installed directly

So I check my gcc version is 4.4.7. And google told me to use c++11 it need to be at lease 4.7.x. After I upgraded my gcc to 4.7.4, it was installed successfully.

Hi,

Thanks for the development of this package ! It's working quite well !

It's more of a question rather than an issue.

I wanted to know if there was a way to convert a pair of coordinates like in the liftOver standalone which is taking a bed file as input with a start and stop coordinates along with the strand information ?

Thanks for you help !

Trying to run the example in your "Usage" section of documentation, script fails on the first line:

$ pip3 freeze | grep liftover
liftover==1.1.10

$ cat liftover.py
#!/usr/bin/env python3
from liftover import get_lifter

error:

$ ./liftover.py
Traceback (most recent call last):
  File "./liftover.py", line 3, in <module>
    from liftover import get_lifter
  File "./liftover.py", line 3, in <module>
    from liftover import get_lifter
ImportError: cannot import name 'get_lifter'

Hi Jeremy,

I successfully installed the package, then when I tried to run the example code, I happened to have this error

Traceback (most recent call last):

File "liftover/chain_file.pyx", line 72, in liftover.chain_file.ChainFile.getitem

KeyError: '1'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/Users/yuchen/Desktop/CI-RA/Liftover genome version/liftover_genome_DocFile.py", line 26, in
converter[chrom][pos]

File "liftover/chain_file.pyx", line 74, in liftover.chain_file.ChainFile.getitem

KeyError: 'chr1'

Is there any idea why is error appear and how may I fix this? Thanks for your time and help.

Although python2 reached the end of its life on Jan 1st, it is still the only support version for several classic bio packages. I tried to install liftover (by both pip install liftover and by python setup.py build && python setup.py install) under python2.7.18 with cython==0.29.21 but it raised importError: No module named chain_file error. I wonder if this fast liftover package is designed for python3+ support?

Hi Jeremy,

I am wondering is there any trick part for PAR1 and PAR2 liftover from hg19 to hg38?

Thanks

Shicheng

import liftover
liftover.__version__

'1.1.13'

lifter = liftover.get_lifter("hg19", "hg38")
def _lift(row):
    chrom = row[0]

    print("Input chrom: {}".format(chrom))
    print("Input position: {}".format(row[1]))
    
    print(lifter[chrom][row[1]], end="\n\n")

[out]:

Input chrom: chr1
Input position: 1583534
[('chr1', 1715324, '+')]

Input chrom: chr1
Input position: 1583534
[('chr1', 1715324, '+')]

Input chrom: chr1
Input position: 1583627
[('chr1', 1715417, '+')]

Input chrom: chr1
Input position: 1583654
[]

You can see that the up until the final input position, the output of the lifter is as expected. However, this position returns an empty list. Is there any meaning to infer from this? Or am I perhaps doing something else that the program wasn't expecting? Thanks for any help you may be able to provide!

Use pip to install:

$ pip install liftover
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting liftover
  Using cached https://pypi.tuna.tsinghua.edu.cn/packages/72/fe/2c951c8f118a492ecbf68efe928b1633c57b4ac94d4f7ae0aab7da130f8f/liftover-1.1.12.tar.gz (185 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: requests in ./miniconda3/lib/python3.9/site-packages (from liftover) (2.25.1)
Requirement already satisfied: certifi>=2017.4.17 in ./miniconda3/lib/python3.9/site-packages (from requests->liftover) (2021.10.8)
Requirement already satisfied: chardet<5,>=3.0.2 in ./miniconda3/lib/python3.9/site-packages (from requests->liftover) (4.0.0)
Requirement already satisfied: idna<3,>=2.5 in ./miniconda3/lib/python3.9/site-packages (from requests->liftover) (2.10)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in ./miniconda3/lib/python3.9/site-packages (from requests->liftover) (1.26.6)
Building wheels for collected packages: liftover
  Building wheel for liftover (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/yangjk/miniconda3/bin/python /home/yangjk/miniconda3/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmphabyp249
       cwd: /tmp/pip-install-2b2tklwt/liftover_8134422265364a15a21cd9526c4bc2f5
  Complete output (7 lines):
  running bdist_wheel
  running build
  running build_py
  running build_ext
  building 'liftover.chain_file' extension
  gcc: error: src/gzstream/gzstream.C: 没有那个文件或目录
  error: command '/usr/bin/gcc' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for liftover
Failed to build liftover
ERROR: Could not build wheels for liftover which use PEP 517 and cannot be installed directly

It says the package doesn't have gzstream.C.

As the repo has gzstream.C in submodule in src/gzstream directory so I moved to install from source.

$ git clone https://github.com/jeremymcrae/liftover.git
$ cd liftover
$ ls src/gzstream/

Sadly, no file in this directory.

Adding an argument in git will help.

$ git clone https://github.com/jeremymcrae/liftover.git --recurse-submodules
$ cd liftover
$ ls src/gzstream/
COPYING.LIB  index.html  README         test_gzip.C
gzstream.C   logo.gif    README.md      version
gzstream.h   Makefile    test_gunzip.C

And the installation is a success.

$ python setup.py install

I think all these problems are caused by the submodules like gzstream in this repo and I recommend to handle these carefully.

Hi, thanks for developing such a useful package. Could you support batch processing to convert many coordinates at the same time? Doing for-loop in Python to convert many is very slow.

Something like

chrom_list = ['chr1', 'chr2', 'chr1']
pos_list = [123123123, 234534, 34623452]
converter.convert_coordinate_batch(chrom_list, pos_list)