py7zlib fails with `IndexError: list index out of range` when opening specific 7z archive about pylzma HOT 13 CLOSED

The example file seems to be built with -m0=LZMA2:d23 option, but it cannot be reproduced with another files.

Type = 7z
Physical Size = 1161371
Headers Size = 5574
Method = LZMA2:23
Solid = +
Blocks = 1

from pylzma.

Could you please provide a small sample file containing only a "Hello world!" textfile?

from pylzma.

I cannot reproduce it with small files. It is a smallest file in which I observed same error.

The file has 418 files, 14 folders. compressed 1155797
When run it with PyCharm debugger, I got

on p7zlib.py: 884

• fidx: 1
• len(folders): 1
• len(subinfo.unpackstreams): 1
• subinfo.unpackstreams[0]: 417
• info['filename'] = 5.9.7/gcc_64/lib/libQt5XmlPatterns.prl as same as a last file shown when running 7zr l command that file size is 1088
• obidx: 417
• packsizes[0]: 1155797
• streams.packinfo.numstreams: 1
• streams.packinfo.packpos: 1155797
• streams.packinfo.packsizes[0]: 5505
• unpackinfo.folders[0].solid: False
• len(unpacksizes[]) : 417

It seems a problem may happen in last file in an archive.

from pylzma.

Could you please provide a small sample file containing only a "Hello world!" textfile?

test with "Hello World!" data passed.

    def test_github_59(self):
        fp = self._open_file(os.path.join(ROOT, 'data', 'github_59_small.7z'), 'rb')
        archive = Archive7z(fp)
        self._test_decode_all(archive)

a.zip

from pylzma.

Generating random 418 files become no luck, too.

import os
import random

os.mkdir('github_59')
for i in range(13):
    os.mkdir('github_59/dir%d' % i)
for i in range(418):
    with open('github_59/dir1/%d'% i, 'wb') as f:
        for j in range(random.randrange(1024)):
            f.write(random.randrange(65535).to_bytes(2, 'big'))
os.system('7z a -m0=LZMA2:d23 github_59_s.7z github_59/')

Path = github_59_s.7z
Type = 7z
Physical Size = 426246
Headers Size = 2991
Method = LZMA2:19
Solid = +
Blocks = 1

It is difficult to make a small file to reproduce the issue. Any suggenstion?

from pylzma.

I cannot reproduce it with small files. It is a smallest file in which I observed same error.

Feel free to send me the file by mail, you can find the address in my profile.

from pylzma.

Could you wget https://download.qt.io/online/qtsdkrepository/linux_x64/desktop/qt5_597/qt.qt5.597.gcc_64/5.9.7-0-201810181452qtxmlpatterns-Linux-RHEL_7_4-GCC-Linux-RHEL_7_4-X86_64.7z

from pylzma.

@fancycode I just ran into the same issue at my end. I looked around a little bit and saw that commit 9122f07 had failed build, while subsequent commits passed all the builds. As I am not familiar with the CI process, I am not sure if this means that the failed build was addressed by subsequent ones and therefore, the fix was still shipped eventually. However, a quick inspection of the commit and a comparison with the pypi release version 0.5 seems to indicate that changes from this commit were not eventually shipped.

from pylzma.

@fancycode Okay, nevermind. I just found out that released version 0.5.0 has the codebase from commit 56a23b7 from way back in Sept 2018. Are there any plans for newer releases?

from pylzma.

I have the same issue. pylzma 0.5.0 installed with pip3 on Ubuntu Focal:

 echo 'from py7zlib import Archive7z; fp = open("recoll.7z", "rb");a = Archive7z(fp)'  |  python3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/py7zlib.py", line 884, in __init__
    folder = folders[fidx]
IndexError: list index out of range

The archive has source files for the open source project recoll in it. You can download it from: https://www.lesbonscomptes.com/recoll/files/recoll.7z

from pylzma.

@medoc92 you can see alternative 7z library that can extract the file I reported here.

from pylzma.

@miurahr the problem is that I need to read the data to memory, and I get the impression that the py7zr memory only works for extracting to files ?

from pylzma.

@medoc92 you should put a question for another project, on that project.

from pylzma.