ossystems / compress-tools-rs Goto Github PK

View Code? Open in Web Editor NEW

76.0 12.0 27.0 235 KB

A Swiss Army Knife for handling compressed data in Rust

License: Apache License 2.0

Rust 95.64% Nix 1.40% Shell 2.96%

rust compression libarchive iot embedded-linux

compress-tools-rs's People

Contributors

Stargazers

Watchers

compress-tools-rs's Issues

Panic over `libarchive_seek_callback` when using compress-tools with tokio

I am trying to list content from this
test.zip. However, it gives me error like this

internal error: entered unreachable code: We need to use libarchive_seek_callback() underlying.
thread 'tokio-runtime-worker' panicked at 'internal error: entered unreachable code: We need to use libarchive_seek_callback() underlying.'

Here is my Cargo.toml and code

#Cargo.toml

[dependencies]
compress-tools = { version = "0.14", features = ["tokio_support"] }
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

#[tokio::test]
async fn test_list() -> Result<()> {
    let test_zip = concat!(env!("CARGO_MANIFEST_DIR"), "/test_resources/test.zip");
    let zip_file = File::open(test_zip).await?;
    let content = compress_tools::tokio_support::list_archive_files(zip_file).await?;
    println!("content={:?}", content);
    Ok(())
}

Environment:

OS: macOS 13.1
libarchive: 3.6.2 (installed from brew)

Any help is appreciated

unsafe precondition(s) violated: slice::from_raw_parts requires the pointer to be aligned and non-null, and the total size of the slice not to exceed `isize::MAX`

compress-tools = "0.15.0"

    uncompress_archive_file(&mut source, &mut outfile, &internal)?;

     Running unittests src\main.rs (target\debug\deps\jetico_search-6aa345a38c78f12b.exe)
thread 'archives::rar::tests::test_parse_file_from_archive' panicked at library\core\src\panicking.rs:156:5:
unsafe precondition(s) violated: slice::from_raw_parts requires the pointer to be aligned and non-null, and the total size of the slice not to exceed `isize::MAX`
stack backtrace:
   0: std::panicking::begin_panic_handler
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\std\src\panicking.rs:645
   1: core::panicking::panic_nounwind_fmt::runtime
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\core\src\panicking.rs:110
   2: core::panicking::panic_nounwind_fmt
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\core\src\panicking.rs:123
   3: core::panicking::panic_nounwind
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\core\src\panicking.rs:156
   4: core::slice::raw::from_raw_parts::precondition_check
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\core\src\intrinsics.rs:2799
   5: core::slice::raw::from_raw_parts<u8>
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\core\src\slice\raw.rs:98
   6: compress_tools::libarchive_write_data_block<ref_mut$<std::fs::File> >
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\compress-tools-0.15.0\src\lib.rs:621
   7: compress_tools::uncompress_archive_file_with_encoding::closure$0<ref_mut$<std::fs::File>,ref_mut$<std::fs::File> >
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\compress-tools-0.15.0\src\lib.rs:367
   8: compress_tools::run_with_archive::closure$0<compress_tools::uncompress_archive_file_with_encoding::closure_env$0<ref_mut$<std::fs::File>,ref_mut$<std::fs::File> >,ref_mut$<std::fs::File>,usize>
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\compress-tools-0.15.0\src\lib.rs:468
   9: compress_tools::run_with_archive<compress_tools::uncompress_archive_file_with_encoding::closure_env$0<ref_mut$<std::fs::File>,ref_mut$<std::fs::File> >,ref_mut$<std::fs::File>,usize>
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\compress-tools-0.15.0\src\lib.rs:409
  10: compress_tools::uncompress_archive_file_with_encoding<ref_mut$<std::fs::File>,ref_mut$<std::fs::File> >
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\compress-tools-0.15.0\src\lib.rs:343
  11: compress_tools::uncompress_archive_file<ref_mut$<std::fs::File>,ref_mut$<std::fs::File> >
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\compress-tools-0.15.0\src\lib.rs:395
  12: jetico_search::archives::rar::impl$0::parse_file_from_archive::async_fn$0
             at .\src\archives\rar.rs:47
  13: jetico_search::archives::rar::tests::test_parse_file_from_archive::async_block$0
             at .\src\archives\rar.rs:100
  14: core::future::future::impl$1::poll<ref_mut$<dyn$<core::future::future::Future<assoc$<Output,tuple$<> > > > > >
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\core\src\future\future.rs:123
  15: core::future::future::impl$1::poll<ref_mut$<core::pin::Pin<ref_mut$<dyn$<core::future::future::Future<assoc$<Output,tuple$<> > > > > > > >
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\core\src\future\future.rs:123
  16: tokio::runtime::scheduler::current_thread::impl$8::block_on::closure$0::closure$0::closure$0<core::pin::Pin<ref_mut$<core::pin::Pin<ref_mut$<dyn$<core::future::future::Future<assoc$<Output,tuple$<> > > > > > > > >
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.32.0\src\runtime\scheduler\current_thread\mod.rs:665
  17: tokio::runtime::coop::with_budget
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.32.0\src\runtime\coop.rs:107
  18: tokio::runtime::coop::budget
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.32.0\src\runtime\coop.rs:73
  19: tokio::runtime::scheduler::current_thread::impl$8::block_on::closure$0::closure$0<core::pin::Pin<ref_mut$<core::pin::Pin<ref_mut$<dyn$<core::future::future::Future<assoc$<Output,tuple$<> > > > > > > > >
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.32.0\src\runtime\scheduler\current_thread\mod.rs:665
  20: tokio::runtime::scheduler::current_thread::Context::enter<enum2$<core::task::poll::Poll<tuple$<> > >,tokio::runtime::scheduler::current_thread::impl$8::block_on::closure$0::closure_env$0<core::pin::Pin<ref_mut$<core::pin::Pin<ref_mut$<dyn$<core::future::f
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.32.0\src\runtime\scheduler\current_thread\mod.rs:410
  21: tokio::runtime::scheduler::current_thread::impl$8::block_on::closure$0<core::pin::Pin<ref_mut$<core::pin::Pin<ref_mut$<dyn$<core::future::future::Future<assoc$<Output,tuple$<> > > > > > > > >
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.32.0\src\runtime\scheduler\current_thread\mod.rs:664
  22: tokio::runtime::scheduler::current_thread::impl$8::enter::closure$0<tokio::runtime::scheduler::current_thread::impl$8::block_on::closure_env$0<core::pin::Pin<ref_mut$<core::pin::Pin<ref_mut$<dyn$<core::future::future::Future<assoc$<Output,tuple$<> > > > >
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.32.0\src\runtime\scheduler\current_thread\mod.rs:743
  23: tokio::runtime::context::scoped::Scoped<enum2$<tokio::runtime::scheduler::Context> >::set<enum2$<tokio::runtime::scheduler::Context>,tokio::runtime::scheduler::current_thread::impl$8::enter::closure_env$0<tokio::runtime::scheduler::current_thread::impl$8:
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.32.0\src\runtime\context\scoped.rs:40
  24: tokio::runtime::context::set_scheduler::closure$0<tuple$<alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core,alloc::alloc::Global>,enum2$<core::option::Option<tuple$<> > > >,tokio::runtime::scheduler::current_thread::impl$8::enter::closure_e
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.32.0\src\runtime\context.rs:176
  25: std::thread::local::LocalKey<tokio::runtime::context::Context>::try_with<tokio::runtime::context::Context,tokio::runtime::context::set_scheduler::closure_env$0<tuple$<alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core,alloc::alloc::Global>,
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\std\src\thread\local.rs:284
  26: std::thread::local::LocalKey<tokio::runtime::context::Context>::with<tokio::runtime::context::Context,tokio::runtime::context::set_scheduler::closure_env$0<tuple$<alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core,alloc::alloc::Global>,enum
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\std\src\thread\local.rs:260
  27: tokio::runtime::context::set_scheduler<tuple$<alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core,alloc::alloc::Global>,enum2$<core::option::Option<tuple$<> > > >,tokio::runtime::scheduler::current_thread::impl$8::enter::closure_env$0<tokio:
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.32.0\src\runtime\context.rs:176
  28: tokio::runtime::scheduler::current_thread::CoreGuard::enter<tokio::runtime::scheduler::current_thread::impl$8::block_on::closure_env$0<core::pin::Pin<ref_mut$<core::pin::Pin<ref_mut$<dyn$<core::future::future::Future<assoc$<Output,tuple$<> > > > > > > > >
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.32.0\src\runtime\scheduler\current_thread\mod.rs:743
  29: tokio::runtime::scheduler::current_thread::CoreGuard::block_on<core::pin::Pin<ref_mut$<core::pin::Pin<ref_mut$<dyn$<core::future::future::Future<assoc$<Output,tuple$<> > > > > > > > >
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.32.0\src\runtime\scheduler\current_thread\mod.rs:652
  30: tokio::runtime::scheduler::current_thread::impl$0::block_on::closure$0<core::pin::Pin<ref_mut$<dyn$<core::future::future::Future<assoc$<Output,tuple$<> > > > > > >
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.32.0\src\runtime\scheduler\current_thread\mod.rs:175
  31: tokio::runtime::context::runtime::enter_runtime<tokio::runtime::scheduler::current_thread::impl$0::block_on::closure_env$0<core::pin::Pin<ref_mut$<dyn$<core::future::future::Future<assoc$<Output,tuple$<> > > > > > >,tuple$<> >
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.32.0\src\runtime\context\runtime.rs:65
  32: tokio::runtime::scheduler::current_thread::CurrentThread::block_on<core::pin::Pin<ref_mut$<dyn$<core::future::future::Future<assoc$<Output,tuple$<> > > > > > >
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.32.0\src\runtime\scheduler\current_thread\mod.rs:167
  33: tokio::runtime::runtime::Runtime::block_on<core::pin::Pin<ref_mut$<dyn$<core::future::future::Future<assoc$<Output,tuple$<> > > > > > >
             at C:\Users\igumn\.cargo\registry\src\index.crates.io-6f17d22bba15001f\tokio-1.32.0\src\runtime\runtime.rs:347
  34: jetico_search::archives::rar::tests::test_parse_file_from_archive
             at .\src\archives\rar.rs:102
  35: jetico_search::archives::rar::tests::test_parse_file_from_archive::closure$0
             at .\src\archives\rar.rs:97
  36: core::ops::function::FnOnce::call_once<jetico_search::archives::rar::tests::test_parse_file_from_archive::closure_env$0,tuple$<> >
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\core\src\ops\function.rs:250
  37: core::ops::function::FnOnce::call_once
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\core\src\ops\function.rs:250
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
thread caused non-unwinding panic. aborting.
error: test failed, to rerun pass `-p jetico-search --bin jetico-search`

Caused by:
  process didn't exit successfully: `C:\Users\igumn\jetico-search\target\debug\deps\jetico_search-6aa345a38c78f12b.exe archives::rar::tests::test_parse_file_from_archive --format=json --exact -Z unstable-options --show-output` (exit code: 0xc0000409, STATUS_STACK_BUFFER_OVERRUN)
note: test exited abnormally; to see the full output pass --nocapture to the harness.
error: 1 target failed:
    `-p jetico-search --bin jetico-search`

Process finished with exit code 101

Cannot build on Windows, link.exe exits with error code 1102

Hi,

First off thanks for this crate.

So far I have the minimal example compiling under WSL2, but the same code fails to build with the following note:

= note: Creating library C:\Users\wanty\Documents\ssmm_rust\target\debug\deps\StarsectorModManager.lib and object C:\Users\wanty\Documents\ssmm_rust\target\debug\deps\StarsectorModManager.exp archive.lib(filter_fork_windows.c.obj) : error LNK2019: unresolved external symbol __imp_WaitForInputIdle referenced in function __archive_create_child libcrypto.lib(cryptlib.obj) : error LNK2019: unresolved external symbol __imp_GetProcessWindowStation referenced in function OPENSSL_isservice libcrypto.lib(cryptlib.obj) : error LNK2019: unresolved external symbol __imp_GetUserObjectInformationW referenced in function OPENSSL_isservice libcrypto.lib(cryptlib.obj) : error LNK2019: unresolved external symbol __imp_MessageBoxW referenced in function OPENSSL_showfatal libcrypto.lib(e_capi.obj) : error LNK2019: unresolved external symbol __imp_CertOpenStore referenced in function capi_list_certs libcrypto.lib(e_capi.obj) : error LNK2019: unresolved external symbol __imp_CertCloseStore referenced in function capi_find_key libcrypto.lib(e_capi.obj) : error LNK2019: unresolved external symbol __imp_CertEnumCertificatesInStore referenced in function capi_find_cert libcrypto.lib(e_capi.obj) : error LNK2019: unresolved external symbol __imp_CertFindCertificateInStore referenced in function capi_find_cert libcrypto.lib(e_capi.obj) : error LNK2019: unresolved external symbol __imp_CertDuplicateCertificateContext referenced in function capi_load_ssl_client_cert libcrypto.lib(e_capi.obj) : error LNK2019: unresolved external symbol __imp_CertFreeCertificateContext referenced in function capi_dsa_free libcrypto.lib(e_capi.obj) : error LNK2019: unresolved external symbol __imp_CertGetCertificateContextProperty referenced in function capi_cert_get_fname C:\Users\wanty\Documents\ssmm_rust\target\debug\deps\StarsectorModManager.exe : fatal error LNK1120: 11 unresolved externals

I have libarchive installed using vcpkg, using the triplet x86_64-windows-static-md.

I'm not sure if there's an issue surrounding how I've installed libarchive or something between that and trying to build my project.

If there's any other information you need please do let me know.

Error using `uncompress_archive_file` on 7zip archive

I'm getting this error when trying to extract a single file from a 7zip archive:

Error: Extraction("Current client reader does not support seeking a device")

That message originates from libarchive. There's a possibly-related issue on their tracker, but I tried patching seek capability into archive_read_open_FILE with no change.

I tried inspecting the compress-tools source, but couldn't identify the libarchive code path that triggers this error. Until I know that, I can't formulate a proper bug report for libarchive. Any guidance would be appreciated.

Entry information

It would be nice to be able to get information about entries, e.g. file type or the creation time.

7z uncompress_archive_file fail

When I process 7z files, some 7z files will have errors, but if I use the command to decompression, it is normal.

thread 'main' panicked at 'Failed to get the file: Extraction("Truncated 7-Zip file body")'

let written = uncompress_archive_file(&mut source, &mut target, "tree/branch2/leaf")
        .expect("Failed to get the file");

RUSTSEC-2020-0016: `net2` crate has been deprecated; use `socket2` instead

net2 crate has been deprecated; use socket2 instead

Details
Status	unmaintained
Package	`net2`
Version	`0.2.34`
URL	deprecrated/net2-rs@`3350e38`
Date	2020-05-01

The net2 crate has been deprecated
and users are encouraged to considered socket2 instead.

See advisory page for additional details.

Clearer errors/warnings when using list_archive_files followed by uncompress_archive_file

I'm new to rust so forgive me if this should have been assumed from the use of a mutable reference, but it took me a long time to realize that after using list_archive-files() you must rewind the file pointer in order to use uncompress_archive_file() (and I assume other decompression functions on the same file handle).

I assume calling list_archive_files() before another decompression function would be relatively common, so some documentation, warnings, or automatic seeking/rewinding would be appreciated.

Failed to decompress compressed files (tar.gz, zip) on Windows

When using uncompress_archive, it works well on Linux platforms but failed with the below error on Windows.

use compress_tools::{uncompress_archive, Ownership};
uncompress_archive(&download_file, &extract_dir, Ownership::Ignore)?;

Error: Extraction error: 'INTERNAL ERROR: Function 'archive_write_data_block' invoked with archive structure in state 'header', should be in state 'data''

Feature request: List files in an archive

I'd like to use this crate to show comic book archives in emulsion. To do this, I need to iterate over all the files in an archive. Would it be possible to add a function that returns all file names in the archive?

Absolute paths in archives will be extracted outside of destination directory

I have created a PR to fix this here #83

This is similar to #63

The behaviour in my fix (stripping the leading '/') matches the behaviour of GNU tar and BSD tar, which will print this message when unarchiving: tar: Removing leading '/' from member names

Extraction("Failed to set file flags") on libalpm's file db

Extraction("Failed to set file flags") gets returned whenever I try to extract Arch an linux's libalpm file db with uncompress_archive
Tried both Ownership::Ignore and Ownership::Preserve (desired behaviour is ignore).

This only happens specifically with .tar.gzip files.

Output of the file command on one of the target files: gzip compressed data, from Unix, original size modulo 2^32.

Files are generated with pacman -Fyb /target/dir/ and are located in /target/dir/sync/.
Tried to extract /target/dir/sync/* to /target/dir/extracted/* (of course without the *, it's just to represent the individual file names).

Filesystem in use: ext4 + luks;
Package manager libarchive version: 3.6.1-1;
OS: EndeavourOS (based on Arch Linux);

Both GNU tar and bsdtar don't present this problem but GNU tar presents an error regarding an unknown keyword SCHILY.fflags (Perhaps this is problem?)

UTF8Error when listing or uncompressing zip archive with utf-8 filename

I tried extracting this archive on Ubuntu 20.04.1 with both the apt libarchive-dev and manually compiled latest libarchive 3.5.1:

test.zip

use compress_tools::list_archive_files;
use std::fs::File;

fn main() {
    let file = File::open("test.zip").unwrap();
    for file_name in list_archive_files(&file).unwrap() {
        dbg!(&file_name);
    }
}

This returns:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Utf(Utf8Error { valid_up_to: 0, error_len: Some(1) })', src/main.rs:6:48

I tried a couple of the latest revisions of this library but it still fails on every one.

Output of locale:

LANG=en_US.UTF-8
LANGUAGE=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=en_US.UTF-8

Unfortunately, I'm getting a separate linking error trying to install compress-tools on Windows so I'm unable to test on that platform.

Proper way to include your code in my library?

Hi, I am building a similar library (inspired by yours but crafted to my own needs), and I am using one of your file locale.rs.

I am wondering:

If I can include the file in my library (using MIT License)
I did not make change to the file, which includes the Copyright comments. Is this enough? I have following lines at the beginning of the file:

//
// This file is taken from https://github.com/OSSystems/compress-tools-rs/blob/master/src/ffi/locale.rs
// All the credit goes to https://github.com/OSSystems
//
// Copyright (C) 2021 O.S. Systems Software LTDA
//
// SPDX-License-Identifier: MIT OR Apache-2.0
// ...

Please let me know. If this is inappropriate, I apologize and I will delete the file

Typo in the company name

There is definitely a typo in file headers and in the license:

Copyright (C) 2019, 2020 O.S. Systems Sofware LTDA

Iterate archive without reading all data.

The current archive iterator reads through all data in the archive when I'm only interested in very little of it.

It would be nice if each new file returned an iterator that you could optionally iterate to get the data.

Segment fault on Debian when listing archive files with Chinese characters

Segment fault happens on Debian when list_archive_files or list_archive_files_with_encoding is called for archives whose files contain Chinese characters.

The two functions work fine on macOS when listing files with or without Chinese characters.

Error message:

qemu: uncaught target signal 11 (Segmentation fault) - core dumped

I realized the problem when I put my binary into docker. I don't have a Debian environment except docker, so the reproduce steps are using docker. Sorry about that.

Reproduce:

Unzip compress_tools_test.zip
docker build . -t compress-tools-test
docker run --rm -it --entrypoint sh compress-tools-test
cargo run

The zip used for the test is included in the zip above, but if you want to download it separately, here it is: test.zip

RUSTSEC-2021-0153: `encoding` is unmaintained

encoding is unmaintained

Details
Status	unmaintained
Package	`encoding`
Version	`0.2.33`
URL	lifthrasiir/rust-encoding#127
Date	2021-12-05

Last release was on 2016-08-28. The issue inquiring as to the status of the crate has gone unanswered by the maintainer.

Possible alternatives

encoding_rs

See advisory page for additional details.

Bundle libarchive

Hey, there it would be great if bundled feature would be added to library so that it can compile libarchive itself as many rusqlite project does.

Cross compilation failed for linux

I have tried to do cross compilation for following distributions:

armv7-unknown-linux-musleabihf
arm-unknown-linux-musleabihf

But the compilation would fail at the link stage

Error message

  = note: /usr/local/bin/../lib/gcc/arm-linux-musleabihf/9.2.0/../../../../arm-linux-musleabihf/bin/ld: /usr/lib/x86_64-linux-gnu/libc.a: error adding symbols: file format not recognized
          collect2: error: ld returned 1 exit status

Reproduce

Download the reduced test project: cross-compiler-test.zip, and unzip it
Install cross-rs/cross
Open docker
Run cross build --release --target armv7-unknown-linux-musleabihf

RARv5 archives containing directories fail to decompress

Hi,

Currently, attempting to decompress a RARv5 archive containing directories fails with the error Can't decompress an entry marked as a directory.

Listing the contents of the archive works as expected however, as does decompressing an archive without directories. Testing with RARv4 archives shows no issues at all with file listing or decompression.

uncompress 7z failed in new libarchive version 13.5.2

panicked at 'Extraction("LZMA codec is unsupported")'

old libarchive version 13.3.3 it's ok.

Zip-slip vulnerability

uncompress_archive is vulnerable to zip-slip.

This can be verified with the test files in https://github.com/snyk/zip-slip-vulnerability/tree/master/archives. Using zip-slip.zip I was able to extract good.txt to my chosen directory but evil.txt was extracted to /tmp/.

This is usually worthy of security advisory.

Test fails

thread 'iterate_zip_with_cjk_pathname' panicked at library/core/src/panicking.rs:156:5:
unsafe precondition(s) violated: slice::from_raw_parts requires the pointer to be aligned and non-null, and the total size of the slice not to exceed `isize::MAX`
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
thread caused non-unwinding panic. aborting.
error: test failed, to rerun pass `--test integration_test`

Option to disable "raw" libarchive format

I'd like to use compress-tools-rs in a situation where I want to treat archive and non-archive files differently; enabling the "raw" format makes it effectively impossible to distinguish archives from non-archives and/or corrupt archives.

Support encryption?

I want to uncompress archive with password.
Is there a way to do it or is there any plan to do it?

Questions about some choices

I am reading the code, but have questions about some function choices, hoping that you can help me with.

Why archive_read_open instead of archive_read_open_filename? The second one seems much easier, and it handles the file reading and seeking for you. It simplifies the code as you don't need to create SeekableReadPipe etc.
Is there some reason on setting the value READER_BUFFER_SIZE to 16384? Or it is just some radom value that works great?

Thanks for your time

Uncompressing 7z archives with more than one file fails

Looks similar to #46 - this code will fail with 7z archive that contains two or more files

use compress_tools::*;
use std::fs::File;
use std::path::Path;

fn main() -> Result<()> {
    let args: Vec<String> = std::env::args().collect();
    if args.len() != 2 {
        panic!("Usage: {} <7z archive>", args[0]);
    }

    let mut source = File::open(&args[1])?;
    let dest = Path::new("/tmp/dest");
    uncompress_archive(&mut source, &dest, Ownership::Ignore)?;

    Ok(())
}

I prepared a repository which allows to easily reproduce the issue
https://github.com/muttleyxd/compress-tools-rs-7z-unpack-failure

This happens on Arch Linux, 7z version 17.03, libarchive version 3.5.1-1.

./create-7z-with-two-files.sh
+ rm two_files.7z 1 2
+ touch 1 2
+ 7z a two_files.7z 1 2

7-Zip [64] 17.03 : Copyright (c) 1999-2020 Igor Pavlov : 2017-08-28
p7zip Version 17.03 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,64 bits,32 CPUs x64)

Scanning the drive:
2 files, 0 bytes

Creating archive: two_files.7z

Items to compress: 2

    
Files read from disk: 0
Archive size: 118 bytes (1 KiB)
Everything is Ok
+ cargo build
    Finished dev [unoptimized + debuginfo] target(s) in 0.00s
+ cargo run -- ./two_files.7z
    Finished dev [unoptimized + debuginfo] target(s) in 0.00s
     Running `target/debug/rust_7z_unpack_investigation ./two_files.7z`
Error: Extraction("Current client reader does not support seeking a device")

Feature request: Compression of files and directories, eg. archive creation

This seems like the only viable crate that interfaces with libarchive. Are there any plans on adding compression of files and directories, eg. archive creation?

sub-directories as file names

compress-tools v0.13.0, libarchive v3.6.1

Example archive:
http://ftp.uni-erlangen.de/mirrors/aminet/demo/aga/Hardcore.lha

Test:

use compress_tools::*;
use std::fs::File;
use std::path::Path;

fn main() -> Result<()> {
    let mut source = File::open("Hardcore.lha")?;
    let dest = Path::new("/tmp/dest");
    uncompress_archive(&mut source, dest, Ownership::Ignore)?;
    Ok(())
}

Current result:
File "035.SDM" in root directory named "sdm-hardcore\datas\035.SDM"

$ ls /tmp/dest/
 File_Id.Diz   Hardcore.readme   sdm-hardcore  'sdm-hardcore\datas\035.SDM'

Expected result:
File "035.SDM" in directory "sdm-hardcore/datas"

$ ls /tmp/dest/
 File_Id.Diz   Hardcore.readme   sdm-hardcore

Cannot statically compile on Windows due to linking failure, exit code 1120

Attempting to compile my project or examples statically fails with error error: linking with 'link.exe' failed: exit code: 1120.

The following is also provided:

note:    Creating library C:\Users\wanty\Documents\compress-tools-rs-master\target\debug\examples\uncompress_tool.lib and object C:\Users\wanty\Documents\compress-tools-rs-master\target\debug\examples\uncompress_tool.exp
          archive.lib(filter_fork_windows.c.obj) : error LNK2019: unresolved external symbol __imp_WaitForInputIdle referenced in function __archive_create_child
          libcrypto.lib(cryptlib.obj) : error LNK2019: unresolved external symbol __imp_GetProcessWindowStation referenced in function OPENSSL_isservice
          libcrypto.lib(cryptlib.obj) : error LNK2019: unresolved external symbol __imp_GetUserObjectInformationW referenced in function OPENSSL_isservice
          libcrypto.lib(cryptlib.obj) : error LNK2019: unresolved external symbol __imp_MessageBoxW referenced in function OPENSSL_showfatal
          libcrypto.lib(e_capi.obj) : error LNK2019: unresolved external symbol __imp_CertOpenStore referenced in function capi_list_certs
          libcrypto.lib(e_capi.obj) : error LNK2019: unresolved external symbol __imp_CertCloseStore referenced in function capi_find_key
          libcrypto.lib(e_capi.obj) : error LNK2019: unresolved external symbol __imp_CertEnumCertificatesInStore referenced in function capi_find_cert
          libcrypto.lib(e_capi.obj) : error LNK2019: unresolved external symbol __imp_CertFindCertificateInStore referenced in function capi_find_cert
          libcrypto.lib(e_capi.obj) : error LNK2019: unresolved external symbol __imp_CertDuplicateCertificateContext referenced in function capi_load_ssl_client_cert
          libcrypto.lib(e_capi.obj) : error LNK2019: unresolved external symbol __imp_CertFreeCertificateContext referenced in function capi_dsa_free
          libcrypto.lib(e_capi.obj) : error LNK2019: unresolved external symbol __imp_CertGetCertificateContextProperty referenced in function capi_cert_get_fname
          C:\Users\wanty\Documents\compress-tools-rs-master\target\debug\examples\uncompress_tool.exe : fatal error LNK1120: 11 unresolved externals

This seems to be similar to previous issue #47.

Steps to reproduce:

Install libarchive with vcpkg using tuple "x64-windows-static"
Either set environment variable RUSTFLAGS='-C target-feature=+crt-static' OR add the following to the config file in the .cargo directory:

[target.x86_64-pc-windows-msvc]
rustflags = ["-C", "target-feature=+crt-static"]

The VCPKG_DYNAMIC environment variable must not be set/must be set to 0
(Possibly) Set the environment variable OPENSSL_ROOT_DIR="{vcpkg install directory}\installed\x64-windows-static"
Attempt to build compress-tools examples

Could `EntryFilterCallback` be a Fn trait to allow closures?

Consider the test iterate_archive_with_filter_name. If we move the expected filename (expected_name) on the filter to the outer scope, we get the compilation error:

#[test]
fn iterate_archive_with_filter_name() {
    let source = std::fs::File::open("tests/fixtures/tree.tar").unwrap();
    let expected_name = Some(OsStr::new("leaf"));

    let mut entries = Vec::new();
    for content in ArchiveIteratorBuilder::new(source)
        .filter(|name, _stat| Path::new(name).file_name() == expected_name)
        .build()
        .unwrap()
    {
        if let ArchiveContents::StartOfEntry(name, _stat) = content {
            entries.push(name);
        }
    }

    assert_eq!(
        entries,
        vec![
            "tree/branch1/leaf".to_string(),
            "tree/branch2/leaf".to_string(),
        ],
        "filtered file list inside the archive did not match"
    );
}

$ cargo test
   Compiling compress-tools v0.14.1
error[E0308]: mismatched types
   --> tests/integration_test.rs:738:17
    |
738 |         .filter(|name, _stat| Path::new(name).file_name() == expected_name)
    |          ------ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected fn pointer, found closure
    |          |
    |          arguments to this method are incorrect
    |
    = note: expected fn pointer `for<'a, 'b> fn(&'a str, &'b libc::stat) -> bool`
                  found closure `[closure@tests/integration_test.rs:738:17: 738:30]`
note: closures can only be coerced to `fn` types if they do not capture any variables
   --> tests/integration_test.rs:738:62
    |
738 |         .filter(|name, _stat| Path::new(name).file_name() == expected_name)
    |                                                              ^^^^^^^^^^^^^ `expected_name` captured here
note: method defined here
   --> /home/keysym/compress-tools-rs/src/iterator.rs:443:12
    |
443 |     pub fn filter(mut self, filter: EntryFilterCallback) -> ArchiveIteratorBuilder<R> {
    |            ^^^^^^

For more information about this error, try `rustc --explain E0308`.
error: could not compile `compress-tools` due to previous error

rustc version: 1.69.0

Is it possible to convert EntryFilterCallback to be a Fn trait so filter() accepts closures?

ossystems / compress-tools-rs Goto Github PK

compress-tools-rs's People

Contributors

Stargazers

Watchers

Forkers

compress-tools-rs's Issues

Possible alternatives

Error message

Reproduce

Recommend Projects

Recommend Topics

Recommend Org