gen2brain / go-unarr Goto Github PK

There is a vulnerability in unarr, which will lead to path traversal vulnerability
Go unarr does not check the contents of the archive.

Exploit process

1. An attacker can construct a malicious tar package (or any compressed archive file).
   As shown in the figure below, obviously, this will not succeed under the tar command, because the tar command fixes the vulnerability.
   

2. The victim uses go unarr to unzip the archive
   As shown in the figure below, path traversal occurs during go unarr decompression, and we upload the file to the.. / directory
   

3. By triggering the path traversal vulnerability, an attacker can store any file in any privileged place (which means that rce can be caused under root privileges)

Two things:

1. Cross-Platform Releases Now Possible

You can now cross-build Github releases from MacOS using goreleaser (as described in the latest PR).

If you get a Github Access Token you can publish versioned builds directly to github:

git tag -a v1.0.0 -m "First CLI release"
git push origin v1.0.0

export GITHUB_TOKEN=xxxxxx
goreleaser

2. Binary name change

I'd like to add this to https://webinstall.dev as a cross-platform method of un7z.

There are already many, many, many easily available cross-platform tools, such as tar, unzip, and arc, for the "normal" package and compression formats.

There are ZERO easy-to-install cross-platform CLI tools for un7z.

1. Would it be alright if I created a duplicate cmd/un7z/un7z.go (or replaced the existing cmd/un7z) for the purposes of making it easier to identify as the "correct" solution that people are looking for?
2. Would you be willing (and able) to publish cross-platform binaries for un7z to github releases with goreleaser?

when compiling a project which depends on go-unarr package on centos/redhat 6 the following errors are thrown:

/go/pkg/mod/github.com/gen2brain/[email protected]/external/lzma/Types.h:55: note: previous declaration of 'Byte' was here
In file included from /go/pkg/mod/github.com/gen2brain/[email protected]/external/bzip2/blocksort.c:22,
                 from /go/pkg/mod/github.com/gen2brain/[email protected]/unarr_cgo.go:21:
/go/pkg/mod/github.com/gen2brain/[email protected]/external/bzip2/bzlib_private.h:44: error: redefinition of typedef 'Int32'
/go/pkg/mod/github.com/gen2brain/[email protected]/external/lzma/Types.h:63: note: previous declaration of 'Int32' was here
/go/pkg/mod/github.com/gen2brain/[email protected]/external/bzip2/bzlib_private.h:45: error: redefinition of typedef 'UInt32'
/go/pkg/mod/github.com/gen2brain/[email protected]/external/lzma/Types.h:64: note: previous declaration of 'UInt32' was here
/go/pkg/mod/github.com/gen2brain/[email protected]/external/bzip2/bzlib_private.h:46: error: redefinition of typedef 'Int16'
/go/pkg/mod/github.com/gen2brain/[email protected]/external/lzma/Types.h:56: note: previous declaration of 'Int16' was here
/go/pkg/mod/github.com/gen2brain/[email protected]/external/bzip2/bzlib_private.h:47: error: redefinition of typedef 'UInt16'
/go/pkg/mod/github.com/gen2brain/[email protected]/external/lzma/Types.h:57: note: previous declaration of 'UInt16' was here```

wonko@deepthought:~/unarr$ ls -l
total 2595896
-rw-r--r-- 1 wonko wonko 2658193139 Sep 17 22:30 unarr-Corrupt_file.mkv
-rw-r--r-- 1 wonko wonko 2658193139 Sep 18 10:22 unrar-e-Working_file.mkv
wonko@deepthought:~/unarr$ md5sum *
8f739c886345d1ab0e90e1c784270bac  unarr-Corrupt_file.mkv
4109ac34d20fe03494e12cdce24f270b  unrar-e-Working_file.mkv
wonko@deepthought:~/unarr$

From here: https://github.com/bhechinger/tv/blob/master/donescript/main.go#L63

The version of go-unarr I have is: 6fec559

go 1.8.3 on Ubuntu 17.04

Doesn't open winRar archive with .rar extension

About inside of file:
Tested it with any file, including empty txt file

Returned error No valid RAR archive

test.zip
test.txt

What i've done:
Literally the first example of opening the archive

        src := `D:\test.rar`
        r, err := unarr.NewArchive(src)
	if err != nil {
		return err //<-- error here
	}
	defer r.Close()

UPD: Увидел в "более верхнем уровне" github issue, что на данный момент не поддерживается последний формат rar, понял-принял. Прошу прощения за нынешние неудобства
/
UPD: Saw in "upstream" github issue that rar 5.0 format isn't available right now - sorry about the issue

When compiling with newest apple clang (14.0.3) I now get the warning :
warning: a function declaration without a prototype is deprecated in all versions of C and is not supported in C2x [-Wdeprecated-non-prototype]
on zlib/adler32.c, zlib/crc32.c, zlib/inffast.c, zlib/inflate.c, zlib/inftrees.c, zlib/zutil.c.

My guess is the embedded zlib will need .h files in the near future? , but I am no C expert...

Have you considered supporting GBK encoding?

Note that I'm using a very large file: pwned-passwords-sha1-ordered-by-count-v4.7z

Example:

  x := 100
  a, _ := unarr.NewArchive("pwned-passwords-sha1-ordered-by-count-v4.7z");
  a.Entry()
  log.Println(a.Name())
  var total int                                                                              
  for {                                                                                      
    data := make([]byte, 100)                                                                
    n, err := a.Read(data)                                                                   
    if err != nil && err != io.EOF {                                                         
      log.Fatalf("Error reading: %s\n", err)                                                 
    }                                                                                        
    total += n                                                                               
    log.Printf("Read %d bytes (%d total)\n", n, total)                                       
    if err == io.EOF {                                                                       
      return                                                                      
    }                                                                                        
  }

prints:

pwned-passwords-sha1-ordered-by-count-v4.txt
Read 100 bytes
EOF

And if you change the value of X to any other number (10, 1024, 99999) you only get one "Read X bytes" line, followed by "EOF".

Absolute paths have been used, but as long as directory paths or file names contain Chinese, they always exit incorrectly.

unarr: File not found

`var 当前目录, _ = os.Getwd() //获取当前目录 C:\

var 压缩文件路径 = filepath.Join(当前目录, "vrt.7z") //合成 文件 文件路径
log.Println("压缩文件路径 ", 压缩文件路径)
//  打开存档
压缩文件缓存, err := unarr.NewArchive(压缩文件路径)
if err != nil {
	log.Fatal("打开 压缩文件存档 错误 ", err)
}
defer 压缩文件缓存.Close()`

error: ! tar.c:54: Unknown entry type '2', but the tar -xf c.tar is ok.

The example in the README shows:

a, err := unarr.NewArchive("test.7z")
if err != nil {
    panic(err)
}
defer a.Close()

I found out the hard way that this ends up leaking file descriptors, which was unexpected (to me). I had assumed that if NewArchive there would be no need to call Close. (that is, for example, how archive/zip.OpenReader behaves)

Looking through the code, the fd returned by C.ar_open_file(p) is created but not cleaned if a.open fails.

Is this expected behavior? If so, it would be helpful to update the docs. If not, a.stream should probably be closed if a.open fails`

I would be happy to submit a PR for either.

i have some necessary empty floders in the 7z compressed package, but i can't decompress them, they are not recognized
so, how can i do?

centos7.9
go version go1.22.2 linux/amd64

$ go install github.com/gen2brain/go-unarr/cmd/unarr@latest

go: downloading github.com/gen2brain/go-unarr v0.2.0
# github.com/gen2brain/go-unarr/unarrc
In file included from /opt/go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/bzip2_bzlib.go:4:0:
/opt/go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/external/bzip2/bzlib.c: In function ‘bzopen_or_bzdopen’:
/opt/go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/external/bzip2/bzlib.c:1431:7: warning: implicit declaration of function ‘fdopen’ [-Wimplicit-function-declaration]
       fp = fdopen(fd,mode2);
       ^
/opt/go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/external/bzip2/bzlib.c:1431:10: warning: assignment makes pointer from integer without a cast [enabled by default]
       fp = fdopen(fd,mode2);
          ^
cgo-gcc-prolog: At top level:
cc1: warning: unrecognized command line option "-Wno-typedef-redefinition" [enabled by default]

$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/4.8.5/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-linker-hash-style=gnu --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto --enable-plugin --enable-initfini-array --disable-libgcj --with-isl=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/isl-install --with-cloog=/builddir/build/BUILD/gcc-4.8.5-20150702/obj-x86_64-redhat-linux/cloog-install --enable-gnu-indirect-function --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
gcc version 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) 

works fine on v0.1.3 bit not with v0.1.4 or above

centos:7.9.2009 in docker
GOVERSION="go1.18.5"

# github.com/gen2brain/go-unarr/unarrc
	In file included from /go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/common_crc32.go:4:0:
	/go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/external/unarr/common/crc32.c: In function 'ar_crc32':
	/go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/external/unarr/common/crc32.c:25:9: error: 'for' loop initial declarations are only allowed in C99 mode
	         for (unsigned int i = 128; i; i >>= 1) {
	         ^
	/go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/external/unarr/common/crc32.c:25:9: note: use option -std=c99 or -std=gnu99 to compile your code
	/go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/external/unarr/common/crc32.c:27:13: error: 'for' loop initial declarations are only allowed in C99 mode
	             for (unsigned int j = 0; j < 256; j += 2 * i) {
	             ^
	/go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/external/unarr/common/crc32.c:32:27: error: redefinition of 'i'
	         for (unsigned int i = 0; i < 256; i++) {
	                           ^
	/go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/external/unarr/common/crc32.c:25:27: note: previous definition of 'i' was here
	         for (unsigned int i = 128; i; i >>= 1) {
	                           ^
	/go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/external/unarr/common/crc32.c:32:9: error: 'for' loop initial declarations are only allowed in C99 mode
	         for (unsigned int i = 0; i < 256; i++) {
	         ^
	/go/pkg/mod/github.com/gen2brain/[email protected]/unarrc/external/unarr/common/crc32.c:33:13: error: 'for' loop initial declarations are only allowed in C99 mode
	             for (unsigned int j = 1; j < 8; j++) {
	             ^

When extracting Simplified Chinese directory and file name, the name is confused .
解压简体中文目录和文件名时，名字错乱。

Hi, i try to get sub file name from 7z file, which compress by "7z a -m=0". and it get the wrong filename

Go version: 1.11
Platform: Windows 10

go-unarr installed with the following command:

go get github.com/gen2brain/go-unarr

Trying to extract data from a .7z archive.
The archive contains exactly one file. However, go-unarr can't find any entries, e.g. a.List() returns an empty slice and a.Entry() returns io.EOF

My code:

package main

import (
    "github.com/gen2brain/go-unarr"
    "fmt"
    "io"
)

func main() {
	a, err := unarr.NewArchive("something.7z")
	if err != nil {
		panic(err)
	}

	defer a.Close()

	err = a.Entry()
	if err != nil {
		panic(err)
	}

	list, err := a.List()
	if err != nil {
		panic(err)
	}
	fmt.Println(list)

	err = a.Entry()
	if err != nil {
		if err == io.EOF {
			fmt.Println("No files found")
			return
		}
		panic(err)
	}

	data, err := a.ReadAll()
	fmt.Println(len(data))
}

This prints

[]
No files found

But should print

[something.txt]
16

Base64 encoded contents of something.7z:

N3q8ryccAASTCs7jFAAAAAAAAABiAAAAAAAAAF5649QBAA/vu79oZWxsbywgd29ybGQhAAEEBgABCRQABwsBAAEhIQEADBAACAoBuW/kcgAABQEZDAAAAAAAAAAAAAAAABEdAHMAbwBtAGUAdABoAGkAbgBnAC4AdAB4AHQAAAAUCgEAL22u3BFM1AEVBgEAIAAAAAAA

Hi! I try to extract rar file but it's present this error:

! rar.c:214: RAR 5 format isn't supported

Use dd create 1GB 10GB 100GB file
dd if=/dev/random of=data1GB.bin bs=40m count=25
dd if=/dev/random of=data10GB.bin bs=400m count=25
dd if=/dev/random of=data100GB.bin bs=4000m count=25
and use command '7z a' archive to data1GB.7z data10GB.7z data100GB.7z
When un7zip size of 100GB.7z and 10GB.7z files report an error ：'unarr: No valid RAR, ZIP, 7Z or TAR archive'
But, un7zip size under 1GB file successfully!!!

Last，I test 2GB / 3GB / 4GB /5GB ,only 2GB file success, and above 2GB file failed!

OS : macOS 12.4
GOVERSION="go1.18.3"

RT

I see this repo has tag 0.1.0. However, that tag is not following Go Modules semantic versioning rules, since it is missing the v prefix. The Go Modules compatible version should look like v0.1.0 (Source). Without that, users need to rely on commit hashes pseudo-versions to resolve this module.

Can you please add a Go Modules compatible version so we can resolve actual release tags instead of commit hashes pseudo versions?

Hi there, may I know if your library supports to decompress the 7z archives with password? Normally 7z archives are able to compress file and filename with password, however when I use your library to decompress a 7z archives with password, it reports an error as below

unarr ./encrypt_file_and_name.7z ./encrypt_file_and_name
! _7z.c:186: Invalid 7z archive (failed with error 4)
error: unarr: No valid RAR, ZIP, 7Z or TAR archive

plus it does works when i decompress the same content without a password though.

Hi,
I'm using go 1.13.4.
When trying to open 7z file with NewArchive the app colapse with runtime.sigpanic()

fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0xb01dfacedebac1e pc=0x7fff72777476]

runtime stack:
runtime.throw(0x53f81e8, 0x2a)
	/usr/local/Cellar/go/1.13.4/libexec/src/runtime/panic.go:774 +0x72
runtime.sigpanic()
	/usr/local/Cellar/go/1.13.4/libexec/src/runtime/signal_unix.go:378 +0x47c

goroutine 72 [syscall]:
runtime.cgocall(0x4fe9770, 0xc000c85a28, 0xc000000000)
	/usr/local/Cellar/go/1.13.4/libexec/src/runtime/cgocall.go:128 +0x5b fp=0xc000c859f8 sp=0xc000c859c0 pc=0x400520b
github.com/gen2brain/go-unarr._Cfunc_ar_open_7z_archive(0xa700630, 0x0)
	_cgo_gotypes.go:194 +0x4a fp=0xc000c85a28 sp=0xc000c859f8 pc=0x41673ba
github.com/gen2brain/go-unarr.(*Archive).open.func3(0xc000346650, 0x0)
	pkg/mod/github.com/gen2brain/[email protected]/unarr.go:81 +0x5e fp=0xc000c85a68 sp=0xc000c85a28 pc=0x4168cde
github.com/gen2brain/go-unarr.(*Archive).open(0xc000346650, 0xa700630, 0xa7005d0)
	pkg/mod/github.com/gen2brain/[email protected]/unarr.go:81 +0x107 fp=0xc000c85a90 sp=0xc000c85a68 pc=0x4167cb7
github.com/gen2brain/go-unarr.NewArchive(0xc0000b06c0, 0x55, 0xc000346650, 0x0, 0x0)
	pkg/mod/github.com/gen2brain/[email protected]/unarr.go:41 +0xe0 fp=0xc000c85af8 sp=0xc000c85a90 pc=0x4167b10

I downgraded to go 1.12.12 and it works fine.
This led me to the conclusion it doesn't work with go 1.13.

I know the password for this package. How do I extract a 7z file with a password?

how to decompress multi-volumes 7z file like below?

a.7z.001
a.7z.002
a.7z.003
a.7z.004

The reason that I'm using this is for a cross-platform un7z, which has not been implemented in a language that can be easily cross-compiled.

Go has builtin natives for:

• zip
• tar
• gzip

And other native Go code for most other formats:

• github.com/mholt/archiver (brotli, snappy, xz, rar etc)
  • bzip2
  • rardecode
  • pgzip
  • brotli
  • ...
• webinstall.dev/arc (installer for archiver ^^)

Is there are reason that all of these already-go-native formats are included in this project?

https://github.com/gen2brain/go-unarr/blob/master/unarr.go#L50

In this line, you're using cap over len, which seems wrong to me. Assuming you pass a smaller slice s with len(s) < cap(s), and an archive is only valid within the range of the new slice, isn't this going to cause problems?