hatching / sflock Goto Github PK

View Code? Open in Web Editor NEW

82.0 82.0 48.0 18.97 MB

Sample staging & detonation utility to be used in combination with Cuckoo Sandbox.

Python 70.93% JavaScript 0.06% PowerShell 0.04% HTML 28.77% VBScript 0.19%

sflock's People

Contributors

Stargazers

Watchers

sflock's Issues

Outdated zipjail binary

The compiled zipjail binary on the master branch is outdated (version 0.5 vs 0.5.2 from the tracy project master)
As a result, its syscall whitelist is missing getdents64 and getpid:
sflock master:

.rodata:000000000049B914 aGettimeofday   db 'gettimeofday',0     ; DATA XREF: .data:00000000006C81C0↓o
.rodata:000000000049B921 aStat64         db 'stat64',0           ; DATA XREF: .data:00000000006C81C8↓o
.rodata:000000000049B928 aSysDevicesSyst_1 db '/sys/devices/system/cpu',0

compiled tracy master:

.rodata:000000000049F0B4 aGettimeofday   db 'gettimeofday',0     ; DATA XREF: .data:00000000004CE1C0↓o
.rodata:000000000049F0C1 aStat64         db 'stat64',0           ; DATA XREF: .data:00000000004CE1C8↓o
.rodata:000000000049F0C8 aGetdents64     db 'getdents64',0       ; DATA XREF: .data:00000000004CE1D0↓o
.rodata:000000000049F0D3 aGetpid_0       db 'getpid',0           ; DATA XREF: .data:00000000004CE1D8↓o
.rodata:000000000049F0DA aSysDevicesSyst_1 db '/sys/devices/system/cpu',0

This will cause some archives to be incorrectly dropped: (unpacking an iso archive using 7zip)

ERROR:root:Blocked system call occurred during sandboxing!
ip=0x7fd816c6d07b sp=0x7fffbe193978 abi=0 nr=217 syscall=getdents64
Killing child 7064

Wrong package identified - ident package overwriting File init (pick) package within main.unpack()

I noticed some large ELF binaries having a package set as 'js' instead of 'generic' due to strings found within & that the ident.py identifiers check by looking across the entire file contents. As one mitigation method should the buffer/contents being checked be limited to 0x1000 to align with the check at the top of the identifier function (if not f.stream.read(0x1000):)?

Was the purpose of overwriting the package predominantly for archives? Is it worth placing a conditional within main.unpack() along the line of-
if f.type == "container": # or != "file" ?
ident(f)

Gzip Files requires native tooling

Is there a reason you're not using the gzip module to handle pure gzip file? You do that for .tar.gz, so it is confusing.

And if not, do you mind if I implement a pure python module to handle simple gzip files without requiring 7zip?

Zipbombs & maximum limit of files in archive

Properly handle zip bombs as well as archives with more than X selected files in it.
No need for Cuckoo to freeze on such inputs.

Supported test fails

This assertion throws an error when running the unit tests: https://github.com/hatching/sflock/blob/master/tests/test_main.py#L30

`> assert count == len(supported())
E AssertionError: assert 14 == 15
E + where 15 = len([b'.msg', b'.7z', b'.iso', b'.gzip', b'.lzh', b'.lha', ...])
E + where [b'.msg', b'.7z', b'.iso', b'.gzip', b'.lzh', b'.lha', ...] = supported()

tests/test_main.py:30: AssertionError`

The full string of supported() is: [b'.msg', b'.7z', b'.iso', b'.gzip', b'.lzh', b'.lha', b'.daa', b'.eml', b'.bup', b'.zip', b'.mso', b'.tar', b'.tar.gz', b'.tar.bz2', b'.pdf']

Accept in-memory files for extraction in .ace support

Might affect other unpackers as well, though.

zipjail `Detected potential out-of-path arbitrary overwrite!`

data/zipjail.elf 8091063456b4c1ef1771b88f39bd3fda4b25ccb4bbeafa538edcf4b18ed0f9e5 /tmp/ -- /usr/bin/7z x -mmt=off -o/tmp/ 8091063456b4c1ef1771b88f39bd3fda4b25ccb4bbeafa538edcf4b18ed0f9e5

7-Zip [64] 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,8 CPUs)

Processing archive: 8091063456b4c1ef1771b88f39bd3fda4b25ccb4bbeafa538edcf4b18ed0f9e5

Detected potential out-of-path arbitrary overwrite!
filepath=/tmp/stunnel dirpath=/tmp/
Killing child 7176

cabextract is already the newest version (1.6-1).
rar is already the newest version (2:5.3.b2-1).
unace-nonfree is already the newest version (2.5-8).
p7zip-full is already the newest version (9.20.1~dfsg.1-4.2ubuntu0.1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

SFlock (0.3.10)  - Sample staging and detonation utility
  INSTALLED: 0.3.10


from sflock import unpack
q = unpack('8091063456b4c1ef1771b88f39bd3fda4b25ccb4bbeafa538edcf4b18ed0f9e5')
q.children
[]
>>> q.mime
'application/x-7z-compressed'


7z x -mmt=off -o/tmp/ 8091063456b4c1ef1771b88f39bd3fda4b25ccb4bbeafa538edcf4b18ed0f9e5

7-Zip [64] 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,8 CPUs)

Processing archive: 8091063456b4c1ef1771b88f39bd3fda4b25ccb4bbeafa538edcf4b18ed0f9e5

Extracting  stunnel
Extracting  stunnel/bin
Extracting  stunnel/config
Extracting  stunnel/engines
Extracting  stunnel/bin/Microsoft.VC90.CRT.Manifest
Extracting  stunnel/config/ca-certs.pem
Extracting  stunnel/config/openssl.cnf
Extracting  stunnel/config/stunnel.conf
Extracting  stunnel/bin/libeay32.dll
Extracting  stunnel/bin/msvcr90.dll
Extracting  stunnel/bin/openssl.exe
Extracting  stunnel/bin/ssleay32.dll
Extracting  stunnel/bin/stunnel.exe
Extracting  stunnel/bin/tstunnel.exe
Extracting  stunnel/engines/capi.dll
Extracting  stunnel/engines/padlock.dll
Extracting  stunnel/engines/pkcs11.dll

sflock -e /tmp/ '8091063456b4c1ef1771b88f39bd3fda4b25ccb4bbeafa538edcf4b18ed0f9e5'
application/x-7z-compressed
{"relaname": null, "extrpath": [], "password": null, "children": [], "size": 1180641, "preview": false, "filepath": "8091063456b4c1ef1771b88f39bd3fda4b25ccb4bbeafa538edcf4b18ed0f9e5", "package": "7z", "selected": true, "filename": "8091063456b4c1ef1771b88f39bd3fda4b25ccb4bbeafa538edcf4b18ed0f9e5", "platform": null, "duplicate": false, "finger": {"mime_human": "7z compressed", "magic": "7-

sflock_identify thinks that a PE file is Javascript if it has the a "<script" tag embedded

This occurs with recent Danabot malware samples (e.g. https://www.virustotal.com/gui/file/21917cd411471ca69b1c99e31021daca3abb99230847fabd831efa9f262c9d91)

>>> import sflock
>>> sflock.__version__
'0.3.22'
>>> from sflock.ident import identify as sflock_identify
>>> from sflock.abstracts import File as SflockFile
>>> f = SflockFile.from_path(b'/tmp/9a9c29b7c86f85ba69867fe9eca96032')
>>> f.contents[:16]
b'MZP\x00\x02\x00\x00\x00\x04\x00\x0f\x00\xff\xff\x00\x00'
>>> sflock_identify(f)
'js'

Password protected .7z files

I know there's an issue with password protected 7zip files and ZipJail. Is there some workaround for this?

Autofill the target platform

Fill out the target platform, i.e., windows / linux / darwin / android. This information may be passed along to Cuckoo.

Support TNEF

Sflock does not recognize CDFV2 files

Package is not the appropriate (should be doc) and it is not selected by default.

cuckoo bug

sample:
https://www.virustotal.com/gui/file/d8814847471234f525e090e2959f6aed7b8583941176a41580b2ecdff9a9c24a/details

Dependency restriction for python-magic and olefile

Currently sflock requires olefile==0.43 and python-magic==0.4.12. These were the latest versions at the time of commit, but are now outdated.

The requirement can be changed to olefile>=0.43 and python-magic>=0.4.12, <0.5 without issue. This would allow oletools to be included at a later version alongside sflock.

Active Mime files for office are not detected as such

These files by the APT32 threat actor:

-rw-r--r--@ 1 fernando  staff    78K Jun 22 10:38 1210384a9d0ca2e089efab14f2e9f6d55a3824031c1e589b96f854fb96411288
-rw-r--r--@ 1 fernando  staff   112K Jun 22 10:38 1eca9dfd04fd5272a656d6e6d41c9ccc21a2700a979addf612a4de3b071253f5
-rw-r--r--@ 1 fernando  staff    78K Jun 22 10:46 1fc1bc4d004ab51398070d8e3025fecf8878229cda8befdbc9a2faf592b8d876
-rw-r--r--@ 1 fernando  staff   223K Jun 22 10:46 209c52bc39e8fa3df3d4d12a4d1913f3751582b34898adf966dd227cd5a0c99a
-rw-r--r--@ 1 fernando  staff   166K Jun 22 10:46 453168b12bdc881bd6763fbc456620fd42efe6a718c6aecb2fa4982a44207999
-rw-r--r--@ 1 fernando  staff   157K Jun 22 10:38 703af242be581aa4c4c73b08ae57caf7c5d90f09f0991a963e07d02fb4209f75
-rw-r--r--@ 1 fernando  staff   641K Jun 22 10:39 84d9af7b24ce85c3e5d97236c8562fcd45d34d99b07412fbdaca697c5961723e
-rw-r--r--@ 1 fernando  staff   234K Jun 22 10:39 8c355092c7aaadb11748fd87ce528d3cdb483104e979d9b560af840eb8089f94
-rw-r--r--@ 1 fernando  staff   208K Jun 22 10:37 c161134bf3330c82eb0278fe54b2975c26301bdfdc4fc35d5344f9becf5574c7
-rw-r--r--@ 1 fernando  staff   640K Jun 22 10:38 d0a725ee4602cd90493103648e6ec453b7987a016c19cff5c79cc42f4510e92f
-rw-r--r--@ 1 fernando  staff    79K Jun 22 10:45 fadb91606e09b86c39aad99b452525217563594dc9c610120860a

Are active mime files crafted to be opened in MS word, but sflock does not detect them as such. They even have VBA macros embedded.

https://github.com/phishme/python-amime

Decrypt password-encrypted Office 2010+ documents

For more information, see also https://github.com/herumi/msoffice spender-sandbox/cuckoo-modified#313.

Support .hta files

Some (or all) .hta files return the zip package. They should return the ie package instead.

Sample HTA (petya):

MD5 0487382a4daf8eb9660f1c67e30f8b25
SHA1 736752744122a0b5ee4b95ddad634dd225dc0f73
SHA256 ee29b9c01318a1e23836b949942db14d4811246fdae2f41df9f0dcd922c63bc6

Unpack Microsoft cab file

I tried adding support for unpacking Microsoft cab files using 7z and Ace as templates. I'm using cabextract as the external binary. I'm getting the following error:

"Detected potential out-of-path arbitrary overwrite!"

Not sure if this is zip jail related or not. I can't reproduce that error running cabextract from the command line.

Here's the code:

import os
import subprocess
import tempfile

from sflock.abstracts import Unpacker

class CabFile(Unpacker):
    name = "cabfile"
    exe = "/usr/bin/cabextract"
    exts = ".cab"
    magic = "Microsoft Cabinet archive"

    def unpack(self, password=None, duplicates=None):
        dirpath = tempfile.mkdtemp()

        if self.f.filepath:
            filepath = os.path.abspath(self.f.filepath)
            temporary = False
        else:
            filepath = self.f.temp_path(".cab")
            temporary = True

        try:
            subprocess.check_call([
                self.zipjail, filepath, dirpath,
                self.exe, "-d {}".format(dirpath), filepath,
            ], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
        except subprocess.CalledProcessError as e:
            self.f.mode = "failed"
            self.f.error = e

        if temporary:
            os.unlink(filepath)

        return self.process_directory(dirpath, duplicates)

Encrypt zip files feature request

give 7z precedence over zipfile

By doing so we get the extra benefits that may be found in 7z as compared to zipfile (e.g., performance) while also adding deflate64 support (see also #10).

upack 7zip,rar error

pip install sflock in centos7,upack 7zip and rar file,the error message:
Traceback (most recent call last):
f = unpack(sys.argv[1])
File "/root/sflock/lib/python2.7/site-packages/sflock/main.py", line 49, in unpack
f.children = plugin.unpack(password, duplicates)
File "/root/sflock/lib/python2.7/site-packages/sflock/unpack/rar.py", line 33, in unpack
filepath, dirpath,
File "/usr/lib64/python2.7/subprocess.py", line 568, in check_output
process = Popen(stdout=PIPE, *popenargs, **kwargs)
File "/usr/lib64/python2.7/subprocess.py", line 711, in init
errread, errwrite)
File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
OSError: [Errno 13] Permission denied

after chmod +x sflock/lib/python2.7/site-packages/sflock/data/zipjail.elf it will be ok。

Force a package type i.e. rar

I have a sample that returns as a PNG file using magic. However, it is in fact a rar file. I can unpack it with the rar command or with 7zip manually.

What would be the best way to pass the package type, rar in this case, to the unpacker.

I'm importing unpack from sflock currently.

Add .rar support

latest zipjail doesn't work on ubuntu 20.04

github actions isn't available yet for ubuntu 20.04, idk about travis, but i have it installed and tried sflock and zipjail fails for everything that were working(test files)

error msg if that helps

Blocked system call occurred during sandboxing!
ip=0x7fc43fa3d2ab sp=0x7ffc770728a8 abi=0 nr=217 syscall=getdents64
\x1b[1;34mKilling child 7920\x1b[0m

zipjail 0.5.2 problem with 7z

7z itself can extract it, not sure what is wrong here, any idea @jbremer ?
sample 9fa28e86c4754d254e22551ed76f2b6771873b4d64c82dc932902bc87c2398f3

/usr/local/lib/python3.8/dist-packages/sflock/data/zipjail.elf -h
zipjail 0.5.2 - safe unpacking of potentially unsafe archives.
Copyright (C) 2016-2018, Jurriaan Bremer <[email protected]>.
Copyright (C) 2018-2019, Hatching B.V.
Based on Tracy by Merlijn Wajer and Bas Weelinck.
    (https://github.com/MerlijnWajer/tracy)


$ /usr/local/lib/python3.8/dist-packages/sflock/data/zipjail.elf guloader /tmp/tmpadhipzsh -v -- /usr/bin/7z x -mmt=off -o/tmp/tmpadhipzsh guloader
openat(/etc/ld.so.cache)
openat(/lib/x86_64-linux-gnu/libc.so.6)
openat(/usr/bin/7z)
openat(/etc/ld.so.cache)
openat(/lib/x86_64-linux-gnu/libpthread.so.0)
openat(/lib/x86_64-linux-gnu/libdl.so.2)
openat(/usr/lib/x86_64-linux-gnu/libstdc++.so.6)
openat(/lib/x86_64-linux-gnu/libgcc_s.so.1)
openat(/lib/x86_64-linux-gnu/libc.so.6)
openat(/lib/x86_64-linux-gnu/libm.so.6)
openat(/usr/lib/locale/locale-archive)
openat(/usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache)

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
openat(/sys/devices/system/cpu)
p7zip Version 16.02 (locale=es_ES.UTF-8,Utf16=on,HugeFiles=on,64 bits,4 CPUs Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz (306C3),ASM,AES-NI)

openat(/usr/lib/p7zip/7z.so)
openat(/usr/lib/p7zip/Codecs)
openat(/usr/lib/p7zip/Codecs/Rar.so)
openat(/usr/lib/p7zip/Formats)
openat(/usr/lib/p7zip/Formats)
Scanning the drive for archives:
  0M Scanopenat(.)
1 file, 28860 bytes (29 KiB)
openat(.)
openat(.)

Extracting archive: guloader
openat(/home/X)
openat(guloader)
--
Path = guloader
Type = 7z
Physical Size = 28860
Headers Size = 154
Method = LZMA2:17 BCJ
Solid = -
Blocks = 1

mkdir(/tmp/tmpadhipzsh)
mkdir(/tmp/tmpadhipzsh)
  0%clone(0x3d0f00, ...)
Killing child 36940
```

```
7z l guloader

7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=es_ES.UTF-8,Utf16=on,HugeFiles=on,64 bits,4 CPUs Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz (306C3),ASM,AES-NI)

Scanning the drive for archives:
1 file, 28860 bytes (29 KiB)

Listing archive: guloader

--
Path = guloader
Type = 7z
Physical Size = 28860
Headers Size = 154
Method = LZMA2:17 BCJ
Solid = -
Blocks = 1

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2020-05-26 01:33:36 ....A       122880        28706  China Post 1292883.exe
------------------- ----- ------------ ------------  ------------------------
2020-05-26 01:33:36             122880        28706  1 files
```

Correctly identify certain .zip files as .docx etc

zipjail.elf not working on AWS

so that it would be noted down the zipjail.elf has different behaviour on AWS rather than on XEN at current outcome. Basically it crashes unpacking rar-s with error :

Blocked system call occurred during sandboxing!
ip=0xffffffffff600409 sp=0x7ffee3c1bbd8 abi=0 nr=201 syscall=time
Killing child 16983

rar versioning does not make any difference ... maybe kernel ?

hatching / sflock Goto Github PK

sflock's People

Contributors

Stargazers

Watchers

Forkers

sflock's Issues

Recommend Projects

Recommend Topics

Recommend Org