hatching / sflock Goto Github PK
View Code? Open in Web Editor NEWSample staging & detonation utility to be used in combination with Cuckoo Sandbox.
Sample staging & detonation utility to be used in combination with Cuckoo Sandbox.
The compiled zipjail binary on the master
branch is outdated (version 0.5
vs 0.5.2
from the tracy project master)
As a result, its syscall whitelist is missing getdents64
and getpid
:
sflock master:
.rodata:000000000049B914 aGettimeofday db 'gettimeofday',0 ; DATA XREF: .data:00000000006C81C0↓o
.rodata:000000000049B921 aStat64 db 'stat64',0 ; DATA XREF: .data:00000000006C81C8↓o
.rodata:000000000049B928 aSysDevicesSyst_1 db '/sys/devices/system/cpu',0
compiled tracy master:
.rodata:000000000049F0B4 aGettimeofday db 'gettimeofday',0 ; DATA XREF: .data:00000000004CE1C0↓o
.rodata:000000000049F0C1 aStat64 db 'stat64',0 ; DATA XREF: .data:00000000004CE1C8↓o
.rodata:000000000049F0C8 aGetdents64 db 'getdents64',0 ; DATA XREF: .data:00000000004CE1D0↓o
.rodata:000000000049F0D3 aGetpid_0 db 'getpid',0 ; DATA XREF: .data:00000000004CE1D8↓o
.rodata:000000000049F0DA aSysDevicesSyst_1 db '/sys/devices/system/cpu',0
This will cause some archives to be incorrectly dropped: (unpacking an iso archive using 7zip)
ERROR:root:Blocked system call occurred during sandboxing!
ip=0x7fd816c6d07b sp=0x7fffbe193978 abi=0 nr=217 syscall=getdents64
Killing child 7064
I noticed some large ELF binaries having a package set as 'js' instead of 'generic' due to strings found within & that the ident.py identifiers check by looking across the entire file contents. As one mitigation method should the buffer/contents being checked be limited to 0x1000 to align with the check at the top of the identifier function (if not f.stream.read(0x1000):)?
Was the purpose of overwriting the package predominantly for archives? Is it worth placing a conditional within main.unpack() along the line of-
if f.type == "container": # or != "file" ?
ident(f)
Is there a reason you're not using the gzip module to handle pure gzip file? You do that for .tar.gz, so it is confusing.
And if not, do you mind if I implement a pure python module to handle simple gzip files without requiring 7zip?
Properly handle zip bombs as well as archives with more than X selected files in it.
No need for Cuckoo to freeze on such inputs.
This assertion throws an error when running the unit tests: https://github.com/hatching/sflock/blob/master/tests/test_main.py#L30
`> assert count == len(supported())
E AssertionError: assert 14 == 15
E + where 15 = len([b'.msg', b'.7z', b'.iso', b'.gzip', b'.lzh', b'.lha', ...])
E + where [b'.msg', b'.7z', b'.iso', b'.gzip', b'.lzh', b'.lha', ...] = supported()
tests/test_main.py:30: AssertionError`
The full string of supported()
is: [b'.msg', b'.7z', b'.iso', b'.gzip', b'.lzh', b'.lha', b'.daa', b'.eml', b'.bup', b'.zip', b'.mso', b'.tar', b'.tar.gz', b'.tar.bz2', b'.pdf']
Might affect other unpackers as well, though.
data/zipjail.elf 8091063456b4c1ef1771b88f39bd3fda4b25ccb4bbeafa538edcf4b18ed0f9e5 /tmp/ -- /usr/bin/7z x -mmt=off -o/tmp/ 8091063456b4c1ef1771b88f39bd3fda4b25ccb4bbeafa538edcf4b18ed0f9e5
7-Zip [64] 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,8 CPUs)
Processing archive: 8091063456b4c1ef1771b88f39bd3fda4b25ccb4bbeafa538edcf4b18ed0f9e5
Detected potential out-of-path arbitrary overwrite!
filepath=/tmp/stunnel dirpath=/tmp/
Killing child 7176
cabextract is already the newest version (1.6-1).
rar is already the newest version (2:5.3.b2-1).
unace-nonfree is already the newest version (2.5-8).
p7zip-full is already the newest version (9.20.1~dfsg.1-4.2ubuntu0.1).
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
SFlock (0.3.10) - Sample staging and detonation utility
INSTALLED: 0.3.10
from sflock import unpack
q = unpack('8091063456b4c1ef1771b88f39bd3fda4b25ccb4bbeafa538edcf4b18ed0f9e5')
q.children
[]
>>> q.mime
'application/x-7z-compressed'
7z x -mmt=off -o/tmp/ 8091063456b4c1ef1771b88f39bd3fda4b25ccb4bbeafa538edcf4b18ed0f9e5
7-Zip [64] 9.20 Copyright (c) 1999-2010 Igor Pavlov 2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,8 CPUs)
Processing archive: 8091063456b4c1ef1771b88f39bd3fda4b25ccb4bbeafa538edcf4b18ed0f9e5
Extracting stunnel
Extracting stunnel/bin
Extracting stunnel/config
Extracting stunnel/engines
Extracting stunnel/bin/Microsoft.VC90.CRT.Manifest
Extracting stunnel/config/ca-certs.pem
Extracting stunnel/config/openssl.cnf
Extracting stunnel/config/stunnel.conf
Extracting stunnel/bin/libeay32.dll
Extracting stunnel/bin/msvcr90.dll
Extracting stunnel/bin/openssl.exe
Extracting stunnel/bin/ssleay32.dll
Extracting stunnel/bin/stunnel.exe
Extracting stunnel/bin/tstunnel.exe
Extracting stunnel/engines/capi.dll
Extracting stunnel/engines/padlock.dll
Extracting stunnel/engines/pkcs11.dll
sflock -e /tmp/ '8091063456b4c1ef1771b88f39bd3fda4b25ccb4bbeafa538edcf4b18ed0f9e5'
application/x-7z-compressed
{"relaname": null, "extrpath": [], "password": null, "children": [], "size": 1180641, "preview": false, "filepath": "8091063456b4c1ef1771b88f39bd3fda4b25ccb4bbeafa538edcf4b18ed0f9e5", "package": "7z", "selected": true, "filename": "8091063456b4c1ef1771b88f39bd3fda4b25ccb4bbeafa538edcf4b18ed0f9e5", "platform": null, "duplicate": false, "finger": {"mime_human": "7z compressed", "magic": "7-
This occurs with recent Danabot malware samples (e.g. https://www.virustotal.com/gui/file/21917cd411471ca69b1c99e31021daca3abb99230847fabd831efa9f262c9d91)
>>> import sflock
>>> sflock.__version__
'0.3.22'
>>> from sflock.ident import identify as sflock_identify
>>> from sflock.abstracts import File as SflockFile
>>> f = SflockFile.from_path(b'/tmp/9a9c29b7c86f85ba69867fe9eca96032')
>>> f.contents[:16]
b'MZP\x00\x02\x00\x00\x00\x04\x00\x0f\x00\xff\xff\x00\x00'
>>> sflock_identify(f)
'js'
I know there's an issue with password protected 7zip files and ZipJail. Is there some workaround for this?
Fill out the target platform, i.e., windows
/ linux
/ darwin
/ android
. This information may be passed along to Cuckoo.
See also https://sourceforge.net/p/ytnef/tnef-samples/1/, https://github.com/Yeraze/ytnef, and most importantly https://github.com/koodaamo/tnefparse.
Currently sflock requires olefile==0.43
and python-magic==0.4.12
. These were the latest versions at the time of commit, but are now outdated.
The requirement can be changed to olefile>=0.43
and python-magic>=0.4.12, <0.5
without issue. This would allow oletools
to be included at a later version alongside sflock.
These files by the APT32 threat actor:
-rw-r--r--@ 1 fernando staff 78K Jun 22 10:38 1210384a9d0ca2e089efab14f2e9f6d55a3824031c1e589b96f854fb96411288
-rw-r--r--@ 1 fernando staff 112K Jun 22 10:38 1eca9dfd04fd5272a656d6e6d41c9ccc21a2700a979addf612a4de3b071253f5
-rw-r--r--@ 1 fernando staff 78K Jun 22 10:46 1fc1bc4d004ab51398070d8e3025fecf8878229cda8befdbc9a2faf592b8d876
-rw-r--r--@ 1 fernando staff 223K Jun 22 10:46 209c52bc39e8fa3df3d4d12a4d1913f3751582b34898adf966dd227cd5a0c99a
-rw-r--r--@ 1 fernando staff 166K Jun 22 10:46 453168b12bdc881bd6763fbc456620fd42efe6a718c6aecb2fa4982a44207999
-rw-r--r--@ 1 fernando staff 157K Jun 22 10:38 703af242be581aa4c4c73b08ae57caf7c5d90f09f0991a963e07d02fb4209f75
-rw-r--r--@ 1 fernando staff 641K Jun 22 10:39 84d9af7b24ce85c3e5d97236c8562fcd45d34d99b07412fbdaca697c5961723e
-rw-r--r--@ 1 fernando staff 234K Jun 22 10:39 8c355092c7aaadb11748fd87ce528d3cdb483104e979d9b560af840eb8089f94
-rw-r--r--@ 1 fernando staff 208K Jun 22 10:37 c161134bf3330c82eb0278fe54b2975c26301bdfdc4fc35d5344f9becf5574c7
-rw-r--r--@ 1 fernando staff 640K Jun 22 10:38 d0a725ee4602cd90493103648e6ec453b7987a016c19cff5c79cc42f4510e92f
-rw-r--r--@ 1 fernando staff 79K Jun 22 10:45 fadb91606e09b86c39aad99b452525217563594dc9c610120860a
Are active mime files crafted to be opened in MS word, but sflock does not detect them as such. They even have VBA macros embedded.
For more information, see also https://github.com/herumi/msoffice spender-sandbox/cuckoo-modified#313.
Some (or all) .hta files return the zip
package. They should return the ie
package instead.
Sample HTA (petya):
MD5 0487382a4daf8eb9660f1c67e30f8b25
SHA1 736752744122a0b5ee4b95ddad634dd225dc0f73
SHA256 ee29b9c01318a1e23836b949942db14d4811246fdae2f41df9f0dcd922c63bc6
I tried adding support for unpacking Microsoft cab files using 7z and Ace as templates. I'm using cabextract as the external binary. I'm getting the following error:
"Detected potential out-of-path arbitrary overwrite!"
Not sure if this is zip jail related or not. I can't reproduce that error running cabextract from the command line.
Here's the code:
import os
import subprocess
import tempfile
from sflock.abstracts import Unpacker
class CabFile(Unpacker):
name = "cabfile"
exe = "/usr/bin/cabextract"
exts = ".cab"
magic = "Microsoft Cabinet archive"
def unpack(self, password=None, duplicates=None):
dirpath = tempfile.mkdtemp()
if self.f.filepath:
filepath = os.path.abspath(self.f.filepath)
temporary = False
else:
filepath = self.f.temp_path(".cab")
temporary = True
try:
subprocess.check_call([
self.zipjail, filepath, dirpath,
self.exe, "-d {}".format(dirpath), filepath,
], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
except subprocess.CalledProcessError as e:
self.f.mode = "failed"
self.f.error = e
if temporary:
os.unlink(filepath)
return self.process_directory(dirpath, duplicates)
By doing so we get the extra benefits that may be found in 7z as compared to zipfile (e.g., performance) while also adding deflate64 support (see also #10).
pip install sflock in centos7,upack 7zip and rar file,the error message:
Traceback (most recent call last):
f = unpack(sys.argv[1])
File "/root/sflock/lib/python2.7/site-packages/sflock/main.py", line 49, in unpack
f.children = plugin.unpack(password, duplicates)
File "/root/sflock/lib/python2.7/site-packages/sflock/unpack/rar.py", line 33, in unpack
filepath, dirpath,
File "/usr/lib64/python2.7/subprocess.py", line 568, in check_output
process = Popen(stdout=PIPE, *popenargs, **kwargs)
File "/usr/lib64/python2.7/subprocess.py", line 711, in init
errread, errwrite)
File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
OSError: [Errno 13] Permission denied
after chmod +x sflock/lib/python2.7/site-packages/sflock/data/zipjail.elf it will be ok。
I have a sample that returns as a PNG file using magic. However, it is in fact a rar file. I can unpack it with the rar command or with 7zip manually.
What would be the best way to pass the package type, rar in this case, to the unpacker.
I'm importing unpack from sflock currently.
github actions isn't available yet for ubuntu 20.04, idk about travis, but i have it installed and tried sflock and zipjail fails for everything that were working(test files)
error msg if that helps
Blocked system call occurred during sandboxing!
ip=0x7fc43fa3d2ab sp=0x7ffc770728a8 abi=0 nr=217 syscall=getdents64
\x1b[1;34mKilling child 7920\x1b[0m
7z itself can extract it, not sure what is wrong here, any idea @jbremer ?
sample 9fa28e86c4754d254e22551ed76f2b6771873b4d64c82dc932902bc87c2398f3
/usr/local/lib/python3.8/dist-packages/sflock/data/zipjail.elf -h
zipjail 0.5.2 - safe unpacking of potentially unsafe archives.
Copyright (C) 2016-2018, Jurriaan Bremer <[email protected]>.
Copyright (C) 2018-2019, Hatching B.V.
Based on Tracy by Merlijn Wajer and Bas Weelinck.
(https://github.com/MerlijnWajer/tracy)
$ /usr/local/lib/python3.8/dist-packages/sflock/data/zipjail.elf guloader /tmp/tmpadhipzsh -v -- /usr/bin/7z x -mmt=off -o/tmp/tmpadhipzsh guloader
openat(/etc/ld.so.cache)
openat(/lib/x86_64-linux-gnu/libc.so.6)
openat(/usr/bin/7z)
openat(/etc/ld.so.cache)
openat(/lib/x86_64-linux-gnu/libpthread.so.0)
openat(/lib/x86_64-linux-gnu/libdl.so.2)
openat(/usr/lib/x86_64-linux-gnu/libstdc++.so.6)
openat(/lib/x86_64-linux-gnu/libgcc_s.so.1)
openat(/lib/x86_64-linux-gnu/libc.so.6)
openat(/lib/x86_64-linux-gnu/libm.so.6)
openat(/usr/lib/locale/locale-archive)
openat(/usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache)
7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
openat(/sys/devices/system/cpu)
p7zip Version 16.02 (locale=es_ES.UTF-8,Utf16=on,HugeFiles=on,64 bits,4 CPUs Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz (306C3),ASM,AES-NI)
openat(/usr/lib/p7zip/7z.so)
openat(/usr/lib/p7zip/Codecs)
openat(/usr/lib/p7zip/Codecs/Rar.so)
openat(/usr/lib/p7zip/Formats)
openat(/usr/lib/p7zip/Formats)
Scanning the drive for archives:
0M Scanopenat(.)
1 file, 28860 bytes (29 KiB)
openat(.)
openat(.)
Extracting archive: guloader
openat(/home/X)
openat(guloader)
--
Path = guloader
Type = 7z
Physical Size = 28860
Headers Size = 154
Method = LZMA2:17 BCJ
Solid = -
Blocks = 1
mkdir(/tmp/tmpadhipzsh)
mkdir(/tmp/tmpadhipzsh)
0%clone(0x3d0f00, ...)
Killing child 36940
```
```
7z l guloader
7-Zip [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=es_ES.UTF-8,Utf16=on,HugeFiles=on,64 bits,4 CPUs Intel(R) Xeon(R) CPU E3-1225 v3 @ 3.20GHz (306C3),ASM,AES-NI)
Scanning the drive for archives:
1 file, 28860 bytes (29 KiB)
Listing archive: guloader
--
Path = guloader
Type = 7z
Physical Size = 28860
Headers Size = 154
Method = LZMA2:17 BCJ
Solid = -
Blocks = 1
Date Time Attr Size Compressed Name
------------------- ----- ------------ ------------ ------------------------
2020-05-26 01:33:36 ....A 122880 28706 China Post 1292883.exe
------------------- ----- ------------ ------------ ------------------------
2020-05-26 01:33:36 122880 28706 1 files
```
so that it would be noted down the zipjail.elf has different behaviour on AWS rather than on XEN at current outcome. Basically it crashes unpacking rar-s with error :
Blocked system call occurred during sandboxing!
ip=0xffffffffff600409 sp=0x7ffee3c1bbd8 abi=0 nr=201 syscall=time
Killing child 16983
rar versioning does not make any difference ... maybe kernel ?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.