Code Monkey home page Code Monkey logo

roamer's Introduction

RoAMer

This repository is the home of RoAMer, the "Robust Automatic Malware Unpacker". RoAMer is a generic malware unpacker based on dynamic analysis.

It is a generic unpacker based on dynamic analysis. The paper has been presented on MALWARE 2019. Citation: Thorsten Jenke, Daniel Plohmann, and Elmar Padilla, "RoAMer: The Robust Automated Malware Unpacker," 2019 14th International Conference on Malicious and Unwanted Software (MALWARE), Nantucket, MA, USA, 2019, pp. 67-74.

This is still a very early version.

Setup

Requirements and Pre-Setup

To run RoAMer you need to have a VirtualBox or KVM environment with a Windows Virtual Machine (VM) to unpack the malware on. So far, we have only tested RoAMer on Windows7 64Bit.

Virtual machine for upacking process

  • Hardened VM: example howto
    • At least Firewall and Windows Defender need to be deactivated in the VM
    • Virtual Network, where the host system is able to communicate with the VM
  • Python (virtual) environment (<= v3.7) that satisfies:

Host System

  • Python3 (virtual) environment (>= v3.6)

Compile on Windows VM

  • clone the git repository
  • execute compile.bat in Windows CMD, this will compile the python scripts into Windows executable files
  • the executables can be found in unpacker/dist, pewhitelister/dist and receiver/dist

Deployment

  • run PeHeaderWhitelister.exe C:\ in Windows CMD in the VM and copy the resulting pe_header_whitelist.json file of this script to the current VM's users home directory (C:\Users\%username%\)
  • Copy the file unpacker/dist/main.exe from the VM to the host system into $Repository/roamer/bin
  • Ensure that the receiver main.exe is not stored at C:\Users\%username%\main.exe (this path is already reserved for the unpacker main.exe)
  • start receiver main.exe in the VM within a command line terminal (cmd.exe) as an administrator
  • move desktop the symbols so that the upper left corner of your desktop is free
  • create a shortcut to notepad as the first icon directly below the free space (right click -> New -> Shortcut: C:\Windows\notepad.exe)
  • open notepad with the new shortcut and move the notepad windows over the notepad shortcut icon, then close notepad
  • create a snapshot of the VM in VirtualBox and name it e.g. init

Screenshot how the VM should look like at the end: Screenshot of vm

Configuration

  • check the host's config.py and set the SNAPSHOT_NAME (e.g. init) and VM_NAME (e.g. win7box)
  • Set host_ip, host_port, guest_ip, guest_port to the correct values of your setup

Check configuration

Make sure that the following files are existent in your setup:

VM: C:\Users\%username%\pe_header_whitelist.json

Host: ../RoAMer/roamer/bin/main.exe

How To Use

  • Adjust config.py parameters as needed. The default configuration was the most successful as determined by the Thesis' evaluation.
  • Just start /run.py <path_to_sample> and RoAMer will then do its magic in the VM and respond with the identified dumps.
  • The script will start the VM with the snapshot you've specified in config.py

FAQ / Known issues

Why do I have to move the desktop icons away then place a notepad window there?

The unpacker is performing mouse moving and click actions, that malware is less likely to detect that it is going to be unpacked. The actions are in the left upper screen area, moving icons away prevents from starting several useless programs.

Whitelister script runs into "InvalidArgument" exception

This may occur when some broken exe files are located in you AppData or Windows Update directory, remove these corrupt files, then whitelister will run through.

The Unpacker cannot connect back to the host: "WinError 10060 - A connection attempt failed because the party did not properly respond..."

In case you encounter a situation where the host is able to start up the virtual machine, transfer files and start the analysis, but no data is transferred back to the host, please ensure that your host does not have firewall settings that block incoming connections (like UFW on Ubuntu or similar).

Compatibility issues on Windows 10

RoAMer was originally developed to unpack on Windows 7 64Bit systems, we monitored some issues on Windows 10 that might or might not be critical for execution. We're currently working on resolving these issues.

Why is the malware not getting unpacked.

RoAMer is doing a really bad job at conveying information on why it was not able to unpack a given sample. We can offer, that you send us the sample and we check, why RoAMer was not able to unpack the malware. Maybe this leads us to the discovery of some bugs.

Mentions

RoAMer was mentioned in TA505's Box of Chocolate - On Hidden Gems packed with the TA505 Packer by Thomas Barabosch.

Sources

roamer's People

Contributors

blattm avatar danielplohmann avatar elanfer avatar larsborn avatar urmelausdemeis avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

roamer's Issues

main.exe crashes after writing files with "No module named win32con"

This might just be a case of the docs needing clarification.

How to reproduce

  • I installed python 3.8 on windows 7 x64 Ultimate
  • Installed dependencies via pip install pyinstaller pywin32
  • Applied workaround mentioned in issue #5
  • Ran main.exe according to README (including the shortcut magic)
  • Ran main.py from host

Error output

VM:

2020-08-14 10:23:38,741 waiting...
2020-08-14 10:25:29,444 received needed files
2020-08-14 10:25:29,897 received everything
2020-08-14 10:25:29,913 writing files...
Traceback (most recent call last):
  File "unpacker\main.py", line 5, in <module>
  File "C:\Users\User\AppData\Roaming\Python\Python38\site-packages\PyInstaller\
loader\pyimod03_importers.py", line 623, in exec_module
  File "unpacker\Orchestrator.py", line 8, in <module>
  File "C:\Users\User\AppData\Roaming\Python\Python38\site-packages\PyInstaller\
loader\pyimod03_importers.py", line 623, in exec_module
  File "unpacker\Unpacker.py", line 6, in <module>
  File "C:\Users\User\AppData\Roaming\Python\Python38\site-packages\PyInstaller\
loader\pyimod03_importers.py", line 623, in exec_module
  File "unpacker\dumping\dumper.py", line 5, in <module>
  File "C:\Users\User\AppData\Roaming\Python\Python38\site-packages\PyInstaller\
loader\pyimod03_importers.py", line 623, in exec_module
  File "unpacker\dumping\dump_task.py", line 1, in <module>
  File "C:\Users\User\AppData\Roaming\Python\Python38\site-packages\PyInstaller\
loader\pyimod03_importers.py", line 623, in exec_module
  File "unpacker\winwrapper\utilities.py", line 8, in <module>
ModuleNotFoundError: No module named 'win32con'
[2916] Failed to execute script main
Traceback (most recent call last):
  File "main.py", line 82, in <module>
  File "main.py", line 78, in main
  File "main.py", line 73, in run
  File "WindowsManipulator.py", line 39, in grant_debug_privilege
TypeError: not all arguments converted during string formatting
[2896] Failed to execute script main

Host:

2020-08-14 10:50:04,221 Unpacking [path redacted]
2020-08-14 10:50:04,255 preparing VM for RoAMer
2020-08-14 10:50:10,344 VM Win7Ultimate should now be running on snapshot RoAMer
('[ip redacted]', 10000)
2020-08-14 10:50:10,344 Connecting to roamer-receiver...
2020-08-14 10:50:10,350 Sending file list...
2020-08-14 10:50:10,350 waiting for needed files...
b'both_EM0_sleep.dll,EP_EM0_sleep.dll,main.exe,TP_EM0_sleep.dll'
2020-08-14 10:50:10,356 start sending
2020-08-14 10:50:10,792 sending of unpacker completed.
2020-08-14 10:50:10,792 waiting for connection

New Console Parameter --output

Add an additional output parameter, that will produce the dump output into a user-provided location instead of into the same directory as the processed sample file, i.e.

python3 run.py /some/sample --output /some_other_folder

h/t @tbarabosch

Win10/Win11 support

Evaluate the current status of compatibility with Win10/Win11.
In case of incompatibility, estimate the effort required for porting to these Windows versions or simply make it happen. :)

h/t @tbarabosch

PeHeaderWhitelister: Crashes on "could not read ..."

When executing PeHeaderWhitelister.exe C:\ I got the following error:

could not read  C:\Users\User\AppData\Local\Microsoft\WindowsApps\GameBarElevatedFT_Alias.exe ... continuing
Traceback (most recent call last):
  File "whitelister\PeHeaderWhitelister.py", line 91, in <module>
  File "whitelister\PeHeaderWhitelister.py", line 80, in generate_pe_header_whitelist
  File "whitelister\PeHeaderWhitelister.py", line 68, in add_entry
  File "whitelister\PeHeaderWhitelister.py", line 45, in normalize_pe_header
  File "c:\program files\python38\lib\re.py", line 248, in finditer
    return _compile(pattern, flags).finditer(string)
TypeError: expected string or bytes-like object
[2420] Failed to execute script PeHeaderWhitelister

It's unclear why the file cannot be read (I am executing PeHeaderWhitelister as admin) but since the message says could not read [...] ... continuing I would expect the tool to continue. However it crashes immediately after.

I also deleted that file and tried again, in which case the same message came with a different file, followed by the same traceback and crash.

I'm running this on the Windows 10 x64 evaluation VM provided by microsoft. Python is version 3.8.

YARA Scanning of output

Add YARA scanning capability to RoAMer.
This could work two ways:

  • Scan all output files produced by RoAMer
    • Show information about all hits on the console
    • Provide a detections txt file along the output folder
  • Use YARA rules to select memory sections for dumping, regardless of their other filter results (e.g. if executable flag is missing etc)

h/t @tbarabosch

[solved via workaround] pywintypes38.dll can't be found (python3.8 on windows 7 ultimate)

Problem

When running compile.bat, I get the following warning:

4437 WARNING: lib not found: pywintypes38.dll dependency of C:\program files\python38\lib\site-packages\win32\win32evtlog.pyd
4453 WARNING: lib not found: pywintypes38.dll dependency of C:\program files\python38\lib\site-packages\win32\win32api.pyd
4468 WARNING: lib not found: pywintypes38.dll dependency of C:\program files\python38\lib\site-packages\win32\win32process.pyd

The compilation completes, but when running main.exe it errors with this message:

[backtrace omitted since can't copy-paste from my vm]
ImportError: DLL load failed while importing win32process: The specified module could not be found.
[544] Failed to execute script main

Solution/Workaround

I found the missing pywintypes38.dll in C:\progam files\python38\lib\site-packages\pywin32_system32\pywintypes38.dll and copied it to C:\windows\system32, then I ran compile.bat again to be on the safe side and it worked. Worked, meaning no warning when running compile and executing the main.exe does not result in a crash.

How to reproduce

I'm running windows 7 x64 Ultimate in a libvirt/qemu VM, installed latest python 3.8 (for all users, i. e. in C:\program files\...) and ran this command as admin to fulfil the dependencies:

pip install pyinstaller pywin32

Not sure if it's a bug in pip, the pywin32 package or a combination of these with win7 ultimate. Mentioning this workaround in the README would be sufficient IMO as I'm not sure this can be fixed from within this project.

(Note, this also happened on my win10 eval vm, but I did not report it then, because I didn't take notes when fixing it.)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.