miwong / intellidroid Goto Github PK

A targeted input generator for Android that improves the effectiveness of dynamic malware analysis.

License: Other

Shell 6.08% Java 81.97% Python 11.95%

static-analysis aosp dynamic-analysis constraints android

intellidroid's Introduction

IntelliDroid

IntelliDroid is an analysis tool for Android applications that extracts call paths leading to specific behavior and executes these paths precisely during run time. When given a set of targeted behaviors, the static analysis component traverses the application's call graph to find paths to these behaviors. It also extracts path constraints, which are used to determine the input values that can trigger these paths. The dynamic component takes the extracted paths/constraints and injects the input values into the Android device, triggering the targeted behaviors.

For further details, please see our paper and slides (NDSS 2016).

Framework Analysis

The 'FrameworkAnalysis' performs static analysis to generate constraints for the Android framework. We currently provide the output framework constraints so that they can be added to the application constraints produced by the 'AppAnalysis' component.

App Analysis

The 'AppAnalysis' directory holds the code that generate constraints for Android applications.

Directory	Description
preprocess	Scripts to extract and preprocess APK file before passing it to the tool.
src	Source code files.
libs	Dependencies, including the necessary WALA libraries.¹
android	Compiled Android framework files (to be analyzed), from AOSP version 4.4.2_r2.

Building and running

This project uses the Gradle build system. Output files are located in the build/ directory. The gradlew script is a wrapper for machines that do not have Gradle already installed. If your development machine already contains Gradle, you can use your own installation by replacing ./gradlew with gradle in the commands below. A network connection is required when compiling the code for the first time so that Gradle can automatically download dependencies.

The included Android framework files in the android directory were compiled using JDK 1.6 but the apktool used in the preprocessing scripts require JDK 1.7. We recommend using JDK 1.7, as that seems to work best. If necessary, you can replace the files in the android directory to try different versions of AOSP or JDK.

Preprocessing APK files:

The target APK file to be analyzed should first be preprocessed using the scripts in the preprocess folder. The resulting directory (containing the APK file and the extracted resources) can then be passed to the static analysis.

The preprocessing scripts uses Apktool and Dare to extract the APK package. You can use your own extraction tools, but you may have to modify the app analysis code so that IntelliDroid can find the bytecode and manifest files for a given application.

./preprocess/PreprocessAPK.sh <APK file>
./preprocess/PreprocessDataset.sh <directory of APK files>

To build:

./gradlew build

To build and run:

./IntelliDroidAppAnalysis -o <output directory> <preprocessed app directory>

To see other command-line options, run:

./IntelliDroidAppAnalysis --help

The output directory is used to store the app info JSON file and the Z3 constraint files. If not specified, these files will be stored in ./pathOutput. The output files are used by the IntelliDroidDynamicClient tool to identify the call paths and to generate the input data to trigger these paths.

The appInfo.json and constraintX_X.py files produced in the output directory are necessary for the dynamic client, but are not very readable. The -y flag can be used to obtain a more readable (but output-heavy) version of the path/constraint results (printed in stdout).

Dynamic Client

The 'DynamicClient' directory contains the python program that communicates with the Android device and sends the inputs that trigger the desired events.

Setting up the environment

The program expects that an Android device or emulator is connected to the system. This device must be running the custom Android OS containing IntelliDroidService (which interprets the commands this program sends and does the actual event invocation). The DynamicClient program will automatically connect to the device via a socket on port tcp:12348. The adb tool will be used to set up the connection and run certain commands, and it should be reachable from PATH (this can be done by adding the Android SDK tools directory to the PATH variable or by setting up the AOSP build environment prior to running the DynamicClient.

In addition, the DynamicClient uses the Z3 constraint solver via its Python API (z3-py). The instructions to build and install Z3 with Python bindings are available here.

Patching IntelliDroidService

The custom Android OS used by IntelliDroid is provided as a series of diff files that can be applied to the base AOSP source tree. IntelliDroid is currently implemented for Android 4.3 (AOSP branch android-4.3_r1). Please refer to the AOSP documentation for instructions on how to download and build Android.

Once you have downloaded and compiled the correct AOSP version, you can apply the IntelliDroid patches, which are located in the androidPatches directory. The patch.sh and unpatch.sh files are provided to help automate this process. After patching, rebuild AOSP (we've found that you might have to run make twice; otherwise, certain files on the emulator become inconsistent). You may also need to run make update-api along with make. Once built, you can use logcat to verify that the IntelliDroidService class is started as a system service when the device boots.

To apply patches:

cd androidPatches
./patch.sh <path to AOSP directory>

Running

To run:

./IntelliDroidDynamicClient.py

Commands:

HELP (get description of commands)
INSTALL <APK file>
START <directory to app information, generated by IntelliDroidAppAnalysis>
TRIGGER <call path ID to trigger, as specified in appInfo.json>
EXECUTE <command to send to IntelliDroidService>¹
INFO <info requested from IntelliDroidService>¹
CLOSE
KILL

_{¹ These commands are for debugging purposes only. You should only use these if you know the exact parameters that IntelliDroidService expects.}

TaintDroid Integration

Integration with TaintDroid is fairly straightforward. For the 'AppAnalysis' component, use the -t flag to specify a different list of targeted methods (i.e. the provided taintdroidTargets.txt file).

./IntelliDroidAppAnalysis -t taintdroidTargets.txt <preprocessed app directory>

On the dynamic side, follow the instructions here to download and build TaintDroid for Android 4.3. Once you have verified that your build of TaintDroid works, apply the patches in DynamicClient/androidPatches in the same way as you would for unmodified AOSP.

Contact

IntelliDroid was initially developed as a Master's thesis project by Michelle Wong at the University of Toronto, supervised by Dr. David Lie.

For any inquiries, please contact:

Michelle Wong ([email protected])

The following have contributed code to IntelliDroid:

Michelle Wong
Dr. David Lie
Mariana D'Angelo
Peter Sun

License

IntelliDroid is released under the MIT License.

intellidroid's People

Contributors

Stargazers

Watchers

intellidroid's Issues

INSTALL_FAILED_OLDER_SDK

Hi,

I am experimenting with Intellidroid and I have succefully setup the Intellidroid AppAnalysis and Dynamic Client. While running the Dynamic client. I am trying to install the apk file for the analaysis on the device.

I am getting the following error

> INSTALL /home/parul/Documents/IntelliDroid/AppAnalysis/1/1.apk
3542 KB/s (57618777 bytes in 15.884s)
	pkg: /data/local/tmp/1.apk
Failure [INSTALL_FAILED_OLDER_SDK]

Please let me know, If I am wrongly doing it or any other command is required. Also, I wanted to know the correct parameters of commands.

Thanks,
P

Missing edges in callgraph

Hi there,

I am using IntelliDroid to test some very simple toy examples, but seems I cannot get paths for some cases when IPC is involved. I checked the callgraph building, and found out that those key edges are missing. The following is an example:

Intent i = new Intent(ctx, MyService.class);
ctx.startService(i);

class MyService {
  @Override
  protected void onHandleIntent(Intent intent) {
    ...
    smsManager.sendTextMessage(...);
  }
}

The edge "ctx.startService --> MyService.onHandleIntent" is missing.

Also, the invocation "smsManager.sendTextMessage" didn't appear in "CallGraphInfoListener.onInvoke", which makes "sendTextMessage" not identified as a target method.

Seems the problems are all because IntelliDroid adopts the default callgraph building phase, I'm not an expert in WALA, I was wondering is there any way to customize the callgraph building? Or IntelliDroid already did this but I missed that?

Thanks!

Can't get AppAnalysis to work following the readme: FileNotfoundException

Hi, can't get Intellidroid to work following the steps at Readme. Once I am at the folder from the code, I get the following message from the terminal at my OS X device:

MacBook-Pro-de-David-2:AppAnalysis dass$ ./preprocess/PreprocessAPK.sh ./Apps/App.apk 
./preprocess/IntelliDroidPreprocessAPK.sh: line 14: greadlink: command not found
mkdir: ./.dir: File exists
usage: mv [-f | -i | -n] [-v] source target
       mv [-f | -i | -n] [-v] source ... directory
mv: ./.dir and ./.dir are identical
Input file (.//.apk) was not found or was not readable.
Unable to open './/.apk' as zip archive
Not Zip, retrying as DEX
ERROR: unable to open './/.apk': No such file or directory
./dare/retargeted/. : no such file or directory
java.io.FileNotFoundException: ./apk/classes.jar (No such file or directory)
	at java.io.FileOutputStream.open0(Native Method)
	at java.io.FileOutputStream.open(FileOutputStream.java:270)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:101)
	at sun.tools.jar.Main.run(Main.java:195)
	at sun.tools.jar.Main.main(Main.java:1288)
rm: .//dare: No such file or directory

Where App.apk is a compiled apk downloaded from a source which I want to analyse. I tried again converting the apk to zip, but I get the same error.

MacBook-Pro-de-David-2:AppAnalysis dass$ ./preprocess/PreprocessAPK.sh ./Apps/App.zip
./preprocess/IntelliDroidPreprocessAPK.sh: line 14: greadlink: command not found
mkdir: ./.dir: File exists
usage: mv [-f | -i | -n] [-v] source target
       mv [-f | -i | -n] [-v] source ... directory
mv: ./.dir and ./.dir are identical
Input file (.//.apk) was not found or was not readable.
Unable to open './/.apk' as zip archive
Not Zip, retrying as DEX
ERROR: unable to open './/.apk': No such file or directory
./dare/retargeted/. : no such file or directory
java.io.FileNotFoundException: ./apk/classes.jar (No such file or directory)
	at java.io.FileOutputStream.open0(Native Method)
	at java.io.FileOutputStream.open(FileOutputStream.java:270)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
	at java.io.FileOutputStream.<init>(FileOutputStream.java:101)
	at sun.tools.jar.Main.run(Main.java:195)
	at sun.tools.jar.Main.main(Main.java:1288)
rm: .//dare: No such file or directory

Any ideas why is this happening? I checked that the dare directories exist, but don't know why the scripts is unable to read the apk or zip. I tried with different apps to check if there's a problem with that apk, but I get the same error with all of them.

Thank you for your help!

App Analysis Exception

Hi,
Every time I run the command ./IntelliDroidAppAnalysis -o <output directory> <preprocessed app directory> ,exception java.lang.IllegalArgumentExceptionoccurs.For example,after preprocessing the apk file com.github.wakhub.tinyclock_5.apk which is from f-droid.org,I run the command ./IntelliDroidAppAnalysis -o output/ ~/com.github.wakhub.tinyclock_5/,the exception information shows as below:

Exception in thread "main" java.lang.IllegalArgumentException: class <Extension,Ljava/lang/Object> is invalid, unexpected classloader
    at com.ibm.wala.ipa.cha.ClassHierarchy.addClass(ClassHierarchy.java:300)
    at com.ibm.wala.ipa.cha.ClassHierarchy.addAllClasses(ClassHierarchy.java:280)
    at com.ibm.wala.ipa.cha.ClassHierarchy.<init>(ClassHierarchy.java:241)
    at com.ibm.wala.ipa.cha.ClassHierarchy.<init>(ClassHierarchy.java:177)
    at com.ibm.wala.ipa.cha.ClassHierarchy.make(ClassHierarchy.java:1217)
    at com.ibm.wala.ipa.cha.ClassHierarchy.make(ClassHierarchy.java:1197)
    at intellidroid.appanalysis.IntelliDroidAppAnalysis.analyze(IntelliDroidAppAnalysis.java:230)
    at intellidroid.appanalysis.IntelliDroidAppAnalysis.main(IntelliDroidAppAnalysis.java:190)
:run FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':run'.
> Process 'command '/usr/java/jdk1.8.0_91/bin/java'' finished with non-zero exit value 1

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Looking forward to your reply.Thank you very much.

Does not work when integrating with taintdroid

Hi,
I am trying to integrate intellidroid with taintdroid, and I follow the readme instruction.

Here is the steps:

patch and build IntelliDroidService successfully with unmodified AOSP(branch android-4.3_r1).
unpatch the IntelldroidService in DynamicClient/androidPatches.
download and build TaintDroid for android-4.3_r1 successfully, and run it in the emulator.
apply and build the patches in DynamicClient/androidPatches in the same way as step1 successfuly
BUT I cannot run the emulator(stuck in the start scene).
I am confused, can you tell me where I did wrong?

thanks!

TargetedPathAnalysis.getCallbackType() returns wrong type for Broadcast Receiver.

Should this be returning "sms" or "receiver"?

private String getCallbackType(MethodReference callbackMethod){
      ....
      if (declaredClassName.equals("Landroid/content/BroadcastReceiver")) {
            return "sms";
      }
}

thanks,
Joey

Unofficial Wala

Hi,

I notice that IntelliDroid is using unofficial version of Wala. Is it created by the authors of IntelliDroid?

Thanks.

Empty callPaths in appInfo.json

Hi, I am using this tool and test on DroidBench APKs(https://github.com/secure-software-engineering/DroidBench/tree/develop/eclipse-project/ArraysAndLists/ArrayAccess1). However, the output files don't contain call paths as expected.
What might be the problem there?
Btw, My environment is MAC and JDK 1.7.

How to trigger the event correctly

Hi,
When I used the tool,I have completed the app analysis ,got the 'appInfo.json' and 'constraintX_X.py' files and patched the Android OS. Then, I want to dynamically execute the app to trigger the event.

However, I met a problem "Not implemented: ui" as below.

> INSTALL ../Apk/7-app-initial-toast/7-app-initial-toast.apk
1648 KB/s (1413529 bytes in 0.837s)
	pkg: /data/local/tmp/7-app-initial-toast.apk
Success
> START ../AppAnalysis/pathOutput/
Starting: Intent { cmp=com.example.dell.testloader1/.MainActivity }
Status: ok
Activity: com.example.dell.testloader1/.MainActivity
ThisTime: 832
TotalTime: 832
Complete
> TRIGGER 0
Starting: Intent { cmp=com.example.dell.testloader1/.MainActivity }
Warning: Activity not started, its current task has been brought to the front
Status: ok
Activity: com.example.dell.testloader1/.MainActivity
ThisTime: 0
TotalTime: 0
Complete
Not implemented: ui

So,I was wondering the reason of the problem and how to dynamically execute the app correctly.