• Introduction
• Getting Started
  • Prerequisites
  • Setup
  • Running
• Statistics
• Structure
• Technical Report

Let D be a deep learning model that classifies programs according to the problem they solve. This project aims to evaluate how D behaves with obfuscated code. We want to know how much the accuracy of D is affected. This study also evaluates different types of program representations.

The top of the image above shows the histogram produced by a specific strategy for program 292. This program belongs to class 11 of the POJ-104 dataset. The bottom of the image shows how each model classifies the variations of program 292.

In this section are the steps to reproduce our experiments.

You need to install the following packages to run this project:

• Docker and Docker Compose to run our experiments
• Python-3 to plot the results in the project's Jupyter Notebook
• Wget, Tar and Sed to run the initial scripts to configure the repository

First, you should copy the .env.example file and rename it to .env. You can now set environment variables in the .env file at the project's root. You can change the following variables:

After that, you need to prepare the environment to run our experiments. Run the following command line:

$ ./setup.sh

This will download the datasets, build the docker image and create the necessary folders for the project.

Now, you can run the following command line:

$ ./run.sh MODE

There are the following values for MODE:

• build: Builds the docker container based on the modifications in the yali project
• custom: Runs the project based on the variables set on .env file
• all: Runs all experiments available in MODE
• speedup: Runs the speedup analysis with the benchmark game
• embeddings: Runs the embedding analysis
• resources: Runs only the resources analysis
• malware: Runs the experiment to detect classes of malware
• game0 Runs the Game 0
• game1: Runs the Game 1
• game2: Runs the Game 2
• game3: Runs the Game 3
• discover: Runs the Discover Game
• histogram_ext: Runs an accuracy analysis with an extended histogram

This will run the docker container with the configurations in the .env file.

The Statistics folder contains Jupyter Notebooks that plot the data generated by the experiments. Each notebook describes each chart and the steps to develop them. There are the following notebooks:

• EmbeddingResults: Presents information about the accuracy of the dgcnn and cnn models with different representations
• GameResults: Presents information about the 4 games proposed in our work.
• ResourceResults: Presents information about resource consumption (memory and time) of each model
• StrategiesResults: Presents the distance between the histograms of the original programs and the histograms generated by the obfuscators

The repository has the following organization:

|-- Classification: "scripts for the classification process"
|-- Compilation: "Scripts for the compilation process"
|-- Docs: "Repository documentation"
|-- Entrypoint: "Container setup"
|-- Extraction: "Script to extract a program representation and convert CSV to Numpy"
|-- HistogramPass: "LLVM pass to get the histograms"
|-- MalwareDataset: "Malware dataset to support experiments in the project"
|-- Representations: "Scripts to extract different program representations"
|-- Statistics: "Jupyter notebooks"
    |-- Experiments: "Extra experiments using the yali infrastructure (each one of them has its own ReadME)"
    |-- Utils: "Python scripts to support the `Experiments` folder and the Jupyter Notebooks"
|-- Volume: "Volume of the container"
    |-- Csv: "CSVs with the histograms"
    |-- Embeddings: "Different representations of programs in the Source folder"
    |-- Histograms: "histograms in the Numpy format"
    |-- Irs: "LLVM IRs of the programs"
    |-- Results: "Results of the training/testing phase"
    |-- Source: "Source code of the programs"

This framework is used in the following published papers:

• A Game-Based Framework to Compare Program Classifiers and Evaders. To cite it:

@inproceedings{damasio23,
    author = {Dam\'{a}sio, Tha\'{\i}s and Canesche, Michael and Pacheco, Vin\'{\i}cius and Botacin, Marcus and Faustino da Silva, Anderson and Quint\~{a}o Pereira, Fernando M.},
    title = {A Game-Based Framework to Compare Program Classifiers and Evaders},
    year = {2023},
    publisher = {Association for Computing Machinery},
    address = {New York, NY, USA},
    url = {https://doi.org/10.1145/3579990.3580012},
    doi = {10.1145/3579990.3580012},
    booktitle = {Proceedings of the 21st ACM/IEEE International Symposium on Code Generation and Optimization},
    pages = {108–121},
    numpages = {14},
    keywords = {algorithm classification, obfuscation},
    location = {Montr\'{e}al, QC, Canada},
    series = {CGO 2023}
}

• Impacto de Ofuscadores e Otimizadores de Código na Acurácia de Classificadores de Programa. To cite it:

@inproceedings{damasio22,
    author = {Dam\'{a}sio, Tha\'{\i}s and Canesche, Michael and Pacheco, Vin\'{\i}cius and Faustino, Anderson and Quintao Pereira, Fernando Magno},
    title = {Impacto de Ofuscadores e Otimizadores de C\'{o}Digo Na Acur\'{a}Cia de Classificadores de Programas},
    year = {2022},
    publisher = {Association for Computing Machinery},
    address = {New York, NY, USA},
    url = {https://doi.org/10.1145/3561320.3561322},
    doi = {10.1145/3561320.3561322},
    booktitle = {Proceedings of the XXVI Brazilian Symposium on Programming Languages},
    pages = {68–75},
    numpages = {8},
    keywords = {neural network, compiler optimizations, obfuscation},
    location = {Virtual Event, Brazil},
    series = {SBLP '22}
}