Code Monkey home page Code Monkey logo

uniapr's Introduction

UniAPR: Fast and Precise On-the-fly Patch Validation for All

Introduction

UniAPR is an efficient patch validation framework. Upon this framework, different automatic program repair (APR) techniques can be installed as patch generation add-ons. This has a couple of immediate benefits: (1) one can take advantage of the power of different APR technique each of which is good in handling different kind of bugs; (2) APR research community can focus on developing more and more sophisticated patch generation mechanisms without worrying about the cost of patch validation which dominates end-to-end program repair time.

Installation

Before installation, make sure you have the required softwares on your machine:

  • Version Control System: Git
  • Java Development Kit: JDK 8
  • Java Build System: Maven 3+
  1. Clone the repository of UniAPR source code.

    git clone https://github.com/ise-uiuc/uniapr.git
  2. Compile and install the UniAPR jar file to local maven repository. By default, the jar file will be installed to $HOME/.m2/repository/org/uniapr/uniapr-plugin/1.1/uniapr-plugin-1.1.jar.

    cd uniapr
    mvn clean install

Start Patch Validation

Preparation

Before beginning the patch validation process, you need to set up a patches-pool directory with the appropriate hierarchy under the root path of the project which you want to validate patches for.

For example, let's say you want to validate patches generated for the com.google.javascript.jscomp.ProcessClosurePrimitives class in the Closure project. Under the root path of the Closure project, create a directory named patches-pool with the following structure:

patches-pool
├── 0
│   └── com
│       └── google
│           └── javascript
│               └── jscomp
│                   ├── ProcessClosurePrimitives$1.class
│                   ├── ProcessClosurePrimitives.class
│                   ├── ProcessClosurePrimitives$ProvidedName.class
│                   └── ProcessClosurePrimitives$UnrecognizedRequire.class
├── 1
│   └── com
│       └── google
│           └── javascript
│               └── jscomp
│                   ├── ProcessClosurePrimitives$1.class
│                   ├── ProcessClosurePrimitives.class
│                   ├── ProcessClosurePrimitives$ProvidedName.class
│                   └── ProcessClosurePrimitives$UnrecognizedRequire.class
...

In the example provided, "0" and "1" can be any numerical value that represents the patch ID. Note that you can include multiple class files in a single patch directory.

Validation

To validate patches for a project, run the following command under its root path:

mvn org.uniapr:uniapr-plugin:validate -Dhttps.protocols=TLSv1.2

Optional Configuration

Id Options Description
1 d4jAllTestsFile The path of the all_tests file generated by defects4j test command. This option should be specified when validating patches for Defects4J projects. It ensures that UniAPR will run the same set of tests, in the same order, as they are executed by Defects4J. Note that this option has been tested on Defects4J V1.2.0 projects.
2 restartJVM Controls whether to launch a new JVM for each patch validation. If set to false, UniAPR will try to validate all patches whithin the same JVM (unless the JVM crashs), which means the side effect of test cases may carry over from one patch to another (refer to the resetJVM option for more information). If this option is set to true, UniAPR will start a new JVM for each patch validation, which will result in a slower process but with improved precision. By default, this setting is set to false.
3 argLine Configures JVM options for JVMs spawned by UniAPR.
4 profilerOnly Only run the profiler of UniAPR, i.e., executing all the tests found by UniAPR without validating any patches. This setting is mainly used for debugging purposes, such as checking if UniAPR has found the correct set of tests and executes them in the same order as Defects4J. By default, this setting is set to false.
5 failingTests The tests that fail in the buggy project (the project that UniAPR is being used to validate). If not specified, UniAPR will first run the profiler to collect the failing tests, and then execute these tests before any other tests during the validation process.
6 whiteListPrefix The class name prefix UniAPR used to distinguish application classes from library classes for profiling. By default, groupId of the project will be used as a whitelist prefix.
7 patchesPool The name of the directory that holds the patches for UniAPR validation. By default, UniAPR looks for the directory named patches-pool under the base directory.
8 resetJVM Turn on or off UniAPR's JVM-reset functionality. If disabled and the restartJVM is set to false, the effects of test case execution may carry over from one patch to another, potentially leading to imprecise results from UniAPR. However, UniAPR will run faster in this mode. By default, this setting is set to false.

Configure within Command Line

If you do not want to modify the pom.xml in the project for validation, you can directly run and configure UniAPR whithin command line. For example:

mvn org.uniapr:uniapr-plugin:validate -Dhttps.protocols=TLSv1.2 -Dd4jAllTestsFile=all_tests -DrestartJVM=true

Configure within pom.xml

You can also configure UniAPR within the pom.xml file:

<plugin>
    <groupId>org.uniapr</groupId>
    <artifactId>uniapr-plugin</artifactId>
    <version>1.1</version>
    <!-- <configuration>                                                            -->
    (1)   <!-- <d4jAllTestsFile>./all_tests</d4jAllTestsFile>                       -->
  
    (2)   <!-- <restartJVM>true</restartJVM>                                        -->
  
    (3)   <!-- <argLine>-Xmx8g</argLine>                                            -->
  
    (4)   <!-- <profilerOnly>false</profilerOnly>                                   -->
  
    (5)   <!-- <failingTests>                                                       -->
          <!--  <failingTest>fully.qualified.test.Class1::testMethod1</failingTest> -->
          <!--    ...                                                               -->
          <!--  <failingTest>fully.qualified.test.ClassN::testMethodN</failingTest> -->
          <!-- </failingTests>                                                      -->

    (6)   <!-- <whiteListPrefix>${project.groupId}</whiteListPrefix>                -->
    
    (7)   <!-- <patchesPool>patches-pool</patchesPool>                              -->

    (8)   <!-- <resetJVM>false</resetJVM>                                           -->
        
    <!-- </configuration>                                                           -->
</plugin>

Support for Validating Defects4J Projects

The scripts in d4j-maven directory aid in the automatic conversion of Defects4J projects into properly configured maven projects, ensuring that UniAPR's profiler produces consistent test results with Defects4J. Currently, only Defects4J V1.2.0 projects are supported, and support for Defects4J V2.0.0 projects is still underway. For further details, please refer to the README.md in https://github.com/ise-uiuc/uniapr/tree/main/d4j-maven.

Example

The directory example/Lang-6 is an Defects4J maven project created by the script in the d4j-maven directory, containing a preconfigured POM file and a prepared all_tests file (generated by the defects4j test command). We also include the patches generated by CapGen, a mutation-/template-based APR technique, in the example/Lang-6/patches-pool directory. To run the example, you can run the following commands:

cd example/Lang-6
mvn org.uniapr:uniapr-plugin:validate -Dhttps.protocols=TLSv1.2 -Dd4jAllTestsFile=all_tests

Please note that we need to include the option -Dhttps.protocols=TLSv1.2 when invoking Maven to satisfy a security requirement in place since June 2018.

Running UniAPR without JVM-reset feature activated will result in UniAPR failing to find the plausible patch (see the output in example/Lang-6/validateWithoutResetJVM.log). This is due to the fact that test case execution side-effects are being propagated from one patch validation session to another (as UniAPR strives to do all patch validations in the same process as long as it is possible). To mitigate this, you may want to activate JVM-reset feature. In order to run UniAPR with JVM-reset feature activated, the user can either modify the POM file and invoke UniAPR as mentioned before, or simply try the following command:

mvn org.uniapr:uniapr-plugin:validate -Dhttps.protocols=TLSv1.2 -Dd4jAllTestsFile=all_tests -DresetJVM=true

As the output in example/Lang-6/validateWithResetJVM.log shown, the plausible patch with ID 216 is found successfully by UniAPR.

We have tested the use of UniAPR with Defects4J V1.2.0 projects on our machine. If you encounter any problems, whether in V1.2.0 or V2.0.0 Defects4J projects, please don't hesitate to create an issue to bring it to our attention.

uniapr's People

Contributors

instein98 avatar

Stargazers

 avatar owhvayifuqq avatar  avatar  avatar KevinYoung avatar Ginni avatar Yuxiang Wei avatar claudeyj avatar  avatar Chenyuan Yang avatar

Watchers

 avatar  avatar

uniapr's Issues

Questions about the implementation of UniAPR

Hello authors, I want to express my gratitude for the insightful information you've shared about program testing and the UniAPR project. I have three points of curiosity regarding the implementation of UniAPR, and I hope the authors can provide some answers:

  1. UniAPR currently executes failing tests before other tests and exits the process if any test fails. I'm curious about the possibility of creating a priority queue of tests that would be executed before other tests. These tests are ranked by their failure frequency throughout the entire process. With this queue, the tool would execute tests that are more likely to fail first. Would this approach be more efficient, or would it have a minimal impact?
  2. UniAPR leverages pitest to execute tests, even though pitest ultimately invokes the JUnit runner to execute JUnit tests. I'm interested in understanding why the authors chose pitest as the execution backend. Would an implementation using the primitive JUnit runner be more efficient?
  3. I've encountered an exception: 'java.lang.IllegalAccessException: Class org.uniapr.jvm.core.JVMStatus cannot access a member of class.' Is this a known issue, or a feature? I'm interested in whether this exception affects the test results.

Thank you for your time and insights.

buggy patch passes all test cases of Chart-8 of Defects4j.

When I tested my APR (Automatic Program Repair) tool on the defects4j dataset, I found that if I provide the same patch code to UniApr as the original buggy version (i.e., without modifying anything), the patch still manages to pass all test cases. Here's the reproduction method:

  1. Check out the Chart-8 project using defects4j.
  2. Use the 'defects4j test' command to compile and run the test cases (we'll get a failing test case: org.jfree.data.time.junit.WeekTests::testConstructor).
  3. Create a 'patches-pool' folder under the Chart-8 project and copy the org/jfree/data/time/Week.class file from the build folder into the 'patches-pool' folder according to the specified format.
  4. Run: 'mvn org.uniapr:uniapr-plugin:validate -DrestartJVM=true -Dd4jAllTestsFile=all_tests'.
  5. The result shows that the patch can pass all test cases.

That's quite unusual.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.