Code Monkey home page Code Monkey logo

lemur-program-verification's Introduction

Lemur

This repo contains the source code for the paper Lemur: Integrating Large Language Models in Automated Program Verification

Requirements

The implementation is tested on python3.8.10, but should work on more recent python versions.

Python packages

The Python package dependencies can be installed by

pip install -r requirements.txt

OpenAI API

You also need a working OPEN API key, and set the environment variable OPENAI_API_KEY.

export OPENAI_API_KEY=YOUR_KEY

Solvers

You need to download two software verification tools by running

./build.sh

Experiments

There are two experiments that we performed. The source code as well as the log files for each are contained in "code2inv/" and "lemur/", respectively.

Citation

@inproceedings{wu2023lemur,
  title={Lemur: Integrating Large Language Models in Automated Program Verification},
  author={Wu, Haoze and Barrett, Clark and Narodytska, Nina},
  booktitle={The Twelfth International Conference on Learning Representations},
  year={2024}
}

lemur-program-verification's People

Contributors

wu-haoze avatar

Stargazers

breandan avatar  avatar

Watchers

 avatar  avatar

lemur-program-verification's Issues

Error while running the tool

I tried running the run_exp_gpt4.sh script in the code2inv/ directory with the following arguments:

./run_exp_gpt4.sh ./benchmarks/code2inv/c/1

and I got the following exception:

Traceback (most recent call last):
  File "/mnt/c/Users/user/repos/Lemur-program-verification/code2inv/./src/run.py", line 41, in <module>
    v = Verifier(task, VERIFIERS, args)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/c/Users/user/repos/Lemur-program-verification/code2inv/src/verifier.py", line 38, in __init__
    self.program = Program(r.lines_to_verify, r.replacement)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/c/Users/user/repos/Lemur-program-verification/code2inv/src/program.py", line 61, in __init__
    if line.strip().split()[0] in ["for", "do", "while"]:
       ~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

Before running the run_exp_gpt4.sh script, I installed the packages in requirements.txt and ran the ./build.sh script.
I tried running the tool with python 3.11 and python 3.8

Is there any additional setup I need to complete for the tool before running it? Or pass any other CLI arguments?

Benchmark seletion of SV-COMP

Hi,

Fantastic work! I have a small question regarding the selection of 47 benchmarks from SV-COMP: What is the specific implementation of the sampling? Or is there any available code that we can reimplement the sampling? Thank you so much!

Error while running 117.c

I'm trying to reproduce the results.

When I run the command ./run_exp_gpt3.sh ./benchmarks/code2inv/c/117, I got the following error:
error

This looks like the postcondition information is missing during the inference.

I've followed your README.md to download the required package and installed the required tools with ./build.sh.

I'm looking forward to your favourable reply.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.