Light

nl2code / coder Goto Github PK

View Code? Open in Web Editor NEW

144.0 7.0 17.0 26.5 MB

coder's Introduction

📰 News

[Jun. 4, 2024]: 🎉 We release CodeR, which can solve $28.33$% of issues on SWE-bench lite in the case of submitting only once per issue, Read more in our paper.

🌏 Abstract

GitHub issue resolving recently has attracted significant attention from academia and industry. SWE-bench is proposed to measure the performance in resolving issues. In this paper, we propose CodeR, which adopts a multi-agent framework and pre-defined task graphs to Repair & Resolve reported bugs and add new features within code Repository. On SWE-bench lite, CodeR is able to solve $28.33$% of issues, in the case of submitting only once for each issue. We examine the performance impact of each design of CodeR and offer insights to advance this research direction.

🧪 Results on SWE-agent lite

📗 Citation

@misc{chen2024coder,
      title={CodeR: Issue Resolving with Multi-Agent and Task Graphs}, 
      author={Dong Chen and Shaoxin Lin and Muhan Zeng and Daoguang Zan and Jian-Gang Wang and Anton Cheshkov and Jun Sun and Hao Yu and Guoliang Dong and Artem Aliev and Jie Wang and Xiao Cheng and Guangtai Liang and Yuchi Ma and Pan Bian and Tao Xie and Qianxiang Wang},
      year={2024},
      eprint={2406.01304},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

❣️ Question

Please feel free to arise issues If you have any questions.

coder's People

Contributors

Stargazers

Watchers

Forkers

linshaoxin-maker evdcush nashid jjhw krish240574 cdj0311 sxthunder eltociear happyman76589 dreamplayer-zhang ayeshgk liuyanmei22 fredvinegar aredhelnft evelynmitchell evafighter tecworks-dev

coder's Issues

Source code

Please share your source code

Which multi-agent framework and LLM did you use exactly?

Is this not clear from paper
Are you going to publish it?

Code to reproduce experiments?

Wow, congratulations on the great results on SWE-Bench Lite! We would love to reproduce the results, are there plans to release the code?

Question about manager plans and related-issue-retrieval action

Congratulations on the release and thank you for citing the AutoCodeRover paper!

The paper was a great read. After going through it, I have a couple of clarification questions:

The Plan D part above Figure 3 mentions that "Plan D takes a test-driven approach with a ground truth test for issues (such as 'fail-to-pass' and 'pass-to-pass' tests in SWE-bench)."
Does this mean the developer-written tests for the issue (i.e. the test_patch field in SWE-bench instances) were provided to CodeR?
Section 2 mentions that "Action 18 retrieves the top-1 similar issue and its corresponding patch by description." (action 18 is related issue retrieval from Table 1).
This is an interesting approach! I'm curious how you defined "similarity" between issues - was this using a RAG-based approach on the issue descriptions? Besides, how did you construct the corpus of issues to retrieve from?

Thank you very much in advance for your time and assistance!

Does PlanD use ground truth tests?

Hi, does the plan D mentioned in your paper use "fail-to-pass" tests that are actually used to evaluate the patches?

If so, this would be kind of unfair because most of the other methods do not use those.

Could you maybe specify how many instances in the paper are solved by Plan D?

Set of instances for ablation study

Congratulations on the great results on swebench! We would like to reproduce this work. Do you have any plans to release the code and data (instance id of the small set for the ablation study)?

Plan to open-source

Hi, thanks for the nice work! Do you have a plan to release the source code of CodeR, so that people can learn more about it?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.