Applying diffs failing silently about gpt-engineer HOT 7 CLOSED

Hi @TheoMcCabe, sorry for the delayed response—I just returned from my trip today. Axel sent me the files, and I've pinpointed the issue. The problem lies with the diff validation; it failed to correct this Docker hunk, and initially, the problematic hunk wasn’t printed in the console from start to finish.

I will create a PR to: 1) address this failure, and 2) ensure the invalid hunk is printed to both the console and the debug file.

Thanks for highlighting this issue, and I apologize for my oversight regarding the hunk output.

from gpt-engineer.

interactively applying diffs im not bothered about - what i find very difficult from a user perspective is that it is very hard for me to know when diffs have failed and havent been applied - you say that ' Outputs indicating which diffs have been discarded are available in the console and logs. ' - i disagree - it is very unclear which diffs have not been applied and this is the problem

When some of my diffs arent applied- the output in the console makes it look like everything has worked fine - this is the bit that needs improving.

My recommendation is that the last thing sent to the user in the console needs to be the diffs which were not sucessfully applied. These need to be outputted into the console in a really easy to read, and copy and paste format... it should use colouring and wording to clearly show that these diffs were not able to be applied and so should be intepreted manually by the user

from gpt-engineer.

awesome thanks @similato87

from gpt-engineer.

Hello @TheoMcCabe ,

Thank you for bringing up this issue regarding the diff application process. I apologize for the inconvenience you've experienced in your project.

You are correct in your understanding of our current strategy for applying diffs:

1. Validation and Correction: We first validate and correct the diffs based on format. If a diff fails validation, we attempt an automated self-heal using our LLMs.
2. Discard Unrecoverable Diffs: If the self-heal process cannot handle the error, we discard these diffs.
3. Apply Valid Diffs: All corrected diffs are then applied.

Outputs indicating which diffs have been discarded are available in the console and logs. This mechanism is designed to provide a smooth experience for users at all levels and allows for multiple attempts to gradually refine a complex code base.

Additionally, we provide users the option to review and manually decide on applying diffs. You can see the planned changes and make an informed decision at this stage of the process:
View Code Here

Would you suggest an interactive approach for applying diffs? For example, showing each validated and corrected diff and allowing users to choose whether to apply them sequentially by user input?

from gpt-engineer.

Do the same diffs work if applied with python-unidiff? ( https://github.com/matiasb/python-unidiff/ )?

Not applying any of the diffs at all would actually be a better outcome for me, as at least i would have a consistent workflow of copy and pasting...

Regarding the use of copy/paste instead of diffs: not having to use diffs would be ideal, but there are limitations of the AI models that make very difficult to get as output the full original code with just the changes made by the AI.

If you want to understand those limitations, I suggest those 2 excellent articles by the Sweep devs:

https://github.com/sweepai/sweep/blob/main/docs/pages/blogs/gpt-4-modification.mdx

and the follow-up:

https://github.com/sweepai/sweep/blob/main/docs/pages/blogs/refactor-python.mdx

from gpt-engineer.

Hi,

yes they should work with python-unidiff, IF the AI makes them correct enough to be corrected into exact unified diffs. The general problem is not applying diffs, but that the AI sometimes delivers low quality diffs.

from gpt-engineer.

https://github.com/sweepai/sweep/blob/main/docs/pages/blogs/refactor-python.mdx

thanks @Emasoft I'm aware of these sweep articles but good to read them again. You seem to have misunderstood the issue i'm raising here, apologies if I wasn't clear enough.

This issue relates to the behaviour of gpte cli when the AI generated unified diffs are not valid and cannot be applied. Specifically i think there is an issue in how this failure is surfaced to the users.

Im not suggesting we rewrite code files from scratch on every run or change our approach to diffing.

from gpt-engineer.