[feature req] New model for testing: DummyAgent about swe-agent HOT 4 OPEN

Kilian I'd love to hear your thoughts. Just wanted to file this before I forget.

from swe-agent.

Isn't that very similar to the replay model though? Where would we get the commands that fix the issue?

If we had gpt4 solve an issue before, we can take the trajectory and use the replay model to replay everything as if the LM was doing it.

This is the primary means by which I'm testing swe-agent at the moment :) -- though I should add some more trajectories

from swe-agent.

ok let me talk to you about this next time we chat, i think we can make this DummyAgent a bit smarter and therefore it would maybe be better than the replay solution. anyways this is not urgent at all

from swe-agent.

ok I talked with Kilian and one of the tests almost does exactly this and at some point we'll update that test to verify environment outputs too

from swe-agent.