archiki / grips Goto Github PK

Hi Archiki Prasad,
I want to achieve the result after the Instruction + Examples run_search.py of the InstructGPT curie of table 6, which is 59.44 .
But there is only "Instruction + Positive Examples" mode in function custom_instruction_prompt, no "Instruction + Examples" mode. Thus, I failed to achieve the best results. I don't know if there is a problem with my understanding of the code, or the existing code does not achieve this effect.

 I think I need your help to achieve this result, or do you have any suggestions, thanks!

Hi Archiki Prasad,

I want to reproduce the results of GrIPS initializing with task-agnostic instructions with Instruction-Only prompts in your paper.
With the hyperparameters in the paper which is l = 1, m = 5, n = 10, and P = 2, I have achieved the accuracy of model InstructGPT babbage in table 3, which is 54.41. However, I cannot achieve the accuracy of model GPT-2 XL in table 3, which is 54.29. My result is about 53.00.
I have tried several train seeds and different hyperparameters, but still failed to achieve the accuracy 54.29.
I need your help to reproduce the result. Could you give me some suggestions.