Code Monkey home page Code Monkey logo

rims_minimal's Introduction

rims_minimal

이전 코드베이스가 엉망인 관계로 제공함
시작점: https://github.com/fgenie/Model-Selection-Reasoning/tree/59debd8441e7cb4b7d733f256b20870073e36c08

What to do before run

place openai_key.txt into utils/

How to Contribute

pip install pre-commit
pre-commit install

How to run

1 model-selection-resoning baseline

python run_inference.py  baseline_inference \
                --backbone chatgpt \
                --outdir dbgoutdir/ \
                --gsm_jslf ../dataset/dbg.jsonl \
                --dataset_type gsm # [ocw, math, gsm, svamp]
                [--start_idx 0 ] # can start running from the middle of the data
                [--dbg] # runs with tqdm instead of pqdm

2 rims algorithm run the 1's result (it will skip the non-conflict examples!)

python run_inference.py  rims_inference  \
                --backbone  chatgpt  \
                --outdir dbgoutdir/ \
                --prompt_f  prompt_construction_src/prep_rims_prompts/gsm_prompts/3_reflectonce_cot2p2c.pal2cot.pal2p2c.txt_rm_ans   \
                --gsm_jslf dbgoutdir/chatgpt_01_18_04_48_model_selection3_startidx0.jsonl \
                --dataset_type gsm # [ocw, math, gsm, svamp]

3 evaluate

# baseline result
python run_evaluation.py --eval_jslf dbgoutdir/chatgpt_01_18_04_48_model_selection3_startidx0.jsonl
# rims result
python run_evaluation.py --eval_jslf dbgoutdir/chatgpt_rims_01_18_04_49_startidx0.jsonl

todo

  • add
    • NameError when running baseline_inference on MATH dataset?
    • ocw, (math) symbolic including prompts

algorithm

TBA

rims_minimal's People

Contributors

fgenie avatar seanexp avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.