Turn expensive prompts into cheap fine-tuned models.
Hosted App - Running Locally - Experiments
Use powerful but expensive LLMs to fine-tune smaller and cheaper models suited to your exact needs. Evaluate model and prompt combinations in the playground. Query your past requests and export optimized training data. Try it out at https://app.openpipe.ai or run it locally.
-
Experiment
- Compare models and prompts against one another.
- Bulk-test wide-reaching scenarios using code templating.
- Autogenerate scenarios for fresh test perspectives.
-
Fine-Tune (Beta)
- Easy integration with OpenPipe's SDK in both Python and JS.
- Swiftly query logs using intuitive built-in filters.
- Export data in multiple training formats, including Alpaca and ChatGPT, with deduplication.
These are sample experiments users have created that show how OpenPipe works. Feel free to fork them and start experimenting yourself.
- Open-Orca/OpenOrcaxOpenChat-Preview2-13B
- Open-Orca/OpenOrca-Platypus2-13B
- NousResearch/Nous-Hermes-Llama2-13b
- jondurbin/airoboros-l2-13b-gpt4-2.0
- lmsys/vicuna-13b-v1.5
- Gryphe/MythoMax-L2-13b
- NousResearch/Nous-Hermes-llama-2-7b
- Install Postgresql.
- Install NodeJS 20 (earlier versions will very likely work but aren't tested).
- Install
pnpm
:npm i -g pnpm
- Clone this repository:
git clone https://github.com/openpipe/openpipe
- Install the dependencies:
cd openpipe && pnpm install
- Create a
.env
file (cp .env.example .env
) and enter yourOPENAI_API_KEY
. - Update
DATABASE_URL
if necessary to point to your Postgres instance and runpnpm prisma migrate dev
to create the database. - Create a GitHub OAuth App, set the callback URL to
<your local instance>/api/auth/callback/github
, e.g.http://localhost:3000/api/auth/callback/github
. - Update the
GITHUB_CLIENT_ID
andGITHUB_CLIENT_SECRET
values from the Github OAuth app (Note: a PR to make auth optional when running locally would be a great contribution!). - Start the app:
pnpm dev
. - Navigate to http://localhost:3000
- Copy your
.env
file to.env.test
. - Update the
DATABASE_URL
to have a different database name than your development one - Run
DATABASE_URL=[your new datatase url] pnpm prisma migrate dev --skip-seed --skip-generate
- Run
pnpm test