krrishdholakia / betterprompt Goto Github PK

You don't want to provide your API keys?:)

ValueError: Did not find OPENAI_API_KEY, please add an environment variable OPENAI_API_KEY which contains it, or pass OPENAI_API_KEY as a named parameter.

Hi,

I have been fooling around with this and I observe that the perplexity values are usually rather small --

e.g.

args=  GenericIdeaCreator the quick brown fox jumped over the lazy: text-ada-001 None
opening file app/presets/GenericIdeaCreator.json
2023-01-12 12:25:02.071 INFO    openai: message='Request to OpenAI API' method=post path=https://api.openai.com/v1/completions
2023-01-12 12:25:02.731 INFO    openai: message='OpenAI API response' path=https://api.openai.com/v1/completions processing_ms=90 response_code=200
perplexity 0.0006873648694770481

As you might, this is on the high side relative to my more specific or tailored prompts, which generally return values between 0.0002 and 0.0004.

Three questions:

1. do these values seem right?
2. in the paper you rely upon (Demystifying Prompts) the perplexities are given as positive integers with the lowest shown around 10. Is there a conversion happening here?
3. Should the engine make any difference in perplexity?

Contact Details

test

What happened?

A bug happened!

Version

1.0.2 (Default)

What browsers are you seeing the problem on?

Firefox

Relevant log output

test

Code of Conduct

• I agree to follow this project's Code of Conduct

Need to update the calculation of perplexity to be - Not the perplexity of the prompt but the perplexity of the test completions given the prompt.

A potential way to structure this is - have it taken in a couple sample test cases, and your prompt, and output perplexity. Could also have it take in multiple prompts.