Increase in temperature does not seem to induce randomness about ice HOT 5 CLOSED

@stuhlmueller @reppertj @CG80499 could one of you take a look?

from ice.

I suspect the local diskcache is preventing subsequent calls from actually taking place. The cache_id parameter exists to break this cache; if you, e.g., increment this cache_id on each call from the agent (e.g. using itertools.count), you should get randomness. Alternatively you can remove this cache decorator.

from ice.

I mean that does work and is straightforward to implement the iterator, though I might argue sampling completions is a primitive enough action that this behaviour might be more easily accessible; YMMV

from ice.

There are some awkward tradeoffs here re: reproducibility/cost savings to be sure (since the OpenAI API doesn't allow you to parameterize pseudo-randomness in sampling). Depending on the use case, by the way, you could also use the n parameter to get multiple samples through a single call.

from ice.

It sounds like this is just working exactly as intended, but there may be some revisions down the line on functionality, so happy to close it

from ice.