500 error when passing data to the dashboard using validate_api about tonic_validate HOT 3 CLOSED

Thanks @ojpetar.

Its not clear from the code snippet but I reviewed our logs and this happened during a run where you were using our AnswerSimilarityMetric but not setting an 'answer' in your benchmark. We'll get this fixed ASAP. In the meantime, for anyone else encountered the same issue make sure to construct your benchmark as:

Benchmark(questions=["question1", "question2", ...], answers=["", "", ...])

when you are just having Validate score the answer against its context and do not provide reference answers. In our fix we will do away with the need to specify an answers param at all to the Benchmark constructor.

from tonic_validate.

Awesome that worked! Thanks so much 🚀

from tonic_validate.

We fixed this on our backend. We should have the server updated tonight to fix the issue

from tonic_validate.