RagasMetric breaks when used with dataset about deepeval HOT 5 OPEN

@lmpratti You're probably using the DeepEvalBaseLLM wrapper in the ragas metric. Unforuntately the ragas metric only accept langhcain chat models, you can read the docs here: https://docs.confident-ai.com/docs/metrics-ragas#example

from deepeval.

Hey @haarisedhi102 where are you finding this?

self.llm.set_run_config(run_config)

Also what version of deepeval are you on?

from deepeval.

@penguine-ip This happens on the latest version of deepeval too (0.21.62). I suspect it's due to the dependency on ragas library.
It fails even after adding the below set_run_config to the llm.

    def set_run_config(self, run_config: RunConfig = RunConfig()):
        self.run_config = run_config

from deepeval.

@penguine-ip Thanks a ton! Is there a plan to support DeepEvalBaseLLM wrapped models for ragas metric?

from deepeval.

@lmpratti Unfortunately no because ragas does not have their own ecosystem to support custom models, so out of our control. But we have our own similar implementation of the ragas metrics for this reason

from deepeval.