You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
When running Evaluations where gpt4oMini is the judge the response of the judge for the GenkitMetric.ANSWER_RELEVANCY response does not follow the required JSON schema. When I switched the judge to gemini20Flash everything worked fine.
Error for the Relevancy Metric:
Describe the bug
When running Evaluations where
gpt4oMini
is the judge the response of the judge for theGenkitMetric.ANSWER_RELEVANCY
response does not follow the required JSON schema. When I switched the judge togemini20Flash
everything worked fine.Error for the Relevancy Metric:
To Reproduce
Setup of Evaluations and Metrics:
Expected behavior
Evaluations where all three metrics are scored correctly
Runtime (please complete the following information):
** Node version
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: