Custom LLM as a Judge to Detect Hallucinations with Braintrust | OpenAI Cookbook

Oct 14, 2024Custom LLM as a Judge to Detect Hallucinations with Braintrust Let's say you're working on a customer service bot and trying to evaluate the quality of its responses. Consider a question like "What is your return policy?" If the correct answer is "You can return items within 30 days of purchase," but your bot generates "You can return items within 30 days," how would you evaluate wheth

[soratokimitonoaidani]

LLMに頼めばハルシネーションも出力してくれるのか……なるほど……となった。

[arrowKato]

本題とはずれるが、LangSmith, Langfuseっぽい評価システムとして、breantrust というプラットフォームがあるそうな。