The Rakuda Ranking of Japanese AI

Rakuda is a ranking of Japanese Large Language Models based on how well they answer a set of open-ended questions in Japanese about Japanese topics. We hope that Rakuda can help stimulate the development of open-source models that perform well in Japanese, in the spirit of English-language leaderboards like Huggingface's human_eval_llm. For a detailed explanation of how Rakuda works, please check

[kitokitoki]

YuzuAIグループが出した日本語LLMを評価するRakudaベンチマーク

[secondlife]

日本の歴史や地理・国策等々のドメインが要求される課題に対してLLMに出力させ、どの回答がもっともらしいかをGPT-4に評価(Elo)させたランキング。