GitHub - pfnet-research/pfgen-bench: Preferred Generation Benchmark

pfgen-benchmark is a benchmark designed to evaluate Japanese text generation specifically for pretrained models. Unlike conventional benchmarks that use templates containing instructions, this benchmark relies solely on providing numerous examples. By conveying expectations such as the question-answering nature of the task, responses of approximately 100 characters, and outputs resembling formal p

[arrowKato]

LLM as a Judgeの代わりになるかもしれない評価方法。軽いのがメリット。日本語の設問が50個ある。