GitHub - lmarena/arena-hard-auto: Arena-Hard-Auto: An automatic LLM benchmark.

🚨 New feature: Style Control is now added to Arena Hard Auto! Check this section to start using style control! Arena-Hard-Auto-v0.1 (See Paper) is an automatic evaluation tool for instruction-tuned LLMs. It contains 500 challenging user queries sourced from Chatbot Arena. We prompt GPT-4-Turbo as judge to compare the models' responses against a baseline model (default: GPT-4-0314). Notably, Arena