ブックマーク / www.aisnakeoil.com (1)

  • GPT-4 and professional benchmarks: the wrong answer to the wrong question

    OpenAI didn’t release much information about GPT-4 — not even the size of the model — but heavily emphasized its performance on professional licensing exams and other standardized tests. For instance, GPT-4 reportedly scored in the 90th percentile on the bar exam. So there’s been much speculation about what this means for professionals such as lawyers. We don’t know the answer, but we hope to inje

    GPT-4 and professional benchmarks: the wrong answer to the wrong question
    ABA
    ABA 2023/03/22
    GPT-4は競技プログラミングコンテストサイトCodeforcesの問題のうち、2021/9/5までの問題は解けるが、2021/9/12以降の問題が全く解けないという話。GPT-4に入っているトレーニングデータは2021/9までのもの
  • 1