We’ve trained a large-scale unsupervised language model which generates coherent paragraphs of text, achieves state-of-the-art performance on many language modeling benchmarks, and performs rudimentary reading comprehension, machine translation, question answering, and summarization—all without task-specific training. Our model, called GPT-2 (a successor to GPT), was trained simply to predict the
![Better language models and their implications](https://cdn-ak-scissors.b.st-hatena.com/image/square/3e00533aded0c697448dda479dd92c324082c62b/height=288;version=1;width=512/https%3A%2F%2Fimages.openai.com%2Fblob%2F8df9d1ca-5128-41d0-89de-edc1fc75ca57%2Fbetter-language-models.jpg%3Ftrim%3D836%252C0%252C0%252C0%26width%3D1000%26quality%3D80)