Improving language understanding with unsupervised learning

Our system works in two stages; first we train a transformer model on a very large amount of data in an unsupervised manner—using language modeling as a training signal—then we fine-tune this model on much smaller supervised datasets to help it solve specific tasks. We developed this approach following our sentiment neuron work, in which we noted that unsupervised learning techniques can yield sur

[suzukiMY]

教師無し学習による言語理解の改善

[elu_18]

Improving Language Understanding with Unsupervised Learning OpenAI から凄まじい技術が報告された。教師なし学習を利用。言語理解タスクを大幅に改善。かなり多くのデータセット（12種類のデータセット）で最高の成績を記録（SoTA）。 htt