The Inherent Limitations of GPT-3

In my last editorial, I went over the fact that GPT-3 was a big deal and caused a large stir in the world of AI. Some thought it such a big deal as to worry about losing jobs and career paths in a post GPT-3 world, and many thought it to be a major leap towards the goal of AGI. But, as Skynet Today covered at the time of its release, much of the hype surrounding GPT-3 was excessive and overestimat

[rawwell]

“A more fundamental limitation of the general approach described in this paper – scaling up any LM-like model, whether autoregressive or bidirectional – is that it may eventually run into (or could already be running into) the limits of the pretraining objective.”

[skypenguins]

GPT-3の限界について要約