January 30, 2023 In this post, we'll implement a GPT from scratch in just 60 lines of numpy. We'll then load the trained GPT-2 model weights released by OpenAI into our implementation and generate some text. Note: This post assumes familiarity with Python, NumPy, and some basic experience with neural networks. This implementation is for educational purposes, so it's missing lots of features/improv
What We Learned from a Year of Building with LLMs (Part I)
It’s an exciting time to build with large language models (LLMs). Over the past year, LLMs have become “good enough” for real-world applications. The pace of improvements in LLMs, coupled with a parade of demos on social media, will fuel an estimated $200B investment in AI by 2025. LLMs are also broadly accessible, allowing everyone, not just ML engineers and scientists, to build intelligence into
There are millions of GitHub repos and filtering them is an insane amount of work. It takes a huge time, effort, and a lot more. We have done this for you. In this article, we’ll share a curated list of 100+ widely-known, recommended, and most popular repositories and open source GitHub projects for Machine Learning and Deep Learning. So without further ado, Let’s see all the hubs created by exper
The Andrej Karpathy episode. Andrej explains why reinforcement learning is terrible (but everything else is much worse), why model collapse prevents LLMs from learning the way humans do, why AGI will just blend into the previous ~2.5 centuries of 2% GDP growth, why self driving took so long to crack, and what he sees as the future of education. Watch on YouTube; listen on Apple Podcasts or Spotify
Exactly ten years ago today, we published "Why We Created Julia", introducing the Julia project to the world. At this point, we have moved well past the ambitious goals set out in the original blog post. Julia is now used by hundreds of thousands of people. It is taught at hundreds of universities and entire companies are being formed that build their software stacks on Julia. From personalized me
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
GPT in 60 Lines of NumPy | Jay Mody
100+ Best GitHub Repositories For Machine Learning
Andrej Karpathy — AGI is still a decade away
Why We Use Julia, 10 Years Later