microgpt
This is a brief guide to my new art project microgpt, a single file of 200 lines of pure Python with no dependencies that trains and inferences a GPT. This file contains the full algorithmic content of what is needed: dataset of documents, tokenizer, autograd engine, a GPT-2-like neural network architecture, the Adam optimizer, training loop, and inference loop. Everything else is just efficiency.
Introduction to Postgres Indexes – Frontend Masters Blog
This Part 1 (of a 2-part series) is a practical, hands-on, applicable approach to database indexes. We’ll cover what B Trees are with a focus on deeply understanding, and internalizing how they store data on disk, and how your database uses them to speed up queries. This will set us up nicely for part 2, where we’ll explore some interesting, counterintuitive ways to press indexes into service to a
In this paper we propose to study generalization of neural networks on small algorithmically generated datasets. In this setting, questions about data efficiency, memorization, generalization, and speed of learning can be studied in great detail. In some situations we show that neural networks learn through a process of "grokking" a pattern in the data, improving generalization performance from ra
イーロンTwitter買収の意図？ TED2022でのイーロン・マスク 電撃インタビュー by TED クリスアンダーソン｜元Google/アフターデジタル 尾原のＩＴビジネスの原理実践編｜note
尾原3年ぶりの本家TED開催、バンクーバーにおります TEDにてイーロン・マスクが電撃登壇（1度ドタキャンしてた）、TED ヘッドのChrisのインタビューが素晴らしくYouTubeでも公開されてますが、こちらをDeepLをつかった音声認識の文字おこしの日本語訳にしました。 ホストChrisの最初の質問は「なんでTwitterを買収したいの？」 更に話しはテスラ創業の時のHard Thing（多分、初めて話すこと）や自身のアスペルガーの話、そして未来について とめちゃめちゃ展開していきます。 ▼動画（イーロンとの生インタビューからの途中再生です） 事前収録されたインタビューと、リアルタイムでのインタビューがあり、圧倒的に後者がすごいので、そこから読みたい方は「★イーロンマスク：リアルでの生インタビュー」からお読みください（英語の音声認識精度が前半いまいちで調整して後半のいいとこで良くなって
The Andrej Karpathy episode. Andrej explains why reinforcement learning is terrible (but everything else is much worse), why model collapse prevents LLMs from learning the way humans do, why AGI will just blend into the previous ~2.5 centuries of 2% GDP growth, why self driving took so long to crack, and what he sees as the future of education. Watch on YouTube; listen on Apple Podcasts or Spotify
The Art and Mathematics of Genji-Kō by Oran Looney November 26, 2024 Math Visualization History Python You might think it’s unlikely for any interesting mathematics to arise from incense appreciation, but that’s only because you’re unfamiliar with the peculiar character of Muromachi (室町) era Japanese nobles. There has never been a group of people, in any time or place, who were so driven to displa
The Rise (and Lessons Learned) of ML Models to Personalize Content on Home (Part I) | Spotify Engineering
The Rise (and Lessons Learned) of ML Models to Personalize Content on Home (Part I) At Spotify, our goal is to connect listeners with creators, and one way we do that is by recommending quality music and podcasts on the Home page. In this two-part blog series, we will talk about the ML models we build and use to recommend diverse and fulfilling content to our listeners, and the lessons we’ve learn
It's not just you — no one is posting on social media anymore
Tati Bruening, a 22-year-old content creator and photographer, just wants to share memes and post about cooking green beans. Every time she logs onto Instagram, however, her feed is swamped by a combination of perfectly curated photos and professionally created content. "It's really bizarre to me that everyone's gone to this place in their mind that content has to be so curated," Bruening told us.
Engineering Trade-Offs in Automatic Differentiation: from TensorFlow and PyTorch to Jax and Julia - Stochastic Lifestyle
December 25 2021 in Julia, Programming, Science, Scientific ML | Tags: automatic differentiation, compilers, differentiable programming, jax, julia, machine learning, pytorch, tensorflow, XLA | Author: Christopher Rackauckas To understand the differences between automatic differentiation libraries, let’s talk about the engineering trade-offs that were made. I would personally say that none of thes
The Internet Computer is the world’s first blockchain that runs at web speed and can increase its capacity without bound. DFINITY Status Update, New Year 2021I HAVE SOME EXCITING NEWS.On December 18, 2020, a crucial initial stage of Internet Computer blockchain’s decentralization occurred. This means that the Internet Computer’s mainnet now exists, and is hosted by standardized “node machines” tha
Twitter Circle tweets are not that private anymore | TechCrunch
PSA: Do not post your deepest darkest secrets on your Twitter Circle. Numerous Twitter users are reporting a bug in which Circle tweets — which are supposed to reach a select group, like an Instagram Close Friends story — are surfacing on the algorithmically generated For You timeline. That means that your supposedly private posts might breach containment to reach an unintended audience, which cou
In recent years, I’ve read a bunch of people talk about a stagnation in American pop culture. I doubt that this sort of complaint is particularly new. For decades in the mid-20th century, Dwight Macdonald railed against mass culture, which he viewed as polluting and absorbing high culture. In 1980, Pauline Kael wrote an op-ed in the New Yorker entitled “Why Are Movies So Bad? or, The Numbers”, whe
How The New York Times incorporates editorial judgment in algorithms to curate its home page
How The New York Times incorporates editorial judgment in algorithms to curate its home page The Times’ algorithmic recommendations team on responding to reader feedback, newsroom concerns, and technical hurdles. Whether on the web or the app, the home page of The New York Times is a crucial gateway, setting the stage for readers’ experiences and guiding them to the most important news of the day.
Deep-internet bubbles: How microgenres are taking over SoundCloud
Deep-internet bubbles: How microgenres are taking over SoundCloud Once again, Kieran Press-Reynolds makes sense of the SoundCloud craziness. Follow his work at Insider. Consider subscribing to our Patreon. 100% of funds go toward paying our writers, maintaining this website, and further diversifying our reporting efforts. “SO I CREATED A NEW GENRE!!” Behind that text stands a man flexing a Sonic t
The Little Book of Deep Learning
The Little Book of Deep Learning François Fleuret François Fleuret is a professor of computer sci- ence at the University of Geneva, Switzerland. The cover illustration is a schematic of the Neocognitron by Fukushima [1980], a key an- cestor of deep neural networks. This ebook is formatted to fit on a phone screen. Contents Contents 5 List of figures 7 Foreword 8 I Foundations 10 1 Machine Learnin
Potential harms of large language models can be mitigated by watermarking model output, i.e., embedding signals into generated text that are invisible to humans but algorithmically detectable from a short span of tokens. We propose a watermarking framework for proprietary language models. The watermark can be embedded with negligible impact on text quality, and can be detected using an efficient o
Project RADAR: Intelligent Early Fraud Detection System with Humans in the Loop
You’re seeing information for Japan . To see local features and services for another location, select a different city. Show more Introduction Uber is a worldwide marketplace of services, processing thousands of monetary transactions every second. As a marketplace, Uber takes on all of the risks associated with payment processing. Uber partners who use the marketplace to provide services are paid
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets
Andrej Karpathy — AGI is still a decade away
The Art and Mathematics of Genji-Kō
Announcing Internet Computer “Mainnet” and a 20-Year Roadmap
Why has American pop culture stagnated?
A Watermark for Large Language Models