Stanford CS336 | Language Modeling from Scratch

Logistics Lectures: Monday/Wednesday 3:00-4:20pm in Skilling Auditorium Office hours: Percy Liang: Fridays 11-12am in Gates 366 Tatsu Hashimoto: Tuesdays 11-12am in Gates 364 Marcel Rød: Tuesdays 4:30-5:30pm in Gates 498, Wednesdays 4:30-5:30pm in Gates 415 Herman Brunborg: Wednesdays 1:30-2:30pm, Fridays 1:30-2:30pm, location Gates 392 Steven Cao: Mondays 4:30-5:30pm, Thursdays 9:30-10:30am, Gate

[nguyen-oi]

トランスフォーマーからアライメントまでスクラッチで作るの激アツ。これ自習するだけでも相当力つくわ

[misshiki]

Stanford CS336は言語モデルをゼロから作る実装重視講義。Tokenizer、Transformer、Triton/FlashAttention2、分散訓練、Common Crawl処理、SFT/RLまで扱う。