Developers are increasingly relying on AI coding assistants to accelerate our daily workflows. These tools can autocomplete functions, suggest bug fixes, and even generate entire modules or MVPs. Yet, as many of us have learned, the quality of the AI’s output depends largely on the quality of the prompt you provide. In other words, prompt engineering has become an essential skill. A poorly phrased
January 30, 2023 In this post, we'll implement a GPT from scratch in just 60 lines of numpy. We'll then load the trained GPT-2 model weights released by OpenAI into our implementation and generate some text. Note: This post assumes familiarity with Python, NumPy, and some basic experience with neural networks. This implementation is for educational purposes, so it's missing lots of features/improv
Onyx, a new programming language powered by WebAssemblyLearn about Onyx, a new imperative programming language that leverages WebAssembly and Wasmer for seamless cross-platform support What is Onyx? Onyx is a new programming language featuring a modern, expressive syntax, strict type safety, blazingly-fast build times, and out-of-the-box cross platform support thanks to WebAssembly. Over the past
Golang Mini Reference 2022: A Quick Guide to the Modern Go Programming Language (REVIEW COPY)
Golang Mini Reference 2022 A Quick Guide to the Modern Go Programming Language (REVIEW COPY) Harry Yoon Version 0.9.0, 2022-08-24 REVIEW COPY This is review copy, not to be shared or distributed to others. Please forward any feedback or comments to the author. • feedback@codingbookspress.com The book is tentatively scheduled to be published on September 14th, 2022. We hope that when the release da
AST vs. Bytecode: Interpreters in the Age of Meta-Compilation
233 AST vs. Bytecode: Interpreters in the Age of Meta-Compilation OCTAVE LAROSE, University of Kent, UK SOPHIE KALEBA, University of Kent, UK HUMPHREY BURCHELL, University of Kent, UK STEFAN MARR, University of Kent, UK Thanks to partial evaluation and meta-tracing, it became practical to build language implementations that reach state-of-the-art peak performance by implementing only an interprete
Note: I received a lot of great feedback from the discussions at Mastodon and Hacker News, so I've updated the post with some improvements to the font! I've also added some further examples and acknowledgements at the end. Syntax Highlighting in Hand-Coded Websites The problem I have been trying to identify practical reasons why hand-coding websites with HTML and CSS is so hard (by hand-coding, I
Patterns for Building LLM-based Systems & Products [ llm engineering production 🔥 ] · 66 min read Discussions on HackerNews, Twitter, and LinkedIn “There is a large class of problems that are easy to imagine and build demos for, but extremely hard to make products out of. For example, self-driving: It’s easy to demo a car self-driving around a block, but making it into a product takes a decade.”
Solving Quantitative Reasoning Problems With Language Models
Solving Quantitative Reasoning Problems with Language Models Aitor Lewkowycz∗, Anders Andreassen†, David Dohan†, Ethan Dyer†, Henryk Michalewski†, Vinay Ramasesh†, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur∗, Guy Gur-Ari∗, and Vedant Misra∗ Google Research Abstract Language models have achieved remarkable performance on a wide range of tasks that require
The Andrej Karpathy episode. Andrej explains why reinforcement learning is terrible (but everything else is much worse), why model collapse prevents LLMs from learning the way humans do, why AGI will just blend into the previous ~2.5 centuries of 2% GDP growth, why self driving took so long to crack, and what he sees as the future of education. Watch on YouTube; listen on Apple Podcasts or Spotify
The Art and Mathematics of Genji-Kō by Oran Looney November 26, 2024 Math Visualization History Python You might think it’s unlikely for any interesting mathematics to arise from incense appreciation, but that’s only because you’re unfamiliar with the peculiar character of Muromachi (室町) era Japanese nobles. There has never been a group of people, in any time or place, who were so driven to displa
Accelerate Python code 100x by import taichi as ti | Taichi Docs
Python has become the most popular language in many rapidly evolving sectors, such as deep learning and data sciences. Yet its easy readability comes at the cost of performance. Of course, we all complain about program performance from time to time, and Python should certainly not take all the blame. Still, it's fair to say that Python's nature as an interpreted language does not help, especially
Python behind the scenes #13: the GIL and its effects on Python multithreading
As you probably know, the GIL stands for the Global Interpreter Lock, and its job is to make the CPython interpreter thread-safe. The GIL allows only one OS thread to execute Python bytecode at any given time, and the consequence of this is that it's not possible to speed up CPU-intensive Python code by distributing the work among multiple threads. This is, however, not the only negative effect of
This article is a translation, the original version is available here. I started using the OCaml language regularly around 2012, and since then, my interest and enthusiasm for this language have only grown. It has become my preferred choice for almost all my personal projects, and it has also influenced my professional choices. Since 2014, I have been actively participating in public conferences d
Agents for Amazon Bedrock now support memory retention and code interpretation (preview) | Amazon Web Services
AWS News Blog Agents for Amazon Bedrock now support memory retention and code interpretation (preview) With Agents for Amazon Bedrock, generative artificial intelligence (AI) applications can run multistep tasks across different systems and data sources. A couple of months back, we simplified the creation and configuration of agents. Today, we are introducing in preview two new fully managed capab
Try Mastering Emacs for free! Are you struggling with the basics? Have you mastered movement and editing yet? When you have read Mastering Emacs you will understand Emacs. It’s that time again: there’s a new major version of Emacs and, with it, a treasure trove of new features and changes. Notable features include the formal inclusion of native compilation, a technique that will greatly speed up y
vim9script4pythondevelopers.md Vim9 script for Python Developers Vim9 script�Vim script��������������������������������������������������系��� def������義����������Vim script��vim9script�����使����������(vim9script���
Type Parameters Proposal
Ian Lance Taylor Robert Griesemer August 20, 2021 StatusThis is the design for adding generic programming using type parameters to the Go language. This design has been proposed and accepted as a future language change. We currently expect that this change will be available in the Go 1.18 release in early 2022. AbstractWe suggest extending the Go language to add optional type parameters to type an
Faster virtual machines: Speeding up programming language execution - Mort's Ramblings
Date: 2023-01-15 Git: https://gitlab.com/mort96/blog/blob/published/content/00000-home/00015-fast-interpreters.md In this post, I hope to explore how interpreters are often implemented, what a "virtual machine" means in this context, and how to make them faster. Note: This post will contain a lot of C source code. Most of it is fairly simple C which should be easy to follow, but some familiarity w
Plan 9 Desktop Guide
PLAN 9 DESKTOP GUIDE INDEX What is Plan 9? Limitations and Workarounds Connecting to Other Systems VNC RDP SSH 9P Other methods Porting Applications Emulating other Operating Systems Virtualizing other Operating Systems Basics Window Management Copy Pasting Essential Programs Manipulating Text in the Terminal Acme - The Do It All Application Multiple Workspaces Tiling Windows Plumbing System Admin
Large Text Compression Benchmark Matt Mahoney Last update: July 3, 2025. history This competition ranks lossless data compression programs by the compressed size (including the size of the decompression program) of the first 109 bytes of the XML text dump of the English version of Wikipedia on Mar. 3, 2006. About the test data. The goal of this benchmark is not to find the best overall compressi
An AI-native software engineer is one who deeply integrates AI into their daily workflow, treating it as a partner to amplify their abilities. This requires a fundamental mindset shift. Instead of thinking “AI might replace me” an AI-native engineer asks for every task: “Could AI help me do this faster, better, or differently?”. The mindset is optimistic and proactive - you see AI as a multiplier
This December I once again did the Advent of Code, in Rust. If you are interested, my solutions are on Github. I wanted to highlight one particular solution to the day 2 problem as it is both optimized completely beyond the point of reason yet contains a useful technique. For simplicity we’re only going to do part 1 of the day 2 problem here, but the exact same techniques apply to part 2. We’re go
While getting my degree in Physics, I had to take classes in both MatLab and Python for scientific computing. I preferred python, where we used the SciPy and NumPy packages. In fact, I used those packages again (along with matplotlib) in an undergraduate research project simulating bacteria films. There's a catch: I was also pursuing a degree in Computer Science, and Python just wasn't fast enough
Tetris That Hates You StickManStickMan #611, by Sam Hughes. HATETRIS is a version of Tetris written in 2010 by programmer and sci-fi author Sam Hughes. According to his initial description of the game: This is bad Tetris. It’s hateful Tetris. It’s Tetris according to the evil AI from “I Have No Mouth And I Must Scream”. (And if you aren’t familiar with Tetris at all, and don’t know the rules or pi
Pandas is a very popular tool for data analysis. It comes built-in with many useful features, it's battle tested and widely accepted. However, pandas is not always the best tool for the job. SQL databases have been around since the 1970s. Some of the smartest people in the world worked on making it easy to slice, dice, fetch and manipulate data quickly and efficiently. SQL databases have come such
I find blockchain fascinating because it extends open source software development to open source + state. This seems to be a genuine/exciting innovation in computing paradigms; We don’t just get to share code, we get to share a running computer, and anyone anywhere can use it in an open and permissionless manner. The seeds of this revolution arguably began with Bitcoin, so I became curious to dril
Why APL is a language worth knowing
“A language that doesn't affect the way you think about programming, is not worth knowing.”, by Alan J. Perlis. Why APL is a language worth knowing Alan Perlis, the computer scientist recipient of the first Turing award, wrote “A language that doesn't affect the way you think about programming, is not worth knowing.” ― Alan J. Perlis, 1982. Special feature: Epigrams on programming. ACM Sigplan Not
JupyterLab Changelog — JupyterLab 4.5.0a3 documentation
JupyterLab Changelog# v4.4# JupyterLab 4.4 includes a number of new features (described below), bug fixes, and enhancements. This release is compatible with extensions supporting JupyterLab 4.0. Extension authors are encouraged to consult the Extension Migration Guide which lists deprecations and changes to the public API. Code console improvements# The code console prompt can now be positioned on
K-nearest neighbors (k-NN) is a Machine Learning algorithm for supervised machine learning type. It is used for both regression and classification tasks. As we already know, a supervised machine learning algorithm depends on labeled input data, which the algorithm learns to produce accurate outputs when input unlabeled data. k-NN aims to predict the test data set by calculating the distance betwee
Scheduling Internals
A sneak peek to what's coming! I remember when I first learned that you can write a server handling millions of clients running on just a single thread, my mind was simply blown away 🤯 I used Node.js while knowing it is single threaded, I used async / await in Python, and I used threads, but never asked myself "How is any of this possible?". This post is written to spread the genius of concurrenc
I write a lot of reductions: loops that combine every element from a collection in some way. For example, summing a vector of integers: (reduce (fn [sum x] (+ sum x)) 0 [1 2 3]) ; => 6 If you’re not familiar with Clojure’s reduce, it takes a reducing function f, an initial accumulator init, and a collection xs. It then invokes (f init x0) where x0 is the first element in xs. f returns a new accumu
GTF :: Why Haskell?
“Impractical”, “academic”, “niche”. These are a few of the reactions I get when someone discovers that my favourite programming language is Haskell, and not only my favourite in some sort of intellectually-masturbatory way, but favourite for building things, real things, mostly involving web servers. Hobby projects would be one thing, but it gets worse: I have actual teams at Converge working in H
It’s 11 o’clock. Do you know where your variables are pointing? def shout(obj) obj.to_s + "!" end It’s hard to tell just looking at the code what type obj is. We assume it has a to_s method, but many classes define methods named to_s. Which to_s method are we calling? What is the return type of shout? If to_s doesn’t return a String, it’s really hard to say. Adding type annotations would help… a l
Today, a lot of agentic applications (such as, Microsoft Copilot, ChatGPT plugins, AutoGPT, etc) automate a variety of tasks by LLM reasoning, the ability to split a complex task into simpler subtasks. Reasoning+Acting (shortly, ReAct) is essential and origin for these task completion, which is achieved by advanced LLM reasoning. To build such autonomous agents, here I’ll show you the fundamentals
Fitting a Forth in 512 bytes
Fitting a Forth in 512 bytes June 10, 2021 · 31 minute read This article is part of the Bootstrapping series, in which I start from a 512-byte seed and try to bootstrap a practical system. Software is full of circular dependencies if you look deep enough. Compilers written in the language they compile are the most obvious example, but not the only one. To compile a kernel, you need a running kerne
NumPy library on Python is an essential tool for data scientists to work on numerical data, especially when they deal with data arrays, especially multi-dimensional, and need a memory-efficient fast indexing of arrays, However, knowing about other useful packages when solving data science problems is essential. So, let’s see which packages are available in Python programming language and are used
My birthday just passed, and to relax I wrote a parser combinator library. Over the last few years, I have worked quite a bit with Ningning Xie and Jeremy Yallop on parser combinators, which has led to a family of parser combinators which have optimal linear-time performance in theory, and which are many times faster than lex+yacc in practice. But these use advanced multistage programming techniqu
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
The Prompt Engineering Playbook for Programmers
GPT in 60 Lines of NumPy | Jay Mody
Onyx, a new programming language powered by WebAssembly · Blog · Wasmer
Font with Built-In Syntax Highlighting
Patterns for Building LLM-based Systems & Products
Andrej Karpathy — AGI is still a decade away
The Art and Mathematics of Genji-Kō - OranLooney.com
xvw.lol - Why I chose OCaml as my primary language
What's New in Emacs 28.1?
Vim9 script for Python Developers · GitHub
Large Text Compression Benchmark
The AI-Native Software Engineer
The World's Smallest Hash Table | orlp.net
Scientific Computing in Rust - aftix's dominion
Getting the World Record in HATETRIS
Practical SQL for Data Analysis
A from-scratch tour of Bitcoin in Python
k-NN (k-Nearest Neighbors) in Supervised Machine Learning
Loopr: A Loop/Reduction Macro for Clojure
Interprocedural Sparse Conditional Type Propagation
ReAct (Reason+Act) prompting in LLMs
NumPy for Data Science Beginners in Python
Linear-time parser combinators