はてなブックマーク

In this post I’ll attempt the fun stunt of designing a system that could serve the full production load of Twitter with most of the features intact on a single (very powerful) machine. I’ll start by showing off a Rust prototype of the core tweet distribution data structure handling 35x full load by fitting the hot set in RAM and parallelizing with atomics, and then do math around how modern high-p

One day a coworker mentioned that he was thinking about APIs for distributed compute clusters and I jokingly responded “clearly the ideal API would be simply calling telefork() and your process wakes up on every machine of the cluster with the return value being the instance ID”. I ended up captivated by this idea: I couldn’t get over how it was clearly silly, yet way easier than any remote job AP

In some domains of programming it’s common to want to write a data structure or algorithm that can work with elements of many different types, such as a generic list or a sorting algorithm that only needs a comparison function. Different programming languages have come up with all sorts of solutions to this problem: From just pointing people to existing general features that can be useful for the

During my final term at UWaterloo I took the CS444 compilers class with a project to write a compiler from a substantial subset of Java to x86, in teams of up to three people with a language of the group’s choice. This was a rare opportunity to compare implementations of large programs that all did the same thing, written by friends I knew were highly competent, and have a fairly pure opportunity

During my final term at UWaterloo I took the CS444 compilers class with a project to write a compiler from a substantial subset of Java to x86, with a language and two teammates of your choice. My group of three chose to write our compiler in Rust and it was a fun experience. We spent time coming to design decisions that worked out really well and used Rust’s strengths. Our compiler ended up being

During my internship at Jane Street1, one of my projects was a config editing tool that at first sounded straightforward but culminated in me designing a custom tree diffing algorithm using dynamic programming, relentlessly optimizing it and then transforming it into an A* accelerated path finding algorithm. This post is the story of the design and optimization of the algorithm, of interest to any

Edit: Some things in this post are now outdated. I’m currently using Sublime Text with Vim keybindings instead of Spacemacs so I haven’t been keeping up. I’ve fixed some things (Thanks Fabien!) but others may remain. If you want to fix something outdated submit a PR to my website. A few months ago I switched to using Spacemacs as my text editor of choice. It has great vim keybindings and extensive

I am currently working on writing an open source gaze tracker in OpenCV that requires only a webcam. One of the things necessary for any gaze tracker1 is accurate tracking of the eye center. For my gaze tracker I had the following constraints: Must work on low resolution images. Must be able to run in real time. I must be able to implement it with only high school level math knowledge. Must be acc

Edit: I’ve recently switched to using Mjolnir and have posted a new tutorial on that. Switching windows with the keyboard on Mac OSX is hilariously inefficient: it involves repeatedly pressing command+tab through millions of programs until you get to the right one when you could have just clicked the window and been done with it. Moving windows is no better so people have resorted to paying for to