はてなブックマーク

As a FreeBSD developer — and someone who writes in C — I believe strongly in the idea of "tools, not policy". If you want to shoot yourself in the foot, I'll help you deliver the bullet to your foot as efficiently and reliably as possible. UNIX has always been built around the idea that systems administrators are better equipped to figure out what they want than the developers of the OS, and it's

On Thursday I wrote about the problem of zeroing buffers in an attempt to ensure that sensitive data (e.g., cryptographic keys) which is no longer wanted will not be left behind. I thought I had found a method which was guaranteed to work even with the most vexatiously optimizing C99 compiler, but it turns out that even that method wasn't guaranteed to work. That said, with a combination of tricks

In cryptographic applications, it is often useful to wipe data from memory once it is no longer needed. In a perfect world, this is unnecessary since nobody would gain unauthorized access to that data; but if someone is able to exploit an unrelated problem — a vulnerability which yields remote code execution, or a feature which allows uninitialized memory to be read remotely, for example — then en

While at Amazon re:invent I had the opportunity to complain to some Amazonians again about an EC2 bug which has been annoying me for a long time: The default firewall rulset is broken. I discovered this three years ago while debugging odd problems experienced by a Tarsnap user — sending a small amount of traffic worked fine, but as soon as large amounts of traffic started moving around, the TCP co

When I tell North Americans about my time at Oxford University, one aspect of its undergraduate program inevitably surprises them: Final examinations. Rather than writing examinations at the end of each term, students in most subjects write a single set of examinations at the end of their final year, on which their entire degree performance is measured. (In recent years, some subjects have switche

This is part 1 of my software development final exam. If you haven't read that introductory blog post, please go read it now. Algorithms and Data Structures Is O(2^n) equal to O(3^n)? Why? What is the expected run-time of quicksort? What is the worst-case run-time? What is the difference between a binary tree [EDIT: that should be binary search tree] and a B-tree? When and why does this difference

In the world of POSIX, everything is a file. Well, sort of. There's sockets and pipes, which behave rather like files except that you can't seek on them and they have some extra metadata. And there's devices, where sometimes you can only read and write appropriately-sized blocks, not individual bytes. And then there's terminals, which are all sorts of weird. But in all these cases, you've got a fi

One of the questions I am asked most often about FreeBSD Update is "how can I build my own updates?". Usually I've pointed people at the FreeBSD Update server source code and wished them luck; in most cases I've heard back a while later that after spending a few days trying they gave up. I'm happy to say that thanks to Jason Helfman and Experts Exchange I can now point people at a far more useful

The following page detailed the state of FreeBSD/EC2 development until the middle of 2015. EC2 is now fully supported by FreeBSD, with AMIs built by the FreeBSD release engineering team and announced in release and snapshot announcements; and this page is no longer being updated. For lists of FreeBSD AMI IDs, please consult the appropriate FreeBSD release announcement; you can also launch FreeBSD

One of my largest complaints about Amazon EC2 ever since it launched has been my inability to run FreeBSD on it. Judging from the feedback I received to two earlier blog posts, I haven't been alone. The problems keeping FreeBSD out of EC2 have always been more FreeBSD-related than Amazon-related, however, and over the past month I've been hacking away at FreeBSD's Xen code, to the point where I ca

Daily Hacker News for 2025-07-12 The 10 highest-rated articles on Hacker News on July 12, 2025 which have not appeared on any previous Hacker News Daily are: The ChompSaw: A benchtop power tool that's safe for kids to use (comments) Lead pigment in turmeric is the culprit in a global poisoning mystery (2024) (comments) ETH Zurich and EPFL to release a LLM developed on public infrastructure (commen

Last month, Amazon published a code sample which demonstrated the use of SimpleDB as a repository for S3 object metadata. This code sample would probably have gone almost completely unnoticed if it were not for one detail: Using a pool of 34 threads in Java, the code sample sustained 300 SimpleDB operations per second when running on a small EC2 instance. Only 300? We can do better than that... Fi

I recently came across two articles, "Counting characters in UTF-8 strings is fast" by Kragen Sitaker, and "Counting characters in UTF-8 strings is fast(er)" by George Pollard, which provide a series of successively faster ways of (as the article names suggest) counting the number of UTF-8 characters in a NUL-terminated string. We can do better. Kragen takes the approach of examining each byte in

bsdiff and bspatch are tools for building and applying patches to binary files. By using suffix sorting (specifically, Larsson and Sadakane's qsufsort) and taking advantage of how executable files change, bsdiff routinely produces binary patches 50-80% smaller than those produced by Xdelta, and 15% smaller than those produced by .RTPatch (a $2750/seat commercial patch tool). These programs were or

I've posted in the past about using a modified version of FreeBSD Update to upgrade from FreeBSD 6.1 to FreeBSD 6.2. I've spent some time working on this code and I think it's now generic enough that it should work for future release upgrades, so I'll soon be committing it to the FreeBSD base system. Consequently, I'm making this post both for the present (in the middle of the FreeBSD 7.0 release

I posted yesterday about using FreeBSD Update to perform FreeBSD minor version upgrades, e.g., upgrading from FreeBSD 6.2 to FreeBSD 6.3 or from FreeBSD 7.0-BETA1.5 to FreeBSD 7.0-BETA2. Today I'm going to write about the more complicated process needed for major version upgrades, e.g., upgrading from FreeBSD 6.x to FreeBSD 7.x. NOTE: Before proceeding, if you have FreeBSD Update scheduled to run

I've posted here before about a script I wrote for performing binary upgrades of FreeBSD 6.0 system to FreeBSD 6.1; seeing that one of the greatest difficulties of the FreeBSD release cycle is convincing people to install and test all of the -BETA and -RC (release candidate) snapshots produced, I have put (most of) this code together with the new FreeBSD Update code to produce a version of FreeBSD

Many computer systems around the world have been possessed by penguins; some have even been possessed by dead rats. In light of this, it is desireable to exorcize these evil spirits, and replace them with a nice, friendly daemon. (More to the point, there are a number of dedicated server hosting companies which only offer Linux (or, in some cases, Linux and Windows); being able to remotely replace

Portsnap is a system for securely downloading and updating a compressed snapshot of the FreeBSD ports tree, and using this compressed snapshot to extract or update a (uncompressed) copy of the ports tree. Historically, most people have used CVSup to keep their ports tree up to date, but CVSup has a number of limitations: CVSup is insecure. The protocol uses no encryption or signing, and any attack

In February 2005 I put instructions online explaining how to perform a binary upgrade from FreeBSD 4.8 to FreeBSD 4.11. When it came time to upgrade my FreeBSD 5.3 system to instead run FreeBSD 5.4, I found that some slightly different steps were needed, so I decided to put together a 5.3-to-5.4 upgrade guide as well. What follows are the steps I used to upgrade my FreeBSD 5.3 system to FreeBSD 5.