はてなブックマーク

I recently read an excellent article in the Amazon Builder’s Library by Clare Liguori which goes into great detail about AWS’s CI/CD architecture. It’s a truly brilliant post and I recommend everyone interested in CI/CD infrastructures read the article. The article covers the gamut from unit testing to integration testing (in production) to staged rollouts in “waves” to automated rollbacks. I guar

As 2019 draws to a close, I wanted to jot down some thoughts on some of the most important technological adoptions and innovations in tech this past decade. I also look a bit into the future, enumerate a list of pain points and opportunities that can be addressed in the coming decade. I must add the caveat here that this post doesn’t cover developments in fields like data science, artificial intel

Author’s Note: Thanks, as ever, to Fred Hebert, for reading a draft of this post and making some sterling suggestions. This is the third installment in my series on testing distributed systems. The posts in this series are the following: Testing Microservices, the sane way (published December 2017) Testing in Production, the safe way (published March 2018) Testing in Production: the hard parts (pu

My previous post covered the fundamentals of file descriptors as well as some of the most commonly used forms on non-blocking I/O operations on Linux and BSD. I had some people wonder why it didn’t cover epoll at all, but I’d mentioned in the conclusion of that post that epoll is by far the most interesting of all and as such warranted a separate post in its own right. epoll stands for event poll

Author’s Note: I’m greatly indebted to Marc McBride for reading a draft of this post and offering some excellent suggestions. This is the second installment in my series on testing distributed systems. The posts in this series are the following: Testing Microservices, the sane way (published December 2017) Testing in Production, the safe way (published March 2018) Testing in Production: the hard p

Author’s Note: Thanks, as ever, to Fred Hebert, for reading a draft of this post and making some sterling suggestions. This is the first installment in my series on testing distributed systems. The posts in this series are the following: Testing Microservices, the sane way (published December 2017) Testing in Production, the safe way (published March 2018) Testing in Production: the hard parts (pu

What really are descriptors?The fundamental building block of all I/O in Unix is a sequence of bytes. Most programs work with an even simpler abstraction — a stream of bytes or an I/O stream. A process references I/O streams with the help of descriptors, also known as file descriptors. Pipes, files, FIFOs, POSIX IPC’s (message queues, semaphores, shared memory), event queues are all examples of I/

At last Wednesday’s Go meetup, one of the speakers mentioned: This wasn’t posed as a question. This was a statement. This wasn’t the first time I was hearing this. A lot of people conflate the two or mistake one for the other, perhaps because many monitoring tool vendors claim to offer both. To be fair, there is a certain amount of overlap between the two even if they differ in enough ways to be c

Note — Huge thanks to Jamie Wilkinson and Julius Volz, Google SREs present and past, for reading a draft of this and giving me invaluable suggestions. All mistakes and opinions however, are solely mine. This was, in essence, my Velocity 2017 talk. The infrastructure space is in the midst of a paradigm-shifting change. The way organizations — from the smallest of startups to established companies —

During lunch with a few friends in late July, the topic of observability came up. I have a talk coming up at Velocity in less than a month called Monitoring in the time of Cloud Native, so I’ve been speaking with friends about how they approach monitoring where they work. During this conversation, one of my friends mentioned: He was only half joking. I’ve heard several variations of this zinger, s

In this post, I aim to: — Shed light on some of the presumed benefits of small functions — Explain why I personally think some of the benefits don’t really pan out as well as advertised — Explain why small functions can actually prove counterproductive sometimes — Explain the times when I do think smaller functions truly shine General programming advice doled out invariably seems to extoll the ele

Yesterday was Sysadmin Appreciation Day. There was a lot of chatter about what the future of Operations will look like, a recurrent theme being that in this day and age, Operations is “everyone’s job” or that “everyone is Ops”. While I think people who believe this have their hearts in the right place, it’s a tad simplistic or opportunistic view. The reality on the ground happens to be more nuance

This post aims to understand: — the purpose of schedulers the way they were originally envisaged and developed at Google — how well (or not) they translate to solve the problems of the rest of us — why they come in handy even when not running “at scale” — the challenges of retrofitting schedulers into existing infrastructures — running hybrid deployment artifacts with schedulers — why at imgix we

It was November 1st, 2016. I’d spent the better part of the day at a training on Go + Distributed Computing. O’Reilly had offered me a free ticket in exchange for feedback. During the training, I’d got to re-familiarize myself with a number of concepts around distributed systems in general and the Go microservices ecosystem in particular. On the first day the topics the training covered were servi