Pysa: An open source static analysis tool to detect and prevent security issues in Python code Today, we are sharing details about Pysa, an open source static analysis tool we’ve built to detect and prevent security and privacy issues in Python code. Last year, we shared how we built Zoncolan, a static analysis tool that helps us analyze more than 100 million lines of Hack code and has helped engi

Facebook’s infrastructure consists of many geo-distributed data centers that host millions of servers. These servers run a number of systems, ranging from front-end web servers to News Feed aggregators to our messaging and live video applications. In addition to regular code pushes, we also push thousands of configuration changes to our servers every day. Thus it is fairly common for our servers t

As a first step toward encrypting the last portion of internet traffic that has historically been cleartext, we have partnered with Cloudflare DNS on a pilot project. This pilot takes advantage of the benefits of Transport Layer Security (TLS) — a widely adopted and proven mechanism for providing authentication and confidentiality between two parties over an insecure channel — in conjunction with

Open-sourcing oomd, a new approach to handling OOMs As our global community has grown to more than 2.2 billion people, Facebook’s infrastructure has grown to span News Feed, Messenger, Instagram, WhatsApp, Oculus, and a range of other products. These products and the systems powering them run on millions of servers spread across multiple geo-distributed data centers. As our infrastructure has scal

To the billions of people using Facebook, our services may look like a single, unified mobile app or website. From inside the company, the view is different. Facebook is built using thousands of services, with functions ranging from balancing internet traffic to transcoding images to providing reliable storage. The efficiency of Facebook as a whole is the sum of the efficiencies of its individual

Open-sourcing Katran, a scalable network load balancer Katran creates a software-based solution to load balancing with a reengineered forwarding plane that takes advantage of recent innovations in kernel engineering. With billions of people around the globe using Facebook services, our infrastructure engineers have created a range of systems to optimize traffic and to enable fast, reliable access

A log is the simplest way to record an ordered sequence of immutable records and store them reliably. Build a data intensive distributed service and chances are you will need a log or two somewhere. At Facebook, we build a lot of big distributed services that store and process data. Want to connect two stages of a data processing pipeline without having to worry about flow control or data loss? Ha

Facebook’s DHCP infrastructure and traffic imbalance Last year we talked about our production DHCP infrastructure and how we use ISC KEA in production to deploy a stateless and dynamic DHCP server across the fleet. At the time, we used a hash-based ECMP selection algorithm for our BGP anycast IP distribution. A question arose about whether the DHCP traffic was well-balanced across our KEA servers.

Consistent security controls and high reliability are common expectations for any systems administrator. How do you deliver both on a network with thousands of servers supporting thousands of engineers? Most off-the-shelf solutions require a compromise in at least one of these areas — and we refused to accept this. Most systems administrators use the industry-standard Secure Shell (SSH) for access

At Facebook we use MySQL to manage many petabytes of data, along with the InnoDB storage engine that serves social activities such as likes, comments, and shares. While InnoDB provides great performance and reliability for a variety of workloads, it has inefficiencies on space and write amplification when used with flash storage. A few years ago, we built RocksDB, an embeddable, persistent key-val

Derek Jouppi, a hardware design engineer intern at Facebook Inc., tests servers at the company's hardware labs in Menlo Park, California, U.S., on Monday, April 7, 2014. The largest social network said it is buying Oculus VR Inc., pushing into wearable hardware for the first time and stepping into a race with Google Inc. Irvine, California-based Oculus makes a ski-goggles-like device called Rift,

As more and more people around the world start logging on to Facebook, we have an increasingly large responsibility to keep things fast. This is especially true in developing areas, where devices stay in the market longer and people have longer upgrade cycles for new devices. We want to make sure we look into possible opportunities for performance improvements across all of our major mobile platfo

Facebook has a huge IPv6 deployment, so it is no surprise that we’ve pushed the Linux IPv6 stack to the limit. One of our pain points in deploying IPv6 is the size of the routing tree. We’ve solved this scalability issue by creating routing cache on demand. In Linux, the size of the IPv6 routing tree grows as the number of peers that a machine is talking to grows. For example, if a machine has a ‘

Two billion photos are shared daily on Facebook services. Many of these photos are important memories for the people on Facebook and it's our challenge to ensure we can preserve those memories as long as people want us to in a way that's as sustainable and efficient as possible. As the number of photos continued to grow each month, we saw an opportunity to achieve significant efficiencies in how w

Today, we’re proud to announce a first, experimental release of h2tp, or the “HH (Hack) Transpiler,” a tool which allows projects that have converted from PHP to Hack to still make releases that target the PHP language. Since the launch of Hack, many community members have asked us how to manage forward compatibility. Hack is backwards-compatible with PHP – if you’re running PHP on HHVM, Hack code

Introducing data center fabric, the next-generation Facebook data center network The more than 1.35 billion people who use Facebook on an ongoing basis rely on a seamless, “always on” site performance. On the back end, we have many advanced sub-systems and infrastructures in place that make such a real-time experience possible, and our scalable, high-performance network is one of them. Facebook’s

Introducing mcrouter: A memcached protocol router for scaling memcached deployments Most web-based services begin as a collection of front-end application servers paired with databases used to manage data storage. As they grow, the databases are augmented with caches to store frequently-read pieces of data and improve site performance. Often, the ability to quickly access data moves from being an

In a hurry? Try Hack now: http://hacklang.org/ Today we’re releasing Hack, a programming language we developed for HHVM that interoperates seamlessly with PHP. Hack reconciles the fast development cycle of PHP with the discipline provided by static typing, while adding many features commonly found in other modern programming languages. We have deployed Hack at Facebook and it has been a great succ

Every day people upload more than 350 million photos to Facebook (as of Dec 2012) and view many more in their News Feeds and on their friends’ Timelines. Facebook stores these photos on Haystack machines that are optimized to store photos. But there is also a deep and distributed photo-serving stack with many layers of caches that delivers photos to people so they can view them. We recently publis

With thousands of commits a week across hundreds of thousands of files, Facebook’s main source repository is enormous–many times larger than even the Linux kernel, which checked in at 17 million lines of code and 44,000 files in 2013. Given our size and complexity—and Facebook’s practice of shipping code twice a day–improving our source control is one way we help our engineers move fast. Choosing