Overview Sherpa is Yahoo’s distributed NoSQL key-value store offered as a hosted service. It is the cornerstone of Yahoo properties and hosts over 2,000 tables, 1 trillion records, and serves over 1 million queries a second across a wide spectrum of applications. Though core features described in prior publications (see References section) have solidly served applications over the years, we have c

NodeConf 2014 was fantastic! I love the format, and the location. It was my first NodeConf, and I enjoyed every bit of it. Except the Internet connection. While the WiFi itself was fine, with hundreds of us trying to download packages from the npm registry at the same time, the tiny Internet connection at Walker Creek Ranch left something to be desired. Some folks set up their own registry mirrors

Yahoo stores more than 250 Billion objects and half an exabyte of perpetually durable user content such as photos, videos, email, and blog posts. Object storage at Yahoo is growing at 20-25% annually. The growth is primarily driven by mobile, images, video, and user growth. Yahoo is betting on software defined storage to scale storage cost effectively along with the durability and latency guarante

Pistachio: co-locate the data and compute for fastest cloud compute https://github.com/yahoo/Pistachio is a distributed key value store system. Data can be replicated with n replicas with strong consistency guarantees. Up to (n-1) failures can be tolerated. It’s being used as the user profile storage for large scale ads serving products within Yahoo. 10+ billions of user profiles are being stored

Background: We blogged about the evolution of Yahoo Mail to React + Flux and Node.js. It is important to focus on building a strong foundation when you are building a new platform and having a robust test infrastructure is a big part of the foundation. Yahoo Mail today relies on automated testing on our Continuous Integration pipeline before we deploy changes to production. We run Cucumber and Wai

Using redis with your Python applications just got a whole lot easier! The Yahoo Python Platform team is excited to release redislite, a Python module that allows using redis without the need to install and configure a separate redis server. The redislite module is easy to integrate into existing Python applications that use the existing redis module. Check it out at https://pypi.python.org/pypi/r

The NodeJS Platform Team has released several express/connect middleware packages which provide various useful features for NodeJS based Applications. Here is a brief summary for each of them. dnscache - npm - gitIf your app is using node dns api require('dns') a lot and you want to cache your dns calls, this is the module. It comes with a built-in in-memory cache with various configuration params

From the minds that brought you gifshot! Today we’re proud to open source cronshot, a node module that allows you to schedule, take, alter, and store web page screenshots. Cronshot provides an intuitive API for scheduling time-based and/or individual screenshot tasks. This makes it extremely easy to create a customized and automated screenshot solution. Here’s a simple example that takes a screens

[Performance Comparison] NOTES: These are single-process, single-thread timings. LevelDB does not support multi-process usage, and many features must be lock-protected externally. MDBM iteration (sequential read) is un-ordered. Minimal tuning was performed on all of the candidates. How does MDBM achieve this performance? There are two important components. “Memory Mapping” - It leverages the kerne

Yahoo Mail has been around since 1999. Over the years, the code base has evolved from a completely server rendered Web 1.0 app all the way to one of the largest YUI based rich internet web application today. Last month Yahoo hosted the React JS meetup in the Sunnyvale Campus. We had over 120 people attending and it was great to share knowledge and exchange ideas about Javascript, React, Flux etc.

Today, the Yahoo Presentation Technologies team is proud to announce the launch of a new project named Format.JS — a modular collection of JavaScript libraries for internationalization that are focused on formatting numbers, dates, and strings. Format.JS includes a set of core libraries that build on the JavaScript Intl built-ins and industry-wide i18n standards, a set of integrations for Handleba

The Yahoo User Interface library (YUI) has been in use at Yahoo since 2005, and was first announced to the public on February 13, 2006. Although it has evolved tremendously since that time, YUI has always served the same overarching purpose of providing a comprehensive toolkit to make it easier for developers to create rich web applications. As such, YUI is an important part of Yahoo’s history: mi