Byte Down: Making Netflix’s Data Infrastructure Cost-Effective
By Torio Risianto, Bhargavi Reddy, Tanvi Sahni, Andrew Park Background on data efficiencyAt Netflix, we invest heavily in our data infrastructure which is composed of dozens of data platforms, hundreds of data producers and consumers, and petabytes of data. At many other organizations, an effective way to manage data infrastructure costs is to set budgets and other heavy guardrails to limit spendi
Blessing your SSH at Lyft
Like many organizations, Lyft continually looks for ways to address critical risks identified in our organization. Last summer, Lyft’s Security Team identified a lack of two-factor authentication on our ssh logins as an area of concern. Today, we’re sharing our experience implementing two-factor for SSH at Lyft, and announcing our open-source client for interacting with Netflix’s BLESS. Background
Netflix at AWS re:Invent 2017
by Jason Chan Netflix is excited to be heading back to Las Vegas for AWS re:Invent at the end of the month! Many Netflix engineers and recruiters will be in attendance, and we’re looking forward to meeting and reconnecting with cloud enthusiasts and Netflix OSS users. We’re posting the schedule of Netflix talks here to make it a bit easier to find our speakers at re:Invent. We’ll also have a booth
Netflix Billing Migration to AWS — Part III
In the billing migration blog post published a few weeks ago, we explained the overall approach employed in migrating our billing system to the cloud: In this post, the database migration portion will be covered in detail. We hope that our experiences will help you as you undertake your own migrations. Have you ever wondered about the elements that need to come together and align to get a complica
Distributed Neural Networks with GPUs in the AWS Cloud
by Alex Chen, Justin Basilico, and Xavier Amatriain As we have described previously on this blog, at Netflix we are constantly innovating by looking for better ways to find the best movies and TV shows for our members. When a new algorithmic technique such as Deep Learning shows promising results in other domains (e.g. Image Recognition, Neuro-imaging, Language Models, and Speech Recognition), it
Evolution of the Netflix Data Pipeline
Our new Keystone data pipeline went live in December of 2015. In this article, we talk about the evolution of Netflix’s data pipeline over the years. This is the first of a series of articles about the new Keystone data pipeline. Netflix is a data-driven company. Many business and product decisions are based on insights derived from data analysis. The charter of the data pipeline is to collect, ag
Distributed Time Travel for Feature Generation
We want to make it easy for Netflix members to find great content to fulfill their unique tastes. To do this, we follow a data-driven algorithmic approach based on machine learning, which we have described in past posts and other publications. We aspire to a day when anyone can sit down, turn on Netflix, and the absolute best content for them will automatically start playing. While we are still wa
Using Presto in our Big Data Platform on AWS
by Eva Tse, Zhenxiao Luo, Nezih Yigitbasi @ Big Data Platform team At Netflix, the Big Data Platform team is responsible for building a reliable data analytics platform shared across the whole company. In general, Netflix product decisions are very data driven. So we play a big role in helping different teams to gain product and consumer insights from a multi-petabyte scale data warehouse (DW). Th
Genie - Hadoop Platform as a Service at Netflix
In a prior tech blog (http://nflx.it/XoySYR), we had discussed the architecture of our petabyte-scale data warehouse in the cloud. Salient features of our architecture include the use of Amazon’s Simple Storage Service (S3) as our "source of truth", leveraging the elasticity of the cloud to run multiple dynamically resizable Hadoop clusters to support various workloads, and our horizontally scalab
Asgard: Web-based Cloud Management and Deployment
Asgard is deprecated at Netflix. We use Spinnaker ( www.spinnaker.io ).] Web interface for application deployments and… Some of the information in this blog post is also published in the following presentations. Note that Asgard was originally named the Netflix Application Console, or NAC. Building Cloud Tools for Netflix (Video 2011)Visual Language for the CloudTo help people identify various typ
Benchmarking Cassandra Scalability on AWS
by Adrian Cockcroft and Denis Sheahan Netflix has been rolling out the Apache Cassandra NoSQL data store for production use over the last six months. As part of our benchmarking we recently decided to run a test designed to validate our tooling and automation scalability as well as the performance characteristics of Cassandra. Adrian presented these results at the High Performance Transaction Syst
Netflix on AWS – Perspectives
Disclaimer: The opinions expressed here are my own and do not necessarily represent those of current or past employers. Recent CommentsRaffaele Santopaolo on David Patterson Retires After 40 YearsJames Hamilton on Seagate HAMRTom Davies on Seagate HAMRMatt on Seagate HAMRJames Hamilton on Seagate HAMRMatt on Seagate HAMRJames Hamilton on Cost of Power in Large-Scale Data CentersChris K. on Cost of
1
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
GitHub - Netflix/repokid: AWS Least Privilege for Distributed, High-Velocity Deployment
How Netflix Gives All Its Engineers SSH Access To Instances Running In Production
Daniel Weeks of Netflix and Rahul Bhartia of AWS, speak at the AWS Big Data Meetup in San Francisco
GitHub - Netflix/Raigad: Co-Process for backup/recovery, Auto Deployments and Centralized Configuration management for ElasticSearch
GitHub - Netflix/genie: Distributed Big Data Orchestration Service
Real-time Recommendations using Spark Comcast Labs