Evolution of the Netflix Data Pipeline

Our new Keystone data pipeline went live in December of 2015. In this article, we talk about the evolution of Netflix’s data pipeline over the years. This is the first of a series of articles about the new Keystone data pipeline. Netflix is a data-driven company. Many business and product decisions are based on insights derived from data analysis. The charter of the data pipeline is to collect, ag

[deeeet]

NetflixとApache Kafkaだ．似たようなPipelineをつくってるので非常に参考になる

[kimutansk]

途中からリアルタイム要素に応じてKafkaが加わり、その後Kafkaがfrontをはるようになったと。Routing部はSamzaなんですね。ピークは秒間800万イベント＆24Gのデータと。