While at Netflix and LinkedIn Siddharth "Sid" Anand wrote some great articles for HS. Sid is back, this time as a Data Architect at Agari. Original article is here. Data-rich companies (e.g. LinkedIn, Facebook, Google, and Twitter) have historically built custom data pipelines over bare metal in custom-designed data centers. In order to meet strict requirements on data security, fault-tolerance, c
![Leveraging AWS to Build a Scalable Data Pipeline - High Scalability -](https://cdn-ak-scissors.b.st-hatena.com/image/square/a87578626af034a6009cc1be80d2429219b39670/height=288;version=1;width=512/https%3A%2F%2Ffarm1.staticflickr.com%2F289%2F18538985561_189a1ba290_o.jpg)