Gobblin' Big Data With Ease @ QConSF 2014
Gobblin is a unified data ingestion framework developed by LinkedIn to ingest large volumes of data from diverse sources into Hadoop. It provides a scalable and fault-tolerant workflow that extracts data, applies transformations, checks for quality, and writes outputs. Gobblin addresses challenges of operating multiple heterogeneous data pipelines by standardizing various ingestion tasks and metad
Gobblin' Big Data With Ease
Authors: Shirshanka Das, Lin Qiao The holiday season for gobbling is upon us; and at LinkedIn, we’ve been getting better at gobbling large amounts of different kinds of datasets to feed our data hungry analysts. We thought we’d share with you our most recent efforts at simplifying big data ingestion for Hadoop-based warehouses. Got Data? Every day, the LinkedIn platform generates a ton of data whi
1
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
処理を実行中です
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
How to Choose a Data Format — Silicon Valley Date Science
GitHub - miguno/avro-hadoop-starter: Example MapReduce jobs in Java, Hive, Pig, and Hadoop Streaming that work on Avro data.
GitHub - apache/gobblin: A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.
