Kafka with AVRO vs., Kafka with Protobuf vs., Kafka with JSON Schema
Kafka serialisation schemes — playing with AVRO, Protobuf, JSON Schema in Confluent Streaming Platform. The code for these examples available at https://github.com/saubury/kafka-serialization Apache Avro was has been the default Kafka serialisation mechanism for a long time. Confluent just updated their Kafka streaming platform with additional support for serialising data with Protocol buffers (or
How Apache Kafka Inspired Our Platform Events Architecture - Salesforce Engineering Blog
There’s a class of scalability challenges that are best approached using an asynchronous, event-driven architecture — particularly when it comes to doing data integrations. To support these uses cases Salesforce recently added a feature called “Platform Events” which exposes a time-ordered immutable event stream to our customers. Our multitenant setup posed some challenges for traditional messagin
Kafka Ecosystem at LinkedIn
Apache Kafka is a highly scalable messaging system that plays a critical role as LinkedIn’s central data pipeline. Kafka was developed at LinkedIn back in 2010, and it currently handles more than 1.4 trillion messages per day across over 1400 brokers. Kafka’s strong durability and low latency have enabled us to use Kafka to power a number of newer mission-critical use cases at LinkedIn. These incl
Spark Meetup at Uber
1) Uber uses Spark and Hadoop to process large amounts of transportation data in real-time and batch. This includes building pipelines to ingest trip data from databases into a data warehouse within 1-2 hours. 2) Paricon is Uber's first Spark application which infers schemas from raw JSON data, converts it to Parquet format for faster querying, and validates the results. It processes over 15TB of
Introducing Confluent Platform 4.0
I am very excited to announce the general availability of Confluent Platform 4.0, the enterprise distribution of Apache Kafka 1.0. This release includes a number of significant improvements, including enhancements to Confluent Control Center, Kafka Connect, the Streams API, and of course, Apache Kafka 1.0 itself. Looking back on the previous three major releases of the Confluent Platform, we can s
1
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
処理を実行中です
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
Open-Sourcing Yelp's Data Pipeline
GitHub - LinkedInAttic/camus: LinkedIn's previous generation Kafka to HDFS pipeline.
