Manhattan, our real-time, multi-tenant distributed database for Twitter scale
Manhattan, our real-time, multi-tenant distributed database for Twitter scale As Twitter has grown into a global platform for public self-expression and conversation, our storage requirements have grown too. Over the last few years, we found ourselves in need of a storage system that could serve millions of queries per second, with extremely low latency in a real-time environment. Availability and
Hadoop/Storm の統合を実現する Twitter の SummingBird - Okapies' Archive
Twitter が SummingBird を正式リリースして早二ヶ月。「日本語の紹介記事がほとんど出てないな」と気付いたので、調査がてらまとめてみました。 SummingBird とは？ MapReduce なプログラムを書くための Scala/Java ライブラリ。最大の特徴は、ひとたび SummingBird で書いたジョブは Hadoop でも Storm でも同じように実行できること。 SummingBird では、Hadoop を使う「バッチモード」と、Storm を使う「リアルタイムモード」に加えて、二つを同時に実行する「ハイブリッドモード」がある。ハイブリッドモードでは、ジョブの作者が特に配慮しなくても、バッチとリアルタイムの処理結果を自動的にマージできる。 ハイブリッドモードでは、同じジョブを Hadoop と Storm で同時に実行できるので、Hadoop の耐障害性
New Tweets per second record, and how!
Recently, something remarkable happened on Twitter: On Saturday, August 3 in Japan, people watched an airing of Castle in the Sky, and at one moment they took to Twitter so much that we hit a one-second peak of 143,199 Tweets per second. (August 2 at 7:21:50 PDT; August 3 at 11:21:50 JST) To give you some context of how that compares to typical numbers, we normally take in more than 500 million Tw
TwitterOct2008.pdf
This document provides style guidelines for Scala projects at Twitter. It outlines recommendations for imports, implicit usage, reflection, comments, whitespace, logging, project layout, variable naming conventions, and more. The document encourages placing all imports at the top of files, avoiding implicits and reflection when possible, writing Scaladoc comments for all classes and non-trivial me
Finagle, from Twitter
Finagle is an extensible RPC system for the JVM, used to construct high-concurrency servers. Finagle implements uniform client and server APIs for several protocols, and is designed for high performance and concurrency. Most of Finagle’s code is protocol agnostic, simplifying the implementation of new protocols. Finagle is written in Scala, but provides both Scala and Java idiomatic APIs. GitHub p
Private Presentation
Private content!This content has been marked as private by the uploader.
Twitter Streaming API Architecture
Follow a Tweet from creation to timeline and Streaming API delivery. The design of streaming within Twitter is influenced by the entire Twitter architecture, the direction of the platform, data syndication policies and Quality of Service requirements. We'll discuss these influences and our system implementation.Read less
Twitterが分散フレームワーク「Gizzard」公開！ Scalaで書かれたShardingを実現するミドルウェア
Twitterが分散フレームワーク「Gizzard」公開！ Scalaで書かれたShardingを実現するミドルウェア Twitterは独自に開発した分散フレームワークの「Gizzard」をオープンソースとして公開しました。GizzardはScalaで書かれたJavaVM上で動作するミドルウェアで、PHPやRubyといったWebアプリケーションからの要求を自動的にデータベースに分散することで、大規模で可用性の高い分散データベースを容易に実現するためのものです。 Gizzard：フォルトトレラントな分散データベースを実現 The Twitter Engineering Blog: Introducing Gizzard, a framework for creating distributed datastores Twitterのブログにポストされた「Introducing Gizzard
BAY AREA SCALA ENTHUSIASTS WELCOME TO TWITTER! HELP YOURSELF TO BEER AND SUCH IN THE KITCHEN. ♥ TWITTER’S SCALA HEADS ROBEY STEVE ALEX STYLE GUIDELINES IMPORTS ENUMERATE ALL IMPORTS USE WILDCARDS FOR MORE THAN SIX USE BRACE NOTATION FOR MULTIPLE ITEM
BAY AREA SCALA ENTHUSIASTS WELCOME TO TWITTER! HELP YOURSELF TO BEER AND SUCH IN THE KITCHEN. ♥ TWITTER’S SCALA HEADS ROBEY STEVE ALEX STYLE GUIDELINES IMPORTS ENUMERATE ALL IMPORTS USE WILDCARDS FOR MORE THAN SIX USE BRACE NOTATION FOR MULTIPLE ITEMS IN A PACKAGE AVOID “RELATIVE” IMPORTS ALL IMPORTS MUST BE AT THE TOP OF THE FILE IMPLICITS AVOID THEM! THEY MAKE DEBUGGING CONFUSING DOCUMENT THORO
Building on Open Source
Kestral photo by mugley When we plan new engineering projects at Twitter, we measure our requirements against the capabilities of open source offerings, and prefer to use open source whenever it makes sense. By this approach, much of Twitter is now built on open source software. In some cases, our requirements—in particular, the scalability requirements of our service—lead us to develop projects f
artima - Twitter on Scala
Three Twitter developers, Steve Jenson, Alex Payne, and Robey Pointer, talk with Bill Venners about their use of Scala in production at Twitter. Twitter is a fast growing website that provides a micro-blogging service. It began its life as a Ruby on Rails application, and still uses Ruby on Rails to deliver most user-facing web pages. But about a year ago they started replacing some of the back-en
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
English (US)
English (US).pdf)
http://www.beyondlinux.com/files/pub/qconhangzhou2011/Everything%20I%20ever%20learned%20about%20JVM%20performance%20tuning%20@twitter(Attila%20Szegedi).pdf
English (US)
GitHub - twitter/finagle: A fault tolerant, protocol-agnostic RPC system
The Why and How of Scala at Twitter
Rainbird: Realtime Analytics at Twitter (Strata 2011)
GitHub - twitter-archive/snowflake: Snowflake is a network service for generating unique ID numbers at high scale with some simple guarantees.
Prance with the Horses, Skittr with the Mice - David Pollak's Blog