タグ

twitterとviewfsに関するkimutanskのブックマーク (1)

  • Hadoop filesystem at Twitter

    Twitter runs multiple large Hadoop clusters that are among the biggest in the world. Hadoop is at the core of our data platform and provides vast storage for analytics of user actions on Twitter. In this post, we will highlight our contributions to ViewFs, the client-side Hadoop filesystem view, and its versatile usage here. ViewFs makes the interaction with our HDFS infrastructure as simple as a

    Hadoop filesystem at Twitter
    kimutansk
    kimutansk 2015/10/05
    300PBのデータを万以上のホストに蓄積と。フェデレーションもこの規模だと必要と。ViewFSのTwitterカスタム版で複数のHDFSクラスタをラッピングして扱っているわけですか。
  • 1