" we evolved ORCFile to provide a significant boost in compression ratios over RCFile on our warehouse data, going from 5x to 8x. Additionally, on a large representative set of queries and data from our warehouse, we found that the Facebook ORCFile writer is 3x better on average than ORCFile. "

yass のブックマーク 2014/04/20 21:56

<blockquote class="hatena-bookmark-comment"><a class="comment-info" href="https://b.hatena.ne.jp/entry/190283730/comment/yass" data-user-id="yass" data-entry-url="https://b.hatena.ne.jp/entry/s/engineering.fb.com/2014/04/10/core-infra/scaling-the-facebook-data-warehouse-to-300-pb/" data-original-href="https://engineering.fb.com/2014/04/10/core-infra/scaling-the-facebook-data-warehouse-to-300-pb/" data-entry-favicon="https://cdn-ak2.favicon.st-hatena.com/64?url=https%3A%2F%2Fengineering.fb.com%2F2014%2F04%2F10%2Fcore-infra%2Fscaling-the-facebook-data-warehouse-to-300-pb%2F" data-user-icon="/users/yass/profile.png">Scaling the Facebook data warehouse to 300 PB</a><ul class="comment-tag" style="list-style: none; margin: 0px;"><li style="float: left">[<a href="https://b.hatena.ne.jp/q/facebook">facebook</a>]</li><li style="float: left">[<a href="https://b.hatena.ne.jp/q/rcfile">rcfile</a>]</li><li style="float: left">[<a href="https://b.hatena.ne.jp/q/orcfile">orcfile</a>]</li><li style="float: left">[<a href="https://b.hatena.ne.jp/q/hive">hive</a>]</li><li style="float: left">[<a href="https://b.hatena.ne.jp/q/%22columnar%20format%22">columnar format</a>]</li><li style="float: left">[<a href="https://b.hatena.ne.jp/q/lz4">lz4</a>]</li></ul><br><p style="clear: left">&quot; we evolved ORCFile to provide a significant boost in compression ratios over RCFile on our warehouse data, going from 5x to 8x. Additionally, on a large representative set of queries and data from our warehouse, we found that the Facebook ORCFile writer is 3x better on average than ORCFile. &quot;</p><a class="datetime" href="https://b.hatena.ne.jp/yass/20140420#bookmark-190283730"><span class="datetime-body">2014/04/20 21:56</span></a></blockquote><script src="https://b.st-hatena.com/js/comment-widget.js" charset="utf-8" async></script>

このブックマークにはスターがありません。
最初のスターをつけてみよう！

Scaling the Facebook data warehouse to 300 PB

engineering.fb.com2014/04/11

At Facebook, we have unique storage scalability challenges when it comes to our data warehouse. Our warehouse stores upwards of 300 PB of Hive data, with an incoming daily rate of about 600 TB. In ...

23 人がブックマーク・3 件のコメント

他のコメントを読む

＼コメントがサクサク読めるアプリです／

はてなブックマーク

Scaling the Facebook data warehouse to 300 PB

はてなブックマーク

公式Twitter

はてなのサービス