Hadoopのトラブルシューティングに関する資料があったのでめもっとく - wyukawa's diary
Hadoop World 2011でClouderaの人が発表した資料を見つけたのではっておく。 Hadoop Troubleshooting 101 - Kate Ting - Cloudera View more presentations from Cloudera, Inc. Clouderaのサポートチームの極意が詰め込まれているようだ。 内容的にはHadoop徹底入門の10章の「性能向上のためのチューニング」と若干かぶっているが参考になります。 io.sort.mb < mapred.child.java.opts とすることとか（ていうかmapred.child.java.optsを増やすことはあるかもしれないがio.sort.mbっていじるもんなのかな）、プロセス数やファイルディスクリプタいじれとか、map出力のスレッドいじれとか、Jetty 6.1.26は使うなとか、盛り
AmazonS3 - HADOOP2 - Apache Software Foundation
S3 Support in Apache Hadoop Apache Hadoop ships with a connector to S3 called "S3A", with the url prefix "s3a:"; its previous connectors "s3", and "s3n" are deprecated and/or deleted from recent Hadoop versions. Consult the Latest Hadoop documentation for the specifics on using any the S3A connector. For Hadoop 2.x releases, the latest troubleshooting documentation. For Hadoop 3.x releases, the la
riccomini - hadoop pig documentation
blackberry, iphone, android sentiment analysis, string matching social networking, google app engine processing hadoop, aster data It is sometimes difficult for SQL users to learn Pig because their mind is used to working in SQL. In this tutorial, examples of various SQL statements are shown, and then translated into Pig statements. For more detailed documentation, please see the official Pig
The Hadoop Distributed File System: Architecture and Design by Dhruba Borthakur Table of contents 1 Introduction .......................................................................................................................3 2 Assumptions and Go
The Hadoop Distributed File System: Architecture and Design by Dhruba Borthakur Table of contents 1 Introduction .......................................................................................................................3 2 Assumptions and Goals .....................................................................................................3 2.1 Hardware Failure .................
NTTデータのHadoop報告書がすごかった - 科学と非科学の迷宮
業界トップ のエンタープライズ Hadoop 企業 Cloudera に入社しました http://www.cloudera.co.jp/ 今年の6月に、「平成２１年度 産学連携ソフトウェア工学実践事業報告書」というドキュメント群が経産省から公表されました。 そのうちの一つに、NTTデータに委託されたHadoopに関する実証実験の報告書がありましたので、今更ながら読んでみることにしました。 Hadoop界隈の人はもうみんなとっくに読んでるのかもしれませんけど。 http://www.meti.go.jp/policy/mono_info_service/joho/downloadfiles/2010software_research/clou_dist_software.pdf 「高信頼クラウド実現用ソフトウェア開発（分散制御処理技術等に係るデータセンター高信頼化に向けた実証事業）」という
Cloudera Blog
It’s hard to believe it’s been 15 years since the global financial crisis of 2007/2008. While this might be a blast from the past we’d rather leave in the proverbial rear-view mirror, in March of 2023 we were back to the future with the collapse of Silicon Valley Bank (SVB), the largest US bank to […] Read blog post
MapReduce Online | EECS at UC Berkeley
Tyson Condie, Neil Conway, Peter Alvaro, Joseph M. Hellerstein, Khaled Elmeleegy and Russell Sears EECS Department University of California, Berkeley Technical Report No. UCB/EECS-2009-136 October 9, 2009 http://www2.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-136.pdf MapReduce is a popular framework for data-intensive distributed computing of batch jobs. To simplify fault tolerance, the output
Cloudera Blog
It’s hard to believe it’s been 15 years since the global financial crisis of 2007/2008. While this might be a blast from the past we’d rather leave in the proverbial rear-view mirror, in March of 2023 we were back to the future with the collapse of Silicon Valley Bank (SVB), the largest US bank to […] Read blog post
Mass-scale computing: Why Hadoop is hot but Java is not
With the massive amount of data proliferating the Web, companies such as Google and many others are building new technologies to sort it all. Core to that movement is something called MapReduce, a software technique that breaks down huge amounts of data into smaller bits. Operating on the smaller bits, and then piecing results together to form the big picture again has proven extremely successful.
Welcome to Apache Pig!
Apache Pig is a platform for analyzing large data sets that consists of a high-level language for expressing data analysis programs, coupled with infrastructure for evaluating these programs. The salient property of Pig programs is that their structure is amenable to substantial parallelization, which in turns enables them to handle very large data sets. At the present time, Pig's infrastructure l
1
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
HDFS.pdf
InfoQ: MapR社がHadoopをベースとした商用ディストリビューションをリリース
Hadoop輪読会第6章
HDFS Architecture Guide
tomo🐧@learning on Twitter: "アザース RT @yutuki_r: HDFSの基礎。 The Hadoop Distributed File System:Architecture and Design http://tinyurl.com/3a7y8r3 #hadoopreading"
https://www.hadoopstudio.org/