データ活用を効率化するHadoop WebUIと権限管理改善事例データを利用するユーザは、増え続けるデータを高速かつ効率的に利用したいと考えています。その一方で、長く利用された仕組みが、そのニーズを満たすにはコストがかかり過ぎる場合があります。 本講演では、ドワンゴのHadoopを用いた分析基盤が、このようなニーズに応えるために、どのように社内ユーザ向けWeb UIとHadoop権限管理を一新したのかを紹介します。Read less
Hourglass: a Library for Incremental Processing on Hadoop
Slides from my talk at IEEE BigData 2013 presenting our paper "Hourglass: a Library for Incremental Processing on Hadoop" Abstract: Hadoop enables processing of large data sets through its relatively easy-to-use semantics. However, jobs are often written inefficiently for tasks that could be computed incrementally due to the burdensome incremental state management for the programmer. This paper in
Cloudera Blog
Riding the wave of the generative AI revolution, third party large language model (LLM) services like ChatGPT and Bard have swiftly emerged as the talk of the town, converting AI skeptics to evangelists and transforming the way we interact with technology. For proof of this megatrend look no further than the instant success of ChatGPT, […] Read blog post
Cloudera Blog
Riding the wave of the generative AI revolution, third party large language model (LLM) services like ChatGPT and Bard have swiftly emerged as the talk of the town, converting AI skeptics to evangelists and transforming the way we interact with technology. For proof of this megatrend look no further than the instant success of ChatGPT, […] Read blog post
Hive, Pig, Scalding, Scoobi, Scrunch and Spark
Comparison of Hadoop Frameworks I had to do simple processing of log files in a Hadoop cluster. Writing Hadoop MapReduce classes in Java is the assembly code of Big Data. There are several high level Hadoop frameworks that make Hadoop programming easier. Here is the list of Hadoop frameworks I tried: Pig Scalding Scoobi Hive Spark Scrunch Cascalog The task was to read log files join with other dat
Automatic optimization on large Hadoop clusters
Chris Olston, Benjamin Reed, Adam Silberstein, and Utkarsh Srivastava at Yahoo Research had a USENIX 2008 paper, "Automatic Optimization of Parallel Dataflow Programs" (PDF), that looks at optimizations for large-scale Hadoop clusters using Pig. The paper says that it is only attempting "to suggest some jumping-off points for [optimization] work." With much of the paper spending a fair amount of t
MacRuby: The Definitive Guide
Thank You! Open Feedback Publishing System (OFPS) is now retired. Thank you to the authors and commenters who participated in the program. OFPS was an O'Reilly experiment that demonstrated the benefits of bridging the gap between private manuscripts and public blogs. Readers gained access to in-progress O'Reilly manuscripts and were able to communicate suggestions with the authors, follow others'
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
GitHub - miguno/avro-hadoop-starter: Example MapReduce jobs in Java, Hive, Pig, and Hadoop Streaming that work on Avro data.
A Birds-Eye View of Pig and Scalding Jobs with hRaven)
404
Cloudera Blog
How Klout is changing the landscape of social media with Hadoop and BI
More Fun with Hadoop In Action Exercises (Pig and Hive)
GitHub - twitter-archive/ambrose: A platform for visualization and real-time monitoring of data workflows
Cloudera | The hybrid data company
Cloudera | The hybrid data company
BigData09 - CSWiki
a little something something...: Haversine formula implemented in Pig
Nuxeo | Open Source ECM
UCI ISG Lecture Series on Scalable Data Management