Marmaray: An Open Source Generic Data Ingestion and Dispersal Framework and Library for Apache Hadoop
Marmaray: An Open Source Generic Data Ingestion and Dispersal Framework and Library for Apache Hadoop Connecting users worldwide on our platform all day, every day requires an enormous amount of data management. When you consider the hundreds of operations and data science teams analyzing large sets of anonymous, aggregated data, using a variety of different tools to better understand and maintain
Apache Kudu - Fast Analytics on Fast Data
Home Overview Documentation Releases Blog Community GET IN TOUCH Mailing Lists Slack Channel Events and Meetups Project Committers Ecosystem How to Contribute DEVELOPER RESOURCES GitHub Gerrit Code Review JIRA Issue Tracker SOCIAL MEDIA Twitter Reddit APACHE SOFTWARE FOUNDATION Security Sponsorship Thanks License FAQ Apache Kudu is an open source distributed data storage engine that makes fast ana
Cloudera Blog
The ongoing progress in Artificial Intelligence is constantly expanding the realms of possibility, revolutionizing industries and societies on a global scale. The release of LLMs surged by 136% in 2023 compared to 2022, and this upward trend is projected to continue in 2024. Today, 44% of organizations are experimenting with generative AI, with 10% having […] Read blog post
Apache Hadoop Goes Realtime at Facebook
Apache Hadoop Goes Realtime at Facebook Dhruba Borthakur Kannan Muthukkaruppan Karthik Ranganathan Samuel Rash Joydeep Sen Sarma Nicolas Spiegelberg Dmytro Molkov Rodrigo Schmidt Facebook {dhruba,jssarma,jgray,kannan, nicolas,hairong,kranganathan,dms, aravind.menon,rash,rodrigo, amitanand.s}@fb.com Jonathan Gray Hairong Kuang Aravind Menon Amitanand Aiyer ABSTRACT Facebook recently deployed Facebo
RCFile，Parquet，ORCFile
この2ヶ月で，Cloudera/Twitter，Hortonworks からそれぞれ別の列指向ファイルフォーマットが公開されました．Parquet と ORCFile です． この記事では，まず RCFile の復習をして，その後 Parquet と ORCFile それぞれの共通点と違いをおおまかに見ていこうと思います．コードレベルの詳細な違いについては，次回以降で見ていきます． RCFile の復習 RCFile は Record Columnar File の略で，Hive から利用できるストレージフォーマットです．特に，HDFS や S3 といった分散ストレージ上でパフォーマンスがでるように設計されています． HDFS/S3 といったストレージでは，基本的にデータを計算機間で同じ負荷になるようにデータを分散配置します．このため，従来の列指向ストレージフォーマットのように適当に列毎に
Near Real-time Processing Over Hadoop and HBase
From MapReduce to realtime This post covers much of the Near-Realtime Processing Over HBase talk I’m giving at ApacheCon NA 2013 in blog form. It also draws from the Hadoop, HBase, and Healthcare talk from StrataConf/Hadoop World 2012. The first significant use of Hadoop at Cerner came in building search indexes for patient charts. While creation of simple search indexes is almost commoditized, we
Hadoop at Yahoo!: More Than Ever Before · YDN Blog
A lot has changed at Yahoo! last year. We have new leaders, we gained millions in new audience*, we saw engagement gains from Social Bar, and we released several successful mobile apps such as Flickr and Yahoo! Mail. But with all that change, there is one thing that has remained constant, and that is our commitment to pioneering new ground for Hadoop. I was well aware of the rich legacy behind Had
Storage infrastructure using HBase behind LINE messages
The document discusses NHN Japan's use of HBase for the LINE messaging platform's storage infrastructure. Some key points: - HBase is used to store tens of billions of message rows per day for LINE, achieving sub-10ms response times and high availability through dual clusters. - The presentation covers their experience migrating HBase clusters between data centers online, handling NameNode failure
Netflix shows off how it does Hadoop in the cloud – Old GigaOm
Netflix (s nflx) is the undeniable king of computing in the cloud — running almost entirely on the Amazon Web Services (s amzn) platform — and its reign expands into big data workloads, too. In a Thursday evening blog post, the company shared the details of its AWS-based Hadoop architecture and a homemade Hadoop Plaform as a Service that it calls Genie. That Netflix is a heavy Hadoop user is hardl
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
Private Site
[HDFS-6740] Make FSDataset support adding data volumes dynamically - ASF JIRA
Cascalog | Data processing on Hadoop without the hassle
[HADOOP-7001] Allow configuration changes without restarting configured nodes - ASF JIRA
[HDFS-4425] NameNode low on available disk space - ASF JIRA
大規模サイトを支えるビッグデータプラットフォーム技術
[HADOOP-8928] Add ability to reset topologies on master nodes - ASF JIRA
[HDFS-1448] Create multi-format parser for edits logs file, support binary and XML formats initially - ASF JIRA
[HDFS-3429] DataNode reads checksums even if client does not need them - ASF JIRA
Hadoop: topology, rack awareness and NameNode caching
Cloudera | ハイブリッドデータカンパニー
