HDFSの現状と今後 (その1) - Qiita
まず、Apache Hadoop 3.0.0リリースおめでとうございます! 公式の記事: https://blogs.apache.org/foundation/entry/the-apache-software-foundation-announces26 ここ最近の大きめのHDFSの機能について、いくつか語ってみたいと思います 3.0.0でリリースされた機能 Erasure Coding Router based federation (2.9.0にもbackportされた) 3.1.0に入ることが確定した機能 (trunkにマージ済) Tiered storage 3.1.0に入るかわからない機能 (trunkにマージ前) Ozone: Object store for HDFS Erasure Coding JIRA: https://issues.apache.org/jira/
Apache Kylin | Analytical Data Warehouse for Big Data
Apache Kylin™ is an open source, distributed Analytical Data Warehouse for Big Data; it was designed to provide OLAP (Online Analytical Processing) capability in the big data era. By renovating the multi-dimensional cube and precalculation technology on Hadoop and Spark, Kylin is able to achieve near constant query speed regardless of the ever-growing data volume. Reducing query latency from minut
Alluxio - Data Orchestration for the Cloud
wHAT’S NEW ALLUXIO WEBINAR | 10X Faster trino queries on your data platform | tue, june 18 @11:00am pst As Trino users increasingly rely on cloud object storage for retrieving data, speed and cloud cost have become major challenges. The separation of compute and storage creates latency challenges when querying datasets; scanning data between storage and compute tiers becomes I/O bound. On the oth
MapR M7: Providing an enterprise quality Apache HBase API
Provides an overview of M7, which is the first unified data platform for tables and files. Does a deep dive into the MapR architecture, especially containers, and how M7 tables integrates with the rest of MapR architecture, including volumes, management and Hadoop. Describes some of the problems with Apache HBase, and how M7 from MapR solves many of these issues.
GitHub - Yelp/mrjob: Run MapReduce jobs on Hadoop or Amazon Web Services
mrjob: the Python MapReduce library mrjob is a Python 2.7/3.4+ package that helps you write and run Hadoop Streaming jobs. Stable version (v0.7.4) documentation Development version documentation mrjob fully supports Amazon's Elastic MapReduce (EMR) service, which allows you to buy time on a Hadoop cluster on an hourly basis. mrjob has basic support for Google Cloud Dataproc (Dataproc) which allows
GitHub - akamai-tech/trecul
The Trecul dataflow library is a C++ library for data processing that is architecturally similar to a data base query engine. The library can be used as an ordinary command line executable for processing file and database data on a single machine. However it primary use is to run in Hadoop as a map reduce component for parallel execution. We are also developing code for running dataflow programs i
MapR simplifies and extends HBase for Hadoop
Hadoop World Back before the Hadoop Distributed File System (HDFS) came out of stealth mode in May 2011, Hadoop distie MapR Technologies didn't like the way it worked, so it rejigged it to look more like a Unix file system from the outside and beefed up its availability. Now, MapR is taking aim at Hadoop's HBase distributed database layer, which allows customers to run SQL queries against informat
Scalable Machine Learning with Hadoop
This document discusses scalable machine learning using Apache Hadoop and Apache Mahout. It describes what scalable machine learning means in the context of large datasets, provides examples of common machine learning use cases like search and recommendations, and outlines approaches for scaling machine learning algorithms using Hadoop. It also describes the capabilities of the Apache Mahout machi
staltz / HAcid / wiki / Home — Bitbucket
Atlassian Sourcetree is a free Git and Mercurial client for Windows. Atlassian Sourcetree is a free Git and Mercurial client for Mac. You caught us doing some quick maintenance, temporarily restricting your repository to read-only access. HAcid HAcid enables multi-row transactions in HBase. This is a decentralized transaction manager that runs on the client side through a jar library, using specia
管理が困難―分散処理の常識はZooKeeperで変わる
管理が困難―分散処理の常識はZooKeeperで変わる：ビッグデータ処理の常識をJavaで身につける（8）（1/3 ページ） Hadoopをはじめ、Java言語を使って構築されることが多い「ビッグデータ」処理のためのフレームワーク／ライブラリを紹介しながら、大量データを活用するための技術の常識を身に付けていく連載 分散処理の課題が「管理」なのは常識 複数の計算機上で動作（分散）するアプリケーション、ソフトウェアが多く存在します。分散ソフトウェアは複数の計算機で動作することで大量のデータを扱えたり、高負荷な状況に対処します。本稿では、複数の計算機（クラスタ）で動作する各サーバを「インスタンス」と呼びます。 本連載で紹介した分散Key-Valueデータベースである「HBase」は複数の計算機で動作する代表的なソフトウェアです。両ソフトウェアはともに「Apache ZooKeeper」（以下、Z
GitHub - decster/nativetask: Hadoop task level native runtime
Introduction NativeTask is a high performance C++ API & runtime for Hadoop MapReduce. Why it is called NativeTask is that it is a native computing unit only focus on data processing, which is exactly what Task do in the Hadoop MapReduce context. In other word, NativeTask is not responsible for resource management, job Scheduling and fault-tolerance. Those are all managed by original Hadoop compone
Twister: Iterative MapReduce
Twister version 0.9 is now available. MapReduce programming model has simplified the implementations of many data parallel applications. The simplicity of the programming model and the quality of services provided by many implementations of MapReduce attract a lot of enthusiasm among parallel computing communities. From the years of experience in applying MapReduce programming model to various sci
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
Apache Hadoop Distribution | MapR
GitHub - spotify/snakebite: A pure python HDFS client
About Hewlett Packard Enterprise: Information and Strategic Vision
GitHub - intel-hadoop/project-panthera-ase: Analytical SQL Engine (ASE) for Hadoop under "Project Panthera"
GitHub - cloudera/Impala: Real-time Query for Hadoop; mirror of Apache Impala
Tall-and-skinny QR factorizations in MapReduce architectures
H2O.ai | The fastest, most accurate AI Cloud Platform
Google Code Archive - Long-term storage for Google Code Project Hosting.