Spark on fire | DBMS 2 : DataBase Management System Services
Spark is on the rise, to an even greater degree than I thought last month. Numerous clients and other companies I talk with have adopted Spark, plan to adopt Spark, or at least think it’s likely they will. In particular: A number of analytic-stack companies are joining ClearStory in using Spark. Most of the specifics are confidential, but I hope some will be announced soon. MapR has joined Clouder
Apache Mahout, Hadoop’s original machine learning project, is moving on from MapReduce – Old GigaOm
Apache Mahout, a machine learning library for Hadoop since 2009, is joining the exodus away from MapReduce. The project’s community has decided to rework Mahout to support the increasingly popular Apache Spark in-memory data-processing framework, as well as the H2O engine for running machine learning and mathematical workloads at scale. While data processing in Hadoop has traditionally been done u
SparkR by amplab-extras
R frontend for Spark Project maintained by amplab-extras Hosted on GitHub Pages — Theme by mattgraham R on Spark SparkR is an R package that provides a light-weight frontend to use Apache Spark from R. SparkR exposes the Spark API through the RDD class and allows users to interactively run jobs from the R shell on a cluster. NOTE: As of April 2015, SparkR has been officially merged into Apache Spa
NECがビッグデータの機械学習を高速化する技術を開発、インメモリー処理やMPIを導入
NECは、ビッグデータ分析を高速化する分散処理フレームワーク「Feliss」を開発した（発表資料）。ビッグデータ分析でよく用いられるHadoopは、Map-Reduce型の単純な分析であれば高速に実行できるが、繰り返し演算を多用する機械学習処理では、ジョブ間でストレージを経由してデータをやり取りするHDFSがボトルネックとなり、演算の効率を上げにくい。 そこでNECのFelissでは、ジョブ間のデータのやり取りをインメモリーで実施するようにした。さらに演算ノード間の通信などにおいて、並列処理の際のメッセージパッシングのAPIとして一般的な「MPI」を同時に使えるようにした。これにより、機械学習のような複雑な演算について、通常のHadoopを用いる場合と比べて10倍ほど高速に実行できるようにした。FelissはHDFSのインタフェースを備えており、最初のデータ読み出しはHDFSから行える。
TechCrunch | Startup and Technology News
Live Nation says its Ticketmaster subsidiary was hacked. A hacker claims to be selling 560 million customer records. An autonomous pod. A solid-state battery-powered sports car. An electric pickup truck. A convertible grand tourer EV with up to 600 miles of range. A “fully connected mobility device” for young urban innovators to be built by Foxconn and priced under $30,000. The next Popemobile. Ov
Spark Programming Guide
NOTE: This guide is for Spark 0.5.x. Find the latest docs on the Spark website Introduction At a high level, every Spark application consists of a driver program that runs the user's main function and executes various parallel operations on a cluster. The main abstraction Spark provides is a distributed dataset, which is a collection of elements partitioned across the nodes of the cluster that can
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
Spark SQL: Manipulating Structured Data Using Apache Spark
AMP Camp 3 Big Data Bootcamp | UC Berkeley AMP Camp
https://people.eecs.berkeley.edu/~matei/papers/2013/sigmod_shark.pdf
Apache Spark™ - Unified Engine for large-scale data analytics
