Building and Maintaining an Amazon S3 Metadata Index without Servers | Amazon Web Services
AWS Big Data Blog Building and Maintaining an Amazon S3 Metadata Index without Servers Mike Deck is a Solutions Architect with AWS Amazon S3 is a simple key-based object store whose scalability and low cost make it ideal for storing large datasets. Its design enables S3 to provide excellent performance for storing and retrieving objects based on a known key. Finding objects based on other attribut
大規模データに対するデータサイエンスの進め方 #CWT2016
Cloudera World Tokyo 2016 有賀発表 データサイエンスを含めたチームづくりと、機械学習を活かしたプロダクトの作り方について話しました。Read less
CodeIQについてのお知らせ
2018年4月25日をもちまして、 『CodeIQ』のプログラミング腕試しサービス、年収確約スカウトサービスは、 ITエンジニアのための年収確約スカウトサービス『moffers by CodeIQ』https://moffers.jp/ へ一本化いたしました。 これまで多くのITエンジニアの方に『CodeIQ』をご利用いただきまして、 改めて心より深く御礼申し上げます。 また、エンジニアのためのWebマガジン「CodeIQ MAGAZINE」は、 リクナビNEXTジャーナル( https://next.rikunabi.com/journal/ )に一部の記事の移行を予定しております。 今後は『moffers by CodeIQ』にて、 ITエンジニアの皆様のより良い転職をサポートするために、より一層努めてまいりますので、 引き続きご愛顧のほど何卒よろしくお願い申し上げます。 また、Cod
Distributed Logging Architecture in the Container Era - Treasure Data Blog
Distributed Logging Architecture in the Container Era Last modified: December 6, 2019 TL;DR: Containers and Microservices are great, but they cause big problems with logging. You should do what Docker does: Use Fluentd. Also, if you need scale and stability, we offer Fluentd Enterprise. Microservices and Macroproblems Modern tech enterprise is all about microservices and, increasingly, containers.
AWS at Strata+Hadoop 2016: Building a Scalable Architecture on AWS to Process Streaming Data | Amazon Web Services
AWS Big Data Blog AWS at Strata+Hadoop 2016: Building a Scalable Architecture on AWS to Process Streaming Data Gone are the days when big data was confined to batch processing. To remain competitive, companies must be able to analyze real-time data streams in areas such as video streaming, real-time recommendation engines, preventive maintenance, and fraud detection applications. Last month, Siva
The LMAX Architecture
LMAX is a new retail financial trading platform. As a result it has to process many trades with low latency. The system is built on the JVM platform and centers on a Business Logic Processor that can handle 6 million orders per second on a single thread. The Business Logic Processor runs entirely in-memory using event sourcing. The Business Logic Processor is surrounded by Disruptors - a concurren
(BDT303) Running Spark and Presto on the Netflix Big Data Platform
(BDT303) Running Spark and Presto on the Netflix Big Data Platform In this session, we discuss how Spark and Presto complement the Netflix big data platform stack that started with Hadoop, and the use cases that Spark and Presto address. Also, we discuss how we run Spark and Presto on top of the Amazon EMR infrastructure; specifically, how we use Amazon S3 as our data warehouse and how we leverage
CaffeOnSpark Open Sourced for Distributed Deep... | Hadoop at Yahoo
By Andy Feng(@afeng76), Jun Shi and Mridul Jain (@mridul_jain), Yahoo Big ML Team Introduction Deep learning (DL) is a critical capability required by Yahoo product teams (ex. Flickr, Image Search) to gain intelligence from massive amounts of online data. Many existing DL frameworks require a separated cluster for deep learning, and multiple programs have to be created for a typical machine learni
Kappa Architecture - Where Every Thing Is A Stream
kappa-architecture.com Repository dedicated to Kappa Architecture. I collect and publish articles, tutorials, talks, projects and examples related to Kappa Architecture. What is Kappa Architecture? Kappa Architecture is a software architecture pattern. Rather than using a relational DB like SQL or a key-value store like Cassandra, the canonical data store in a Kappa Architecture system is an appen
Serverless Architectures on AWS
NEWER EDITION AVAILABLE IN MEAP Serverless Architectures on AWS, Second Edition is now available in the Manning Early Access Program. An eBook of this older edition is included at no additional cost when you buy the revised edition! You may still purchase Serverless Architectures on AWS (First Edition) using the Buy options on this page. Serverless Architectures on AWS teaches you how to build, se
It’s All A/Bout Testing
Ever wonder how Netflix serves a great streaming experience with high-quality video and minimal playback interruptions? Thank the team of engineers and data scientists who constantly A/B test their innovations to our adaptive streaming and content delivery network algorithms. What about more obvious changes, such as the complete redesign of our UI layout or our new personalized homepage? Yes, all
Machine Learning, Recommendation Systems, and Data Analysis at Cloud Academy | Amazon Web Services
AWS News Blog Machine Learning, Recommendation Systems, and Data Analysis at Cloud Academy In today’s guest post, Alex Casalboni and Giacomo Marinangeli of Cloud Academy discuss the design and development of their new Inspire system. — Jeff; Our Challenge Mixing technology and content has been our mission at Cloud Academy since the very early days. We are builders and we love technology, but we al
Registration: Machine learning & data conference | O'Reilly Strata
We’ve made the very difficult decision to cancel all future O’Reilly in-person conferences. Instead, we’ll continue to invest in and grow O’Reilly online learning, supporting the 5,000 companies and 2.5 million people who count on our experts to help them stay ahead in all facets of business and technology. Come join them and learn what they already know. Become an O’Reilly online learning member
Kafka Ecosystem at LinkedIn
Apache Kafka is a highly scalable messaging system that plays a critical role as LinkedIn’s central data pipeline. Kafka was developed at LinkedIn back in 2010, and it currently handles more than 1.4 trillion messages per day across over 1400 brokers. Kafka’s strong durability and low latency have enabled us to use Kafka to power a number of newer mission-critical use cases at LinkedIn. These incl
Stream Computing & Analytics at Uber
Slides of QCon London 2016 talk. How stream processing is used within the Uber's Marketplace system to solve a wide range problems, including but not limited to realtime indexing and querying of geospatial time series, aggregation and computing of streaming data, and extracting patterns from data streams. In addition, it will touch upon various TimeSeries analysis and predictions. The underlying s
AWS re:Invent 2015 | (BDT305) Amazon EMR Deep Dive and Best Practices
Amazon EMR is one of the largest Hadoop operators in the world. In this session, we introduce you to Amazon EMR design patterns such as using Amazon S3 instead of HDFS, taking advantage of both long and short-lived clusters, and other Amazon EMR architectural best practices. We talk about how to scale your cluster up or down dynamically and introduce you to ways you can fine-tune your cluster. We
はじめての Elasticsearch
Developers.IO 2016 C-2 「Elasticsearchで自由自在に検索・分析できるようになるためのヒント」
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く
Building a Data Lake on AWS
Design cube in Apache Kylin
[要約] Building a Real-Time Bidding Platform on AWS #AWSAdTechJP![[要約] Building a Real-Time Bidding Platform on AWS #AWSAdTechJP](https://cdn-ak-scissors.b.st-hatena.com/image/square/08b0c6c64f389d2ae0e5fad6e5795f2e5c2c3b75/height=288;version=1;width=512/https%3A%2F%2Fcdn.slidesharecdn.com%2Fss_thumbnails%2F2016rtbwhitepaper-160310091325-thumbnail.jpg%3Fwidth%3D640%26height%3D640%26fit%3Dbounds)
