I have 15 years of consulting & hands-on build experience with clients in the UK, USA, Sweden, Ireland & Germany. Past clients include Bank of America Merrill Lynch, Blackberry, Bloomberg, British Telecom, Ford, Google, ITV, LeoVegas, News UK, Pizza Hut, Royal Mail, T-Mobile, Williams Formula 1, Wise & UBS. I hold both a Canadian and a British passport. My CV, Twitter & LinkedIn. This blog post wi
The document discusses using Amazon EMR to scale analytics workloads on AWS. It provides an overview of EMR and how it allows users to easily run Hadoop clusters on AWS. It discusses how EMR allows tuning clusters and reducing costs by using Spot instances. It also discusses using various AWS services like S3, HDFS and integrating various Hadoop ecosystem tools on EMR. It provides examples of usin
Amazon EMR is one of the largest Hadoop operators in the world. In this session, we introduce you to Amazon EMR design patterns such as using Amazon S3 instead of HDFS, taking advantage of both long and short-lived clusters, and other Amazon EMR architectural best practices. We talk about how to scale your cluster up or down dynamically and introduce you to ways you can fine-tune your cluster. We
AWS Big Data Blog Submitting User Applications with spark-submit Francisco Oliveira is a consultant with AWS Professional Services Customers starting their big data journey often ask for guidelines on how to submit user applications to Spark running on Amazon EMR. For example, customers ask for guidelines on how to size memory and compute resources available to their applications and the best reso
14. Targeting Infrastructure ELB EC2 EC2 EC2 EC2 request EC2 S3 DynamoDB EMR ELB EC2 EC2 EC2 EC2 http api fluentd fluentd (aggregator) out_exec_filter out_dynamodb servlet (scala) dynamic-dynamo EC2 Growth Forecast EC2 EC2 VPC 1 VPC 1 VPC 2 (targeting) VPC Peering VPC Peering ephemeral cluster 15. ELB EC2 EC2 EC2 EC2 request EC2 S3 DynamoDB EMR ELB EC2 EC2 EC2 EC2 http api fluentd fluentd (aggregator)
AWS Big Data Blog Indexing Common Crawl Metadata on Amazon EMR Using Cascading and Elasticsearch Hernan Vivani is a Big Data Support Engineer for Amazon Web Services A previous post showed you how to get started with Elasticsearch and Kibana on Amazon EMR. In that post, we installed Elasticsearch and Kibana on an Amazon EMR cluster using bootstrap actions. This post shows you how to build a simple
Amazon Web Services – Best Practices for Amazon EMR August 2013 Page 1 of 38 Best Practices for Amazon EMR Parviz Deyhim August 2013 (Please consult http://aws.amazon.com/whitepapers/ for the latest version of this paper) Amazon Web Services – Best Practices for Amazon EMR August 2013 Page 2 of 38 Table of Contents Abstract..........................................................................
AWS News Blog Apache HBase on Amazon EMR – Real-time Access to Your Big Data All Your Base AWS has already given you a lot of storage and processing options to choose from, and today we are adding a really important one. You can now use Apache HBase to store and process extremely large amounts of data (think billions of rows and millions of columns per row) on AWS. HBase offers a number of powerfu
In this post, I'm announcing a new tool for launching Hadoop jobs locally and on AWS's elastic-mapreduce, (EMR) which I call Lemur. I built this tool for The Climate Corporation (TCC), purveyors of Weather Insurance to the Agriculture Industry. I want to thank TCC for allowing me to open source this work, as well as a great deal of feedback and the original use cases. Overview Lemur is a tool to
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
処理を実行中です
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く