サクサク読めて、アプリ限定の機能も多数!
トップへ戻る
Pixel 9
developer.yahoo.com
We had a record turnout for the February 2011 Hadoop User Group at the main Sunnyvale Yahoo! campus with 336 people signed up. Next month and for the rest of the year, we'll be in the larger Yahoo! cafeteria across the street that can hold up to 1000 people. If I remember correctly, the first Hadoop Summit in 2008 was only 400 people. We started with an introduction and discussion of Yahoo's plans
Overview In the Big Data business running fewer larger clusters is cheaper than running more small clusters. Larger clusters also process larger data sets and support more jobs and users. The Apache Hadoop MapReduce framework has hit a scalability limit around 4,000 machines. We are developing the next generation of Apache Hadoop MapReduce that factors the framework into a generic resource schedu
I have a new deadline to beat for this post. Wednesday February 16, 2011 6 pm, that's my new deadline. That’s when we hold the next HUG at the Yahoo! Sunnyvale Campus at the URL’s Café. BTW, in case you are looking for it, here’s the campus in more details: http://www.wikimapia.org/#lat=37.4181633&lon=-122.0250607&z=18&l=0&m=b&search=yahoo Why is that my new deadline? Mostly, because I missed my o
[ANNOUNCEMENT] Yahoo focusing on Apache Hadoop, discontinuing “The Yahoo Distribution of Hadoop” Hi Folks, I'm pleased to announce that after some reflection, Yahoo! has decided to discontinue the "The Yahoo Distribution of Hadoop" and focus on Apache Hadoop. We plan to remove all references to a Yahoo distribution from our website (developer.yahoo.com/hadoop), close our github repo (yahoo.github.
Somewhat to my surprise, I was recently asked why Yahoo has put so much into Apache Hadoop. We currently have nearly 100 people working on Apache Hadoop and related projects, such as Pig, ZooKeeper, Hive, Howl, HBase and Oozie. Over the last 5 years, we've invested nearly 300 person-years into these projects. The Hadoop team at Yahoo is so passionate about our open source mission, and we've been d
Yahoo! Opens to Third-Party Authentication with Facebook/Google IDs Today we're excited to announce that on Thursday, January 20, Yahoo! opens to third-party user authentication with Facebook and Google logins (via OpenID) across the Yahoo! Network. Hundreds of millions of Facebook and Google users will be able to easily sign in and interact on Yahoo! using their Facebook or Google IDs. This elimi
Apache Hadoop has become the de-facto platform for developing large-scale data-intensive applications. It has been used actively in academia and Industry for research and data mining. Our Hadoop Summits provide an opportunity for understanding the latest trends and roadmap for Hadoop Platform and its ecosystem and how Hadoop is leveraged in various domains. Yahoo! India R&D is proud to host the se
Video: Ryan Dahl — Node.js Roadmap Video published 2010-11-30.
NodeJS has been garnering a lot of attention late. Briefly, NodeJS is a server-side JavaScript runtime implemented using Google's highly performant V8 engine. It provides an (almost) completely non-blocking I/O stack which, when combined with JavaScript closures and anonymous functions, makes it an excellent platform for implementing high throughput web services. As an example, a simple "Hello, wo
Thanks to Tsz-Wo Nicholas Sze and Mahadev Konar for this article. The Problem of Many Small Files The Hadoop Distributed File System (HDFS) is designed to store and process large (terabytes) data sets. At Yahoo!, for example, a large production cluster may have 14 PB disk spaces and store 60 millions of files. However, storing a large number of small files in HDFS is inefficient. We call a file sm
The bottom line is that we achieved the target in Petabytes and got close to the target in the number of files. But this is done with a smaller number of nodes and the need to support a workload close to 100,000 clients has not yet materialized. The question now is whether the goals are feasible with the current system architecture. Namespace Limitations HDFS is based on an architecture where the
Video: Dion Almaer, Ben Galbraith, Elaine Wherry, Thomas Sha, Joe Hewitt, Ryan Dahl, Tantek Çelik, and Douglas Crockford — YUIConf 2010 Panel Discussion: The Future of Frontend Engineering Video published 2010-11-08. Download HD video (480p ~608MB) Download iPhone/iPod-compatible video file (320x240, .m4v) Transcript: Transcipt in progress. CC photos used in the pre-roll slide: Dion Almaer: http:/
Introduction Welcome to the Yahoo! Hadoop tutorial! This series of tutorial documents will walk you through many aspects of the Apache Hadoop system. You will be shown how to set up simple and advanced cluster configurations, use the distributed file system, and develop complex Hadoop MapReduce applications. Other related systems are also reviewed. Goals for this Module: Understand the scope of pr
I hope you've all been following along with all the great stuff that's been happening at YUIConf this year. As part of the conference proceedings, I'm excited to announce a preview release of my upcoming book for Yahoo! Press in association with O'Reilly Media. Up and Running With Node.js is a guide to Node.js aimed at giving you the combination of the background knowledge you need to use Node we
Introduction MapReduce is a programming model designed for processing large volumes of data in parallel by dividing the work into a set of independent tasks. MapReduce programs are written in a particular style influenced by functional programming constructs, specifically idioms for processing lists of data. This module explains the nature of this programming model and how it can be used to write
At Yahoo!, we recently implemented a stronger notion of security for the Hadoop platform, based on Kerberos as underlying authentication system. We also successfully enabled this feature within Yahoo! on our internal data processing clusters. I am sure many Hadoop developers and enterprise users are looking forward to get hands-on experience with this enterprise-class Hadoop Security feature. In t
Yahoo! has begun evaluating Hive for use as part of its Hadoop stack. Since, in many peoples' minds, Hive and Pig are roughly equivalent and Pig Latin is very close to SQL, this has led to some confusion. Why are we interested in using both technologies? As we have looked at our workloads and analyzed our use cases, we have come to the conclusion that the different use cases require different tool
Managing Big Data: Architectural Approaches for making batch data available online This is the beginning of an ongoing series of blog posts on “Managing Big Data”. This series will focus on techniques that Yahoo uses to process large volumes of data, ranging from initial collection of data to the end usage of that data. Introduction Over the last several years there are two important trends that r
Well-designed web sites and applications always keep in mind the people who have JavaScript disabled. The whole concept of progressive enhancement is built around the idea that the content of the page should be accessible without JavaScript (or CSS), even if more advanced functionality is available. The “JavaScript-disabled” experience is a part of every discussion — but how often do people actua
Apache Hadoop is a software framework to build large-scale, shared storage and computing infrastructures. Hadoop clusters are used for a variety of research and development projects, and for a growing number of production processes at Yahoo!, EBay, Facebook, LinkedIn, Twitter, and other companies in the industry. It is a key component in several business critical endeavors representing a very sign
Part 1: Understanding event loops and writing great code for Node.js As usual JSConf.eu was one of the best conferences I've been to this year. It's always a delight to hang out with some of the smartest JavaScript coders on the planet, and doing it in the very trendy city of Berlin was a nice bonus. There are a few great recaps of JSConf so I'm not going to focus on those; instead I'm going to ex
Yahoo! PlaceFinder is a geocoding Web service that helps developers make their applications location-aware by converting street addresses or place names into geographic coordinates (and vice versa). PlaceFinder recognizes a large number of place formats and returns rich geographic data about each result, including geographic coordinates, address components, and WOEID. The WOEIDs returned by the se
Big Data and the Power of Hadoop [ video ] Blake Irving, Executive Vice President and Chief Products Officer, Yahoo!
Rate Limiting for Yahoo! Search Web Services This document describes how rate limits apply to our Web Services. The limits for each service are independent and are described in the per-service documentation. Understanding Rate Limiting IP Based Limits Our service rate limits are imposed as a limit on the number of API calls made per IP address during a specific time window. If your IP address chan
A person participating in a social structure expects to develop a reputation and hopes for insight into the reputations of others, but each designed model of participation and reputation embodies its own set of biases and incentive structures. Balancing these forces determines in large measure the success or failure of a social system.
Yahoo! Updates Firehose API Thank you for using the Yahoo! Updates Firehose API. The product has been shut down as of April 16, 2013. If you have any concerns or questions, please don't hesitate to contact us in General Discussion at YDN forum.
New Sign In With Yahoo An easy and convenient way for your users to create an account and sign into your web or mobile app View guide
Spatial entities provided by Yahoo! GeoPlanet are referenced by a 32-bit identifier: the Where On Earth ID (WOEID). WOEIDs are unique and non-repetitive, and are assigned to all entities within the system. A WOEID, once assigned, is never changed or recycled. If a WOEID is deprecated it is mapped to its successor or parent WOEID, so that requests to the service using a deprecated WOEID are served
Video: John Resig — Testing, Performance Analysis, and jQuery 1.4 Video published 2009-12-14. Download iPhone/iPod-compatible video file (320x240, .m4v) Slides Transcript: John Resig: I really appreciate you having me here. I wanted to talk about a few different topics today, all things I'm thinking about at the moment and working on. I wanted to talk a little bit about JavaScript testing, I wante
次のページ
このページを最初にブックマークしてみませんか?
『Yahoo! Developer Network Home』の新着エントリーを見る
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く