Fluentd is a JSON-based, open-source log collector originally written at Treasure Data. Fluentd is specifically designed for solving big data collection problem. Many companies choose Hadoop Distributed Filesystem (HDFS) for big data storage. [1] Until recently, however, the only API interface was Java. This changed with the new WebHDFS interface, which allows users to interact with HDFS via HTTP.
About This post shows how to use Fluentd’s MongoDB plugin to aggregate semi-structured logs in real-time. Background Fluentd is an advanced open-source log collector developed at Treasure Data, Inc (see previous post). Because Fluentd handles logs as semi-structured data streams, the ideal database should have strong support for semi-structured data. There are several databases that meet this crit
The story of an engineer. Here is an anecdote. I am sure some of you have had a similar experience. Alex, an engineer, logs all kinds of events. Since he is the primary consumer of the log, the format is optimized for human-readability. One canonical example is Apache logs: 10.0.1.22 - - [15/Oct/2010:11:46:46 -0700] “GET /favicon.ico HTTP/1.1” 404… 10.0.1.22 - - [15/Oct/2010:11:46:58 -0700] “GET /
リリース、障害情報などのサービスのお知らせ
最新の人気エントリーの配信
処理を実行中です
j次のブックマーク
k前のブックマーク
lあとで読む
eコメント一覧を開く
oページを開く