ログの収集、集約やデータの収集、集約、プラグインを利用した集計などログやデータの収集にまつわるものならなんでも出来るツール。
サービス用のサーバー群
– ログ収集の設定がされたFluentd
ログ集約サーバー
ーログ集約用の設定がされたFluentd
公式
https://www.fluentd.org/
Syslog, apache/nginx logs, mobile/web app logs, sensor/iot
Fluentd is an open source data collector for unified logging layer.
Fluentd allows you to unify data collection and consumption for a better use and understanding of data.
Fluentd is an open source data collector, which lets you unify the data collection and consumption for a better use and understanding of data.
Unified Logging with JSON
Pluggable Architecture
Minimum Resources Required
Built-in Reliability
あと重要そうなところでいくと、list of Data Sources
application logs: java, ruby, python, php, perl, node.js, scala, .net
network protocols: syslog, http, tcp, udp, unix domain soket, snmp, scribe protocol, mqtt
iot devices: rasperryPi
others: docker, twitter, mysql slow query log, postgresql slow query log, gluster fs, kafka, kestrel, munin, dstat, windows event log, netflow, postfix, aws cloud watch, aws elb log, aws sqs
List of data outputs
log management: elasticsearch + kibana, splunk
dig data: hadoop DFS, treasure data, mongoDB
data archiving: file, aws s3
pub sub: aws kinesis, kafka, amqp, rabbitMQ
data warehouse: bigquery, mysql, postgresql, sqlserver, vertica, aws redshift
monitoring system: datadog, librato, ganglia, graphite, growthforecast, stasd, zabbix, macherel, riemann, sentry
Middleware: norikra, couchdb, couchbase, riak, hbase, opentsdb, zeromq
notification system: email, slack, hipchat, irc, twillo, aws sns
なるほど、give & takeか。
