sample

{code}
echo -e ‘logstash\nfluentd\nflume’ | bin/logstash -e ‘input { stdin {} } output { stdout {codec => rubydebug}}’
{
“message” => ‘logstash’,
“@version” => “1”,
“@timestamp” => “2015-01-17T16:18:46.175Z”,
“host” => “hope”
}
{
“message” => “fluentd”,
“@version” => “1”,
“@timestamp” => “2015-01-17T16:18:46.175Z”,
“host” => “hope”
}
{
“message” => “flume”,
“@version” => “1”,
“@timestamp” => “2015-01-17T16:18:46.175Z”,
“host” => “hope”
}
{/code}

{code}
input {
stdin{}
}
filter {
mutate {
replace => {message => “%{message} こんにちは!”}
}
}
output {
stdout {
codec => rubydebug
}
}
{/code}

じゃんがらたべたら

あ、ちゃんと書いてあるやんけ
installing-logstash.html
Add the following in your /etc/yum.repos.d/ directory in a file with a .repo suffix, for example logstash.repo

[logstash-5.x]
name=Elastic repository for 5.x packages
baseurl=https://artifacts.elastic.co/packages/5.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

あれ、/etc/yum.repos.d/ って、そもそも何だっけ?

拡張子が.repoですな。centOS, eple, mariadb, mysql-cocommunity, nginxなどありますね。

試しに、jenkins.repoを見てみましょう。

[jenkins]
name=Jenkins
baseurl=http://pkg.jenkins.io/redhat
gpgcheck=1

なるほどー baseurlを指定して、ここからインストールしてるのかな。
yum instrallとRPMについて、少し理解が深まりました^^

set up logstash

Logstashはjava8のインストールが必要らしい

[vagrant@localhost ~]$ java -version
openjdk version “1.8.0_191”
OpenJDK Runtime Environment (build 1.8.0_191-b12)
OpenJDK 64-Bit Server VM (build 25.191-b12, mixed mode)
OK

[vagrant@localhost ~]$ sudo rpm –import https://artifacts.elastic.co/GPG-KEY-elasticsearch
[vagrant@localhost ~]$ sudo yum install logstash
読み込んだプラグイン:fastestmirror
インストール処理の設定をしています
Determining fastest mirrors
* base: ftp.nara.wide.ad.jp
* extras: ftp.nara.wide.ad.jp
* remi-safe: ftp.riken.jp
* updates: ftp.nara.wide.ad.jp
base | 3.7 kB 00:00
extras | 3.4 kB 00:00
jenkins | 2.9 kB 00:00
jenkins/primary_db | 127 kB 00:01
mariadb | 2.9 kB 00:00
mysql-connectors-community | 2.5 kB 00:00
mysql-connectors-community/primary_db | 36 kB 00:00
mysql-tools-community | 2.5 kB 00:00
mysql-tools-community/primary_db | 49 kB 00:00
mysql56-community | 2.5 kB 00:00
mysql56-community/primary_db | 261 kB 00:00
nginx | 2.9 kB 00:00
nginx/primary_db | 49 kB 00:00
nodesource | 2.5 kB 00:00
remi-safe | 3.0 kB 00:00
remi-safe/primary_db | 1.2 MB 00:00
updates | 3.4 kB 00:00
updates/primary_db | 3.7 MB 00:04
パッケージ logstash は利用できません。
エラー: 何もしません

なにいいいいいいいいいいいいいいいいいいい
なぜだああああああああああああああああああああああああああああああああ
もうヤダ。気分転換に日本橋のアンテナショップ行こ。

Logstashを学ぼう

Input
While data is distributed across many systems in different formats, Logstash is a variety of input plug-ins that capture events from different sources simultaneously. You can easily, continuously and smoothly implement data injection from logs, metrics, webapplications, data stores, or various cloud services.

Filter ここは重要か?
Data analysis and transformation
As data is transferred from source to store, Logstash’s filter parses each event and identifies and structures the fields. It further transforms the data into a common format that is best for analysis that delivers business value.

– Extract structures from unstructured data using grok filters
– Get geographical information from IP address
– Anonymize personal information and completely exclude confidential information fields
– Easy processs data from any source, format or schema.
あ、なんかこれは凄そうだ。。

Output
Choose a storage location, and transfer data.
The ideal destination is Elasticsearch, but other destinations are of course also available. The search and analysis possibilities are not impeded.

Logstash allows you to specify various output destinations and transfer data freely. This high degree of flexibility allows you to work with many downstreams.

うん、サンプルをいじりたくなってきました。OKOK!

ELK(Elasticsearch, Kibana, Logstash)

ELKって何?持田香織? それELT😭
ELKはElasticsearch、 Logstash、Kibanaの頭文字で、ELK

あれ、ElasticsearchはConsoleにあるけど、LogstashはConsoleにないぞ。どういうことだ??

あら、LogstashはAWSのサービスではないのね。elasticというサービスの中の一つだ。む、これはなんか、機械学習と近い領域か。。

こちの図が関係性をよく表しています。
https://www.elastic.co/jp/products/logstash

わかったけど、ちょっと待て。Apacheは普通にわかるけど、JDBC、Netflowって何?