Crash Report

What is a crash report?

When an error occurs in the application, it may be terminated, but the terminal stores the information inside the terminated application at that time.

The screen will be displayed at the end, but if you tap the report, the development team can use the information for defect correction.

The crash report contains the following information, but does not contain personally identifying information.
– App version
– Android version
– Device type
– Error occurrence time
– Error occurrence point
– Number of occurrences
– Message you entered

ドコモの「アプリクラッシュポート」についてを見てみましょう
アプリクラッシュレポート

awsの統合ログ収集

awsでログ管理に必要なサービス
– CloudWatch Logs
– CloudTrail
– AWS Config

Splunkでログデータ運用を実現?
https://www.splunk.com
あああああああ、ログまわりだと、必ず機械学習が出てくるなー なんだかんだで。。

LogStorage For AWS

Purpose
The purpose of managing logs in the system is largely 1)monitoring, 2)management, and 3)auditing. 1)In monitoring, whether the system on AWS is operating normally, if the rules for network access are set properly, or if there is a security problem in the system on AWS, these will be continuously logged. It is checked by monitoring in real time. In addition, if there is any problem in the system, logs are also used to investigate the cause. 2) In management, the above monitoring points can be analyzed from the viewpoint of historical data, and trends in system usage and traffic can be analyzed and used for system expansion and change planning. 3)In audits, logs are important in terms of leaving a system operation trail of who did what and when.

AWSのけるログ管理
AWS上でのログ管理は、ログを収集・蓄積するAWSサービスが既に用意されており、そうしたサービスを利用してログを管理・モニタリングしていく
– AWSサービスに対する管理アクセスログ: AWS CloudTrail、 AWS Config → CloudWatch Logs, S3, SQS
– AWS上の通信トラフィックのログ: VPC Flow Logs, ELB Logs → CloudWatch Logs
– EC2インスタンス内のアクセスログ、アプリケーションログ、システムロ:AWS CloudWatch

なるほどー、肝心なのはCloudWatchやなー

ls -lhs

[vagrant@localhost test]$ ls -lhs
合計 20K
4.0K -rw-rw-r– 1 vagrant vagrant 104 4月 26 08:44 2019 index.php
4.0K -rw-rw-r– 1 vagrant vagrant 18 4月 29 11:39 2019 test
12K -rw-r–r– 1 vagrant vagrant 11K 4月 24 08:36 2019 test.swf

-lオプション:ファイルの詳細も表示
-hオプション:単位を読みやすい形式で表示
-sオプション:ファイルサイズ順で表示

なるほどーーーーーーー

kibanaをインストールしよう

[vagrant@localhost ~]$ cd /etc/yum.repos.d
[vagrant@localhost yum.repos.d]$ ls
CentOS-Base.repo logstash.repo remi-php54.repo
CentOS-Debuginfo.repo mariadb.repo remi-php70.repo
CentOS-Media.repo mysql-community-source.repo remi-php71.repo
CentOS-Vault.repo mysql-community.repo remi-php72.repo
CentOS-fasttrack.repo nginx.repo remi-php73.repo
elasticsearch.repo nodesource-el.repo remi-safe.repo
epel-testing.repo remi-glpi91.repo remi.repo
epel.repo remi-glpi92.repo
jenkins.repo remi-glpi93.repo
[vagrant@localhost yum.repos.d]$ sudo touch kibana.repo
[vagrant@localhost yum.repos.d]$ sudo vi kibana.repo
[vagrant@localhost yum.repos.d]$ sudo yum install kibana
インストール:
kibana.x86_64 0:7.0.0-1

完了しました!

ヒアドキュメント

[vagrant@localhost test]$ echo line1
line1
[vagrant@localhost test]$ echo line2
line2
[vagrant@localhost test]$ echo line3
line3
[vagrant@localhost test]$ cat < line1
> line2
> line3
> EOF
line1
line2
line3

ヒアドキュメントでファイルが作れる!?

[vagrant@localhost test]$ cat < test
> line1
> line2
> line3
> EOF
[vagrant@localhost test]$ cat test
line1
line2
line3

すげーーーーーーーー
で、kibanaのリポジトリを作成しようとすると、、
[vagrant@localhost ~]$ sudo cat << _EOF_ > /etc/yum.repos.d/kibana.repo
> [kibana-4.5]
> name=Kibana repository for 4.5.x packages
> baseurl=http://packages.elastic.co/kibana/4.5/centos
> gpgcheck=1
> gpgkey=http://packages.elastic.co/GPG-KEY-elasticsearch
> enabled=1
> _EOF_
-bash: /etc/yum.repos.d/kibana.repo: 許可がありません

なんだよ。。

Elastic searchをインストール

[vagrant@localhost ~]$ rpm –import https://artifacts.elastic.co/GPG-KEY-elasticsearch
[vagrant@localhost ~]$ cd /etc/yum.repos.d/
[vagrant@localhost yum.repos.d]$ sudo touch elasticsearch.repo
[vagrant@localhost yum.repos.d]$ sudo vi elasticsearch.repo

[elasticsearch-7.x]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

[vagrant@localhost yum.repos.d]$ sudo yum install elasticsearch

### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using chkconfig
sudo chkconfig –add elasticsearch
### You can start elasticsearch service by executing
sudo service elasticsearch start
Created elasticsearch keystore in /etc/elasticsearch
Verifying : elasticsearch-7.0.0-1.x86_64 1/1

インストール:
elasticsearch.x86_64 0:7.0.0-1

完了しました!

[vagrant@localhost yum.repos.d]$ sudo chkconfig –add elasticsearch
[vagrant@localhost yum.repos.d]$ sudo service elasticsearch start
elasticsearch を起動中: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000c5330000, 986513408, 0) failed; error=’Not enough space’ (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 986513408 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /var/log/elasticsearch/hs_err_pid5271.log
[失敗]
なにいいいいいいいいいいいいいいいいいいいいいいいいいい
最近メモリが足りないってエラーが多すぎるんだが。。

[vagrant@localhost ~]$ sudo cat /etc/elasticsearch/jvm.options
## JVM configuration

################################################################
## IMPORTANT: JVM heap size
################################################################
##
## You should always set the min and max JVM heap
## size to the same value. For example, to set
## the heap to 4 GB, set:
##
## -Xms4g
## -Xmx4g
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
## for more information
##
################################################################

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

-Xms1g
-Xmx1g

################################################################
## Expert settings
################################################################
##
## All settings below this section are considered
## expert settings. Don't tamper with them unless
## you understand what you are doing
##
################################################################

## GC configuration
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly

## G1GC Configuration
# NOTE: G1GC is only supported on JDK version 10 or later.
# To use G1GC uncomment the lines below.
# 10-:-XX:-UseConcMarkSweepGC
# 10-:-XX:-UseCMSInitiatingOccupancyOnly
# 10-:-XX:+UseG1GC
# 10-:-XX:InitiatingHeapOccupancyPercent=75

## DNS cache policy
# cache ttl in seconds for positive DNS lookups noting that this overrides the
# JDK security property networkaddress.cache.ttl; set to -1 to cache forever
-Des.networkaddress.cache.ttl=60
# cache ttl in seconds for negative DNS lookups noting that this overrides the
# JDK security property networkaddress.cache.negative ttl; set to -1 to cache
# forever
-Des.networkaddress.cache.negative.ttl=10

## optimizations

# pre-touch memory pages used by the JVM during initialization
-XX:+AlwaysPreTouch

## basic

# explicitly set the stack size
-Xss1m

# set to headless, just in case
-Djava.awt.headless=true

# ensure UTF-8 encoding by default (e.g. filenames)
-Dfile.encoding=UTF-8

# use our provided JNA always versus the system one
-Djna.nosys=true

# turn off a JDK optimization that throws away stack traces for common
# exceptions because stack traces are important for debugging
-XX:-OmitStackTraceInFastThrow

# flags to configure Netty
-Dio.netty.noUnsafe=true
-Dio.netty.noKeySetOptimization=true
-Dio.netty.recycler.maxCapacityPerThread=0

# log4j 2
-Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true

-Djava.io.tmpdir=${ES_TMPDIR}

## heap dumps

# generate a heap dump when an allocation from the Java heap fails
# heap dumps are created in the working directory of the JVM
-XX:+HeapDumpOnOutOfMemoryError

# specify an alternative path for heap dumps; ensure the directory exists and
# has sufficient space
-XX:HeapDumpPath=/var/lib/elasticsearch

# specify an alternative path for JVM fatal error logs
-XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log

## JDK 8 GC logging

8:-XX:+PrintGCDetails
8:-XX:+PrintGCDateStamps
8:-XX:+PrintTenuringDistribution
8:-XX:+PrintGCApplicationStoppedTime
8:-Xloggc:/var/log/elasticsearch/gc.log
8:-XX:+UseGCLogFileRotation
8:-XX:NumberOfGCLogFiles=32
8:-XX:GCLogFileSize=64m

# JDK 9+ GC logging
9-:-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m
# due to internationalization enhancements in JDK 9 Elasticsearch need to set the provider to COMPAT otherwise
# time/date parsing will break in an incompatible way for some date patterns and locals
9-:-Djava.locale.providers=COMPAT

-Xms1g、-Xmx1gの箇所ね。
VMのメモリを変えたいところだ。

デフォルトで2Gってどういうことだよ。。

とりあえず、500mに変える。
[vagrant@localhost ~]$ sudo vi /etc/elasticsearch/jvm.options
[vagrant@localhost ~]$ sudo service elasticsearch start
elasticsearch を起動中: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[ OK ]

Elasticsearchとは

Highly scalable full-text search engine developed by Elastic
Enables various analysis such as real time data analysis, log analysis, full text search
Often used with log aggregation Logstash and fluentd, and visualization tools kibana
Searching across multiple databases is provided as a common use

Elasticsearch
– Strong search performance and scalability
– Indexes are often separated by time intervals
– Throw away indexes that are no longer needed
– Store data for analysis and search

RDS:Elasticsearch
DB -> index
table -> mapping type
column -> field
record -> document

Mapping: Setting of field type and analysis method
Analysis: About processing of field values such as language processing and normalization
Query DSL: Assembling search condition in JSON format

sample

{code}
echo -e ‘logstash\nfluentd\nflume’ | bin/logstash -e ‘input { stdin {} } output { stdout {codec => rubydebug}}’
{
“message” => ‘logstash’,
“@version” => “1”,
“@timestamp” => “2015-01-17T16:18:46.175Z”,
“host” => “hope”
}
{
“message” => “fluentd”,
“@version” => “1”,
“@timestamp” => “2015-01-17T16:18:46.175Z”,
“host” => “hope”
}
{
“message” => “flume”,
“@version” => “1”,
“@timestamp” => “2015-01-17T16:18:46.175Z”,
“host” => “hope”
}
{/code}

{code}
input {
stdin{}
}
filter {
mutate {
replace => {message => “%{message} こんにちは!”}
}
}
output {
stdout {
codec => rubydebug
}
}
{/code}

edit /etc/yum.repos.d directory

yum is an integrated management system for packages.

It manages RPM packages and is more convenient and easier to use than using the rpm command. yum manages and integrates RPM information and resolves dependencies automatically. It is in the same standing position as APT in Debian. By using yum, can update package of distribution, search for package, delete package, display package information, etc.

For the repository, a separate file is prepared and describe in “/etc/yum.repos.d”.

[vagrant@localhost ~]$ cd /etc/yum.repos.d
[vagrant@localhost yum.repos.d]$ ls
CentOS-Base.repo mariadb.repo remi-php54.repo
CentOS-Debuginfo.repo mysql-community-source.repo remi-php70.repo
CentOS-Media.repo mysql-community.repo remi-php71.repo
CentOS-Vault.repo nginx.repo remi-php72.repo
CentOS-fasttrack.repo nodesource-el.repo remi-php73.repo
epel-testing.repo remi-glpi91.repo remi-safe.repo
epel.repo remi-glpi92.repo remi.repo
jenkins.repo remi-glpi93.repo
[vagrant@localhost yum.repos.d]$ sudo touch logstash.repo

公式と同じように書きます。
[vagrant@localhost yum.repos.d]$ sudo vi logstash.repo
[vagrant@localhost yum.repos.d]$ cat logstash.repo
[logstash-5.x]
name=Elastic repository for 5.x packages
baseurl=https://artifacts.elastic.co/packages/5.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

[vagrant@localhost ~]$ sudo yum install logstash
読み込んだプラグイン:fastestmirror
インストール処理の設定をしています
Loading mirror speeds from cached hostfile
* base: ftp.nara.wide.ad.jp
* extras: ftp.nara.wide.ad.jp
* remi-safe: ftp.riken.jp
* updates: ftp.nara.wide.ad.jp
https://artifacts.elastic.co/packages/5.x/yum/repodata/repomd.xml: [Errno 14] PYCURL ERROR 6 – “Couldn’t resolve host ‘artifacts.elastic.co'”
他のミラーを試します。
エラー: Cannot retrieve repository metadata (repomd.xml) for repository: logstash-5.x. Please verify its path and try again

なにいいいいいいいいいいいいいいいいいいいいい
artifacts.elastic.coが違うだと。。。

REDHATの公式を見る
>Satellite または Proxy サーバーに完全修飾ドメイン名 (FQDN) が設定されており、Apache が使用する SSL 証明書の CommonName (CN) が FQDN に設定されていることを確認してください。

https://access.redhat.com/ja/solutions/1307833

[vagrant@localhost yum.repos.d]$ grep CN /etc/httpd/conf/ssl.crt/server.crt
grep: /etc/httpd/conf/ssl.crt/server.crt: そのようなファイルやディレクトリはありません
[vagrant@localhost yum.repos.d]$ grep ^SSLCert /etc/httpd/conf.d/ssl.conf
SSLCertificateFile /etc/pki/tls/certs/localhost.crt
SSLCertificateKeyFile /etc/pki/tls/private/localhost.key

何言いいいいいいいいいいいいいいいい、わからん。
あかん、とりあえずelasticsearchに行こう。

じゃんがらたべたら

あ、ちゃんと書いてあるやんけ
installing-logstash.html
Add the following in your /etc/yum.repos.d/ directory in a file with a .repo suffix, for example logstash.repo

[logstash-5.x]
name=Elastic repository for 5.x packages
baseurl=https://artifacts.elastic.co/packages/5.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

あれ、/etc/yum.repos.d/ って、そもそも何だっけ?

拡張子が.repoですな。centOS, eple, mariadb, mysql-cocommunity, nginxなどありますね。

試しに、jenkins.repoを見てみましょう。

[jenkins]
name=Jenkins
baseurl=http://pkg.jenkins.io/redhat
gpgcheck=1

なるほどー baseurlを指定して、ここからインストールしてるのかな。
yum instrallとRPMについて、少し理解が深まりました^^