Category: Elasticsearch

Amazon OpenSearch Serviceは、旧Amazon Elasticsearch Service

create domain
– domain name: anything
– deployment type: deployment and test
– instance node: t2.small.search
– number of node: 1
– network: public access
createすると、domainが生成されます。

security policyのところがよくわからんが、ドキュメントが少なく、あまり使われてない印象か。。
さー、sqliteやりますか。

Kibanaで、以下の様にアラートが出ているので、7系から5系にダウングレードする。
This version of Kibana requires Elasticsearch v5.6.16 on all nodes. I found the following incompatible nodes in your cluster: v7.10.1 @ 127.0.0.1:9200 (127.0.0.1)

$ yum remove elasticsearch
$ cd /etc/yum.repos.d/

[elasticsearch-5.x]
name=Elasticsearch repository for 5.x packages
baseurl=https://artifacts.elastic.co/packages/5.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

$ sudo yum install elasticsearch-5.6.16
$ sudo vi /etc/elasticsearch/jvm.options

-Xms256m
-Xmx256m

$ sudo service elasticsearch start
$sudo vi /etc/elasticsearch/elasticsearch.yml

network.host: 0.0.0.0

$ sudo service elasticsearch start
Starting elasticsearch (via systemctl): [ OK ]

$ sudo systemctl status elasticsearch -l
localhost elasticsearch[28744]: OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N

何故だ? さっぱりわからん。。。

[vagrant@localhost ~]$ sudo ls /etc/elasticsearch
elasticsearch.keystore jvm.options role_mapping.yml users
elasticsearch.yml log4j2.properties roles.yml users_roles
[vagrant@localhost ~]$ sudo cat /etc/elasticsearch/elasticsearch.yml
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ———————————- Cluster ———————————–
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ———————————— Node ————————————
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ———————————– Paths ————————————
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ———————————– Memory ———————————–
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ———————————- Network ———————————–
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
#network.host: 192.168.0.1
#
# Set a custom port for HTTP:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# ——————————— Discovery ———————————-
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is [“127.0.0.1”, “[::1]”]
#
#discovery.seed_hosts: [“host1”, “host2”]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: [“node-1”, “node-2”]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ———————————- Gateway ———————————–
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ———————————- Various ———————————–
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true

Set IP where ElasticSearch is installed
#network.host: 192.168.0.1
→ 192.168.35.10

#http.port: 9200
→ http.port: 8000

[vagrant@localhost ~]$ sudo service elasticsearch restart
elasticsearch を停止中: [失敗]
elasticsearch を起動中: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[ OK ]

[vagrant@localhost ~]$ rpm –import https://artifacts.elastic.co/GPG-KEY-elasticsearch
[vagrant@localhost ~]$ cd /etc/yum.repos.d/
[vagrant@localhost yum.repos.d]$ sudo touch elasticsearch.repo
[vagrant@localhost yum.repos.d]$ sudo vi elasticsearch.repo

[elasticsearch-7.x]
name=Elasticsearch repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

[vagrant@localhost yum.repos.d]$ sudo yum install elasticsearch

### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using chkconfig
sudo chkconfig –add elasticsearch
### You can start elasticsearch service by executing
sudo service elasticsearch start
Created elasticsearch keystore in /etc/elasticsearch
Verifying : elasticsearch-7.0.0-1.x86_64 1/1

インストール:
elasticsearch.x86_64 0:7.0.0-1

完了しました!

[vagrant@localhost yum.repos.d]$ sudo chkconfig –add elasticsearch
[vagrant@localhost yum.repos.d]$ sudo service elasticsearch start
elasticsearch を起動中: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000c5330000, 986513408, 0) failed; error=’Not enough space’ (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 986513408 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /var/log/elasticsearch/hs_err_pid5271.log
[失敗]
なにいいいいいいいいいいいいいいいいいいいいいいいいいい
最近メモリが足りないってエラーが多すぎるんだが。。

[vagrant@localhost ~]$ sudo cat /etc/elasticsearch/jvm.options
## JVM configuration

################################################################
## IMPORTANT: JVM heap size
################################################################
##
## You should always set the min and max JVM heap
## size to the same value. For example, to set
## the heap to 4 GB, set:
##
## -Xms4g
## -Xmx4g
##
## See https://www.elastic.co/guide/en/elasticsearch/reference/current/heap-size.html
## for more information
##
################################################################

# Xms represents the initial size of total heap space
# Xmx represents the maximum size of total heap space

-Xms1g
-Xmx1g

################################################################
## Expert settings
################################################################
##
## All settings below this section are considered
## expert settings. Don't tamper with them unless
## you understand what you are doing
##
################################################################

## GC configuration
-XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=75
-XX:+UseCMSInitiatingOccupancyOnly

## G1GC Configuration
# NOTE: G1GC is only supported on JDK version 10 or later.
# To use G1GC uncomment the lines below.
# 10-:-XX:-UseConcMarkSweepGC
# 10-:-XX:-UseCMSInitiatingOccupancyOnly
# 10-:-XX:+UseG1GC
# 10-:-XX:InitiatingHeapOccupancyPercent=75

## DNS cache policy
# cache ttl in seconds for positive DNS lookups noting that this overrides the
# JDK security property networkaddress.cache.ttl; set to -1 to cache forever
-Des.networkaddress.cache.ttl=60
# cache ttl in seconds for negative DNS lookups noting that this overrides the
# JDK security property networkaddress.cache.negative ttl; set to -1 to cache
# forever
-Des.networkaddress.cache.negative.ttl=10

## optimizations

# pre-touch memory pages used by the JVM during initialization
-XX:+AlwaysPreTouch

## basic

# explicitly set the stack size
-Xss1m

# set to headless, just in case
-Djava.awt.headless=true

# ensure UTF-8 encoding by default (e.g. filenames)
-Dfile.encoding=UTF-8

# use our provided JNA always versus the system one
-Djna.nosys=true

# turn off a JDK optimization that throws away stack traces for common
# exceptions because stack traces are important for debugging
-XX:-OmitStackTraceInFastThrow

# flags to configure Netty
-Dio.netty.noUnsafe=true
-Dio.netty.noKeySetOptimization=true
-Dio.netty.recycler.maxCapacityPerThread=0

# log4j 2
-Dlog4j.shutdownHookEnabled=false
-Dlog4j2.disable.jmx=true

-Djava.io.tmpdir=${ES_TMPDIR}

## heap dumps

# generate a heap dump when an allocation from the Java heap fails
# heap dumps are created in the working directory of the JVM
-XX:+HeapDumpOnOutOfMemoryError

# specify an alternative path for heap dumps; ensure the directory exists and
# has sufficient space
-XX:HeapDumpPath=/var/lib/elasticsearch

# specify an alternative path for JVM fatal error logs
-XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log

## JDK 8 GC logging

8:-XX:+PrintGCDetails
8:-XX:+PrintGCDateStamps
8:-XX:+PrintTenuringDistribution
8:-XX:+PrintGCApplicationStoppedTime
8:-Xloggc:/var/log/elasticsearch/gc.log
8:-XX:+UseGCLogFileRotation
8:-XX:NumberOfGCLogFiles=32
8:-XX:GCLogFileSize=64m

# JDK 9+ GC logging
9-:-Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m
# due to internationalization enhancements in JDK 9 Elasticsearch need to set the provider to COMPAT otherwise
# time/date parsing will break in an incompatible way for some date patterns and locals
9-:-Djava.locale.providers=COMPAT

-Xms1g、-Xmx1gの箇所ね。
VMのメモリを変えたいところだ。

デフォルトで2Gってどういうことだよ。。

とりあえず、500mに変える。
[vagrant@localhost ~]$ sudo vi /etc/elasticsearch/jvm.options
[vagrant@localhost ~]$ sudo service elasticsearch start
elasticsearch を起動中: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[ OK ]