vi elasticsearch.yml

[vagrant@localhost ~]$ sudo ls /etc/elasticsearch
elasticsearch.keystore jvm.options role_mapping.yml users
elasticsearch.yml log4j2.properties roles.yml users_roles
[vagrant@localhost ~]$ sudo cat /etc/elasticsearch/elasticsearch.yml
# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
# Before you set out to tweak and tune the configuration, make sure you
# understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please consult the documentation for further information on configuration options:
# https://www.elastic.co/guide/en/elasticsearch/reference/index.html
#
# ———————————- Cluster ———————————–
#
# Use a descriptive name for your cluster:
#
#cluster.name: my-application
#
# ———————————— Node ————————————
#
# Use a descriptive name for the node:
#
#node.name: node-1
#
# Add custom attributes to the node:
#
#node.attr.rack: r1
#
# ———————————– Paths ————————————
#
# Path to directory where to store the data (separate multiple locations by comma):
#
path.data: /var/lib/elasticsearch
#
# Path to log files:
#
path.logs: /var/log/elasticsearch
#
# ———————————– Memory ———————————–
#
# Lock the memory on startup:
#
#bootstrap.memory_lock: true
#
# Make sure that the heap size is set to about half the memory available
# on the system and that the owner of the process is allowed to use this
# limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ———————————- Network ———————————–
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
#network.host: 192.168.0.1
#
# Set a custom port for HTTP:
#
#http.port: 9200
#
# For more information, consult the network module documentation.
#
# ——————————— Discovery ———————————-
#
# Pass an initial list of hosts to perform discovery when this node is started:
# The default list of hosts is [“127.0.0.1”, “[::1]”]
#
#discovery.seed_hosts: [“host1”, “host2”]
#
# Bootstrap the cluster using an initial set of master-eligible nodes:
#
#cluster.initial_master_nodes: [“node-1”, “node-2”]
#
# For more information, consult the discovery and cluster formation module documentation.
#
# ———————————- Gateway ———————————–
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
#gateway.recover_after_nodes: 3
#
# For more information, consult the gateway module documentation.
#
# ———————————- Various ———————————–
#
# Require explicit names when deleting indices:
#
#action.destructive_requires_name: true

Set IP where ElasticSearch is installed
#network.host: 192.168.0.1
→ 192.168.35.10

#http.port: 9200
→ http.port: 8000

[vagrant@localhost ~]$ sudo service elasticsearch restart
elasticsearch を停止中: [失敗]
elasticsearch を起動中: OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
[ OK ]

install selenium

[vagrant@localhost ~]$ python3
Python 3.5.2 (default, Jul 28 2018, 11:25:01)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-23)] on linux
Type “help”, “copyright”, “credits” or “license” for more information.
>>>

[vagrant@localhost ~]$ pip install selenium
Requirement already satisfied: selenium in ./.pyenv/versions/3.5.2/lib/python3.5/site-packages (3.13.0)
You are using pip version 18.0, however version 19.1.1 is available.
You should consider upgrading via the ‘pip install –upgrade pip’ command.

from selenium import webdriver
from selenium.webdriver.chrome.options import options

options = Options()

options.add_argument('--headless')
options.add_argument('--no-sandbox')

driver = webdriver.Chrome(executable_path='/path/to/chromedriver',chrome_options=options)

driver.get('https://yahoo.co.jp')
print(driver.title)

Canvas

<!DOCTYPE html>
<html>
	<head>
		<title>canvas tutorial</title>
		<style>
			#canvas {
				background: #666;
			}
		</style>
	</head>
	<body>
		<canvas id="canvas" width="640" height="480"></canvas>
		<script>
			var canvas = document.getElementById("canvas");
		</script>
	</body>
</html>

<!DOCTYPE html>
<html>
	<head>
		<title>canvas tutorial</title>
		<style>
			#canvas {
				background: #666;
			}
		</style>
	</head>
	<body>
		<canvas id="canvas" width="640" height="480"></canvas>
		<script>
			var canvas = document.getElementById("canvas");
			function onDown(e){
				console.log("down");
			}
			function onUp(e){
				console.log("up");
			}
			function onClick(e){
				console.log("click");
			}
			function onOver(e){
				console.log("mouseover");
			}
			function onOut(){
				console.log("mouseout");
			}

			canvas.addEventListener('mousedown', onDown, false);
			canvas.addEventListener('mouseup', onUp, false);
			canvas.addEventListener('click', onClick, false);
			canvas.addEventListener('mouseover', onOver, false);
			canvas.addEventListener('mouseout', onOut, false);
		</script>
	</body>
</html>

Install ChromeDriver and Fonts

Prepare for install
yum -y install libX11 GConf2 fontconfig
[root@localhost test]# wget https://chromedriver.storage.googleapis.com/73.0.3683.68/chromedriver_linux64.zip
–2019-05-24 09:20:33– https://chromedriver.storage.googleapis.com/73.0.3683.68/chromedriver_linux64.zip
chromedriver.storage.googleapis.com をDNSに問いあわせています… 216.58.197.208, 2404:6800:4004:80e::2010
chromedriver.storage.googleapis.com|216.58.197.208|:443 に接続しています… 接続しました。
HTTP による接続要求を送信しました、応答を待っています… 200 OK
長さ: 5008482 (4.8M) [application/zip]
`chromedriver_linux64.zip’ に保存中

100%[======================================>] 5,008,482 1.67M/s 時間 2.9s

2019-05-24 09:20:36 (1.67 MB/s) – `chromedriver_linux64.zip’ へ保存完了 [5008482/5008482]

[root@localhost test]# ls
chromeTest.js chromedriver_linux64.zip index.php node_modules selenium-test
[root@localhost test]# unzip chromedriver_linux64.zip
Archive: chromedriver_linux64.zip
inflating: chromedriver
[root@localhost test]# mv chromedriver /usr/local/bin/
[root@localhost test]# chmod 755 /usr/local/bin/chromedriver

要求: libatspi.so.0()(64bit)

[root@localhost test]# sudo yum whatprovides libatspi.so.0
読み込んだプラグイン:fastestmirror
Repository nodesource is listed more than once in the configuration
Repository nodesource-source is listed more than once in the configuration
Loading mirror speeds from cached hostfile
* base: ftp.nara.wide.ad.jp
* extras: ftp.nara.wide.ad.jp
* remi-safe: ftp.riken.jp
* updates: ftp.nara.wide.ad.jp
警告: yum のバージョン 3.0.x は、誤ってファイル名に対して一致するでしょう。
その振る舞いをするには「*/libatspi.so.0」もしくは「*bin/libatspi.so.0」が使用できます。
No Matches found

SyntaxError: Use of const in strict mode.

JavaScript has introduced const and let in ECMAScript 2015, which is relatively recent. This is enabled by default in Node.js since Node.js 4.x.

$ git clone git://github.com/creationix/nvm.git ~/.nvm
$ source ~/.nvm/nvm.sh
$ nvm help
$ nvm ls-remote
[root@localhost vagrant]# nvm install v8.16.0
Downloading and installing node v8.16.0…
Downloading https://nodejs.org/dist/v8.16.0/node-v8.16.0-linux-x64.tar.xz…
######################################################################## 100.0%
Computing checksum with sha256sum
Checksums matched!
Now using node v8.16.0 (npm v6.4.1)
[root@localhost vagrant]# node -v
v8.16.0

[root@localhost test]# node chromeTest.js
/home/vagrant/local/app/test/chromeTest.js:3
var By = webdriver.By;
^

ReferenceError: webdriver is not defined
at Object. (/home/vagrant/local/app/test/chromeTest.js:3:10)
at Module._compile (module.js:653:30)
at Object.Module._extensions..js (module.js:664:10)
at Module.load (module.js:566:32)
at tryModuleLoad (module.js:506:12)
at Function.Module._load (module.js:498:3)
at Function.Module.runMain (module.js:694:10)
at startup (bootstrap_node.js:204:16)
at bootstrap_node.js:625:3

What…

install selenium

$ npm -g install selenium-webdriver
$ npm install selenium-webdriver

https://sites.google.com/a/chromium.org/chromedriver/downloads

sudo yum install -y google-chrome-unstable libOSMesa google-noto-cjk-fonts

var webdrive = require('selenium-webdriver');
var driver;
var By = webdriver.By;

driver = new webdriver.Builder()
.withCapabilities(webdriver.Capabilities.chrome())
.build();

driver.get('https://www.google.co.jp/').then(function(){
	driver.findElement(By.id('hplog')).click()
	.then(function(){
		console.log('clicked logo');
	});
});

[vagrant@localhost test]$ node chromeTest.js

/home/vagrant/local/app/test/node_modules/selenium-webdriver/index.js:25
const _http = require(‘./http’);
^^^^^
SyntaxError: Use of const in strict mode.
at Module._compile (module.js:439:25)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.require (module.js:364:17)
at require (module.js:380:17)
at Object. (/home/vagrant/local/app/test/chromeTest.js:1:78)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)

Error: EACCES

[vagrant@localhost test]$ sudo chmod 777 /usr/local/lib;

[vagrant@localhost test]$ sudo npm install -g bower
npm http GET https://registry.npmjs.org/bower
npm http 200 https://registry.npmjs.org/bower
npm WARN deprecated bower@1.8.8: We don’t recommend using Bower for new projects. Please consider Yarn and Webpack or Parcel. You can read how to migrate legacy project here: https://bower.io/blog/2017/how-to-migrate-away-from-bower/
npm http GET https://registry.npmjs.org/bower/-/bower-1.8.8.tgz
npm http 200 https://registry.npmjs.org/bower/-/bower-1.8.8.tgz
/usr/bin/bower -> /usr/lib/node_modules/bower/bin/bower
npm WARN unmet dependency /usr/lib/node_modules/block-stream requires inherits@’~2.0.0′ but will load
npm WARN unmet dependency undefined,
npm WARN unmet dependency which is version undefined
npm WARN unmet dependency /usr/lib/node_modules/fstream requires inherits@’~2.0.0′ but will load
npm WARN unmet dependency undefined,
npm WARN unmet dependency which is version undefined
npm WARN unmet dependency /usr/lib/node_modules/fstream-ignore requires inherits@’2′ but will load
npm WARN unmet dependency undefined,
npm WARN unmet dependency which is version undefined
npm WARN unmet dependency /usr/lib/node_modules/fstream-npm requires inherits@’2′ but will load
npm WARN unmet dependency undefined,
npm WARN unmet dependency which is version undefined
npm WARN unmet dependency /usr/lib/node_modules/glob requires inherits@’2′ but will load
npm WARN unmet dependency undefined,
npm WARN unmet dependency which is version undefined
npm WARN unmet dependency /usr/lib/node_modules/npmconf requires inherits@’~2.0.0′ but will load
npm WARN unmet dependency undefined,
npm WARN unmet dependency which is version undefined
npm WARN unmet dependency /usr/lib/node_modules/tar requires inherits@’2′ but will load
npm WARN unmet dependency undefined,
npm WARN unmet dependency which is version undefined
bower@1.8.8 /usr/lib/node_modules/bower

Node.jp update

[vagrant@localhost test]$ node –version
v0.10.48
[vagrant@localhost test]$ npm –version
1.3.6
[vagrant@localhost test]$ n –latest
-bash: n: コマンドが見つかりません

There is a handy version control tool, n, which lets you check and update versions.

[vagrant@localhost test]$ npm install -g n
npm http GET https://registry.npmjs.org/n
npm http GET https://registry.npmjs.org/n
npm http GET https://registry.npmjs.org/n
npm ERR! Error: CERT_UNTRUSTED
npm ERR! at SecurePair. (tls.js:1430:32)
npm ERR! at SecurePair.emit (events.js:92:17)
npm ERR! at SecurePair.maybeInitFinished (tls.js:1029:10)
npm ERR! at CleartextStream.read [as _read] (tls.js:521:13)
npm ERR! at CleartextStream.Readable.read (_stream_readable.js:341:10)
npm ERR! at EncryptedStream.write [as _write] (tls.js:418:25)
npm ERR! at doWrite (_stream_writable.js:226:10)
npm ERR! at writeOrBuffer (_stream_writable.js:216:5)
npm ERR! at EncryptedStream.Writable.write (_stream_writable.js:183:11)
npm ERR! at write (_stream_readable.js:602:24)
npm ERR! If you need help, you may report this log at:
npm ERR!
npm ERR! or email it to:
npm ERR!

npm ERR! System Linux 2.6.32-754.3.5.el6.x86_64
npm ERR! command “node” “/usr/bin/npm” “install” “-g” “n”
npm ERR! cwd /home/vagrant/local/app/test
npm ERR! node -v v0.10.48
npm ERR! npm -v 1.3.6
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! /home/vagrant/local/app/test/npm-debug.log
npm ERR! not ok code 0

The reason is that SSL key validation is performed on the https registry, so once this setting is set to false, npm install will succeed.

vagrant@localhost test]$ npm config set strict-ssl false

[vagrant@localhost test]$ npm install -g n
npm http GET https://registry.npmjs.org/n
npm http 200 https://registry.npmjs.org/n
npm http GET https://registry.npmjs.org/n/-/n-4.1.0.tgz
npm http 200 https://registry.npmjs.org/n/-/n-4.1.0.tgz
npm ERR! Error: EACCES, mkdir ‘/usr/lib/node_modules/n’
npm ERR! { [Error: EACCES, mkdir ‘/usr/lib/node_modules/n’]
npm ERR! errno: 3,
npm ERR! code: ‘EACCES’,
npm ERR! path: ‘/usr/lib/node_modules/n’,
npm ERR! fstream_type: ‘Directory’,
npm ERR! fstream_path: ‘/usr/lib/node_modules/n’,
npm ERR! fstream_class: ‘DirWriter’,
npm ERR! fstream_stack:
npm ERR! [ ‘/usr/lib/node_modules/fstream/lib/dir-writer.js:36:23’,
npm ERR! ‘/usr/lib/node_modules/mkdirp/index.js:37:53’,
npm ERR! ‘Object.oncomplete (fs.js:108:15)’ ] }
npm ERR!
npm ERR! Please try running this command again as root/Administrator.

npm ERR! System Linux 2.6.32-754.3.5.el6.x86_64
npm ERR! command “node” “/usr/bin/npm” “install” “-g” “n”
npm ERR! cwd /home/vagrant/local/app/test
npm ERR! node -v v0.10.48
npm ERR! npm -v 1.3.6
npm ERR! path /usr/lib/node_modules/n
npm ERR! fstream_path /usr/lib/node_modules/n
npm ERR! fstream_type Directory
npm ERR! fstream_class DirWriter
npm ERR! code EACCES
npm ERR! errno 3
npm ERR! stack Error: EACCES, mkdir ‘/usr/lib/node_modules/n’
npm ERR! fstream_stack /usr/lib/node_modules/fstream/lib/dir-writer.js:36:23
npm ERR! fstream_stack /usr/lib/node_modules/mkdirp/index.js:37:53
npm ERR! fstream_stack Object.oncomplete (fs.js:108:15)
npm ERR!
npm ERR! Additional logging details can be found in:
npm ERR! /home/vagrant/local/app/test/npm-debug.log
npm ERR! not ok code 0

This time with a different error, it says “Error: EACCES”.

About PhantomJS

WebKit based headless browser.
Webkit is a rendering engine mainly used in web browsers. PhantomJS uses a JavaScript engine called JavaScriptCore (which also uses safari) which is built-in Web-kit.

An environment in which a browser can be executed by commands programs.
Because the program can control the movement, PhantomJS is good at
– Automation test
– Scraping

In automated testing, Selenium + PhantomJS often come out as a set.