ElasticSearch集群(基础)

这里分享的是ELK集群基础安装文档，这里是 ELK安装基础

环境：Centos 6.6
　　　ElasticSearch 5.1.1
　　　Logstash 5.1.1
　　　Kibana 5.1.1

安装集群管理软件

#安装ElasticSearch
[root@ELK ~]# yum -y install elasticsearch-5.1.1.rpm

#ELK node1配置
[root@ELK ~]# mkdir /etc/elasticsearch/data -p
[root@ELK ~]# id elasticsearch
uid=498(elasticsearch) gid=499(elasticsearch) groups=499(elasticsearch)
[root@ELK ~]# chgrp elasticsearch /etc/elasticsearch/data/ -R
[root@ELK ~]# chmod 775 /etc/elasticsearch/data/ -R
[root@ELK ~]# java -version
java version "1.8.0_112"
Java(TM) SE Runtime Environment (build 1.8.0_112-b15)
Java HotSpot(TM) 64-Bit Server VM (build 25.112-b15, mixed mode)
[root@ELK ~]# ln -s /usr/local/jdk1.8.0_112/bin/java /usr/bin/

[root@ELK ~]# grep -v ^# /etc/elasticsearch/elasticsearch.yml | grep -v ^$
cluster.name: "ES-cluster"
node.name: "es-node01"
node.master: true
node.data: true
http.enabled: true
path.data: /etc/elasticsearch/data
path.logs: /var/log/elasticsearch
network.host: 0.0.0.0
http.port: 9200
http.cors.enabled: true
http.cors.allow-origin: "*"
discovery.zen.ping.unicast.hosts: ["192.168.31.100", "192.168.31.110"]
discovery.zen.minimum_master_nodes: 1
#discovery.zen.minimum_master_nodes（默认是1）：这个参数控制的是，一个节点需要看到的具有master节点资格的最小数量，然后才能在集群中做操作。官方的推荐值是(N/2)+1，其中N是具有master资格的节点的数量（我们的情况是3，因此这个参数设置为2，但对于只有2个节点的情况，设置为2就有些问题了，一个节点DOWN掉后，你肯定连不上2台服务器了，这点需要注意）
gateway.recover_after_nodes: 2
[root@ELK ~]#
[root@ELK ~]# /etc/init.d/elasticsearch start
[root@ELK ~]# netstat -tunlp | grep 9200
tcp        0      0 :::9200                     :::*                        LISTEN      33156/java          
[root@ELK ~]# netstat -tunlp | grep 9300
tcp        0      0 :::9300                     :::*                        LISTEN      33156/java          
[root@ELK ~]#

#ELK node2配置
#node2配置和node1配置相同(略)

测试集群状态

[root@ELK ~]# curl localhost:9200/_cat/nodes?v     #获取集群中节点列表
ip             heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
192.168.31.100            4          81  99    4.32    3.28     3.46 mdi       -      es-node01
192.168.31.110            3          94   5    0.07    0.03     0.01 mdi       *      es-node02
[root@ELK ~]# 
[root@ELK ~]# curl localhost:9200/_cat/health?v    #集群健康检查
epoch      timestamp cluster    status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1482982586 11:36:26  ES-cluster green           2         2     22  11    0    0        0             0                  -                100.0%
[root@ELK ~]#
[root@ELK ~]# curl 'localhost:9200/_cat/indices?v'  #获取ElasticSearch索引
health status index                       uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   logstash-message-2016.12.29 qkvr3jmWQei1oBhEy9VnCA   5   1         28            0    376.3kb        188.1kb
green  open   logstash-nginx-2016.12.29   B9p9qwjsTlaE4fcZcIgJag   5   1          5            0      104kb           52kb
green  open   .kibana                     bNLdONDMRdWK2-HdYkUuAA   1   1          3            0     34.1kb           17kb
[root@ELK ~]#

//注：当ElasticSearch配置完成后，node2会复制node1的索引
[root@ELK2 ~]# curl 'localhost:9200/_cat/indices?v'
health status index                       uuid                   pri rep docs.count docs.deleted store.size pri.store.size
green  open   logstash-message-2016.12.29 qkvr3jmWQei1oBhEy9VnCA   5   1         28            0    376.3kb        188.1kb
green  open   .kibana                     bNLdONDMRdWK2-HdYkUuAA   1   1          3            0     34.1kb           17kb
green  open   logstash-nginx-2016.12.29   B9p9qwjsTlaE4fcZcIgJag   5   1          5            0      104kb           52kb
[root@ELK2 ~]#

#ElasticSearch集群日志文件位置：
[root@ELK ~]# ls /var/log/elasticsearch/
ES-cluster_deprecation.log             ES-cluster_index_search_slowlog.log
ES-cluster_index_indexing_slowlog.log  ES-cluster.log
[root@ELK ~]#

安装elasticsearch-head插件

由于Elasticsearch 5.0 head插件不能以插件形式安装，因此需要单独安装
参考：https://github.com/mobz/elasticsearch-head

Running as a plugin of Elasticsearch

Install elasticsearch-head:
– for Elasticsearch 5.x:
site plugins are not supported. Run elasticsearch-head as a standalone server
– for Elasticsearch 2.x – 4.x:
sudo elasticsearch/bin/plugin install mobz/elasticsearch-head
– for Elasticsearch 1.x:
sudo elasticsearch/bin/plugin -install mobz/elasticsearch-head/1.x
– for Elasticsearch 0.9:
sudo elasticsearch/bin/plugin -install mobz/elasticsearch-head/0.9
open http://localhost:9200/_plugin/head/

Running with built in server：
enable cors by adding http.cors.enabled: true in elasticsearch configuration. Don’t forget to also set http.cors.allow-origin because no origin allowed by default. http.cors.allow-origin: "*" is valid value, however it’s considered as a security risk as your cluster is open to cross origin from anywhere. Check Elasticsearch documentation on this parameter: https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-http.html#modules-http
git clone git://github.com/mobz/elasticsearch-head.git
cd elasticsearch-head
npm install
grunt server
open http://localhost:9100/

#npm命令需要安装nodejs
[root@ELK ~]# wget https://nodejs.org/dist/v0.12.16/node-v0.12.16.tar.gz
[root@ELK ~]# tar -zxf node-v0.12.16.tar.gz 
[root@ELK ~]# cd node-v0.12.16
[root@ELK node-v0.12.16]#
[root@ELK node-v0.12.16]# ./configure --prefix=/usr/local/node-v0.12.16
[root@ELK node-v0.12.16]# make && make install
[root@ELK ~]# ls -l /usr/local/node-v0.12.16/bin/npm
lrwxrwxrwx 1 root root 38 Dec 28 12:43 /usr/local/node-v0.12.16/bin/npm -> ../lib/node_modules/npm/bin/npm-cli.js
[root@ELK ~]# ln -s /usr/local/node-v0.12.16/bin/npm /usr/bin/
[root@ELK ~]# ll /usr/bin/npm 
lrwxrwxrwx 1 root root 32 Dec 28 14:31 /usr/bin/npm -> /usr/local/node-v0.12.16/bin/npm
[root@ELK ~]#
[root@ELK ~]# ln -s /usr/local/node-v0.12.16/bin/node /usr/bin/
[root@ELK ~]# ll /usr/bin/node 
lrwxrwxrwx 1 root root 33 Dec 28 15:06 /usr/bin/node -> /usr/local/node-v0.12.16/bin/node
[root@ELK ~]#

[root@ELK ~]# yum -y install git
[root@ELK ~]# cd /usr/local/
[root@ELK local]# git clone git://github.com/mobz/elasticsearch-head.git
[root@ELK local]# cd elasticsearch-head
[root@ELK elasticsearch-head]# npm install
#如果在elasticsearch-head目录下node_modules/grunt下如果没有grunt二进制程序，则需要执行
[root@ELK ~]# cd /usr/local/elasticsearch-head/
[root@ELK elasticsearch-head]# npm install grunt --save
[root@ELK elasticsearch-head]# ls
Dockerfile                          grunt_fileSets.js  node_modules                  README.textile  test
elasticsearch-head.sublime-project  index.html         package.json                  _site
Gruntfile.js                        LICENCE            plugin-descriptor.properties  src
[root@ELK elasticsearch-head]# ls -l node_modules/grunt/bin/grunt
-rwxr-xr-x 1 root root 53 Apr  6  2016 node_modules/grunt/bin/grunt
[root@ELK elasticsearch-head]#
[root@ELK elasticsearch-head]# head -98 Gruntfile.js | tail -9
		connect: {
			server: {
				options: {
					hostname: '0.0.0.0',   #添加这行
					port: 9100,
					base: '.',
					keepalive: true
				}
			}
[root@ELK elasticsearch-head]# /usr/local/elasticsearch-head/node_modules/grunt/bin/grunt server  #如果执行报错看下面的解决办法
Running "connect:server" (connect) task
Waiting forever...
Started connect web server on http://localhost:9100

#最新安装步骤
yum -y install git make gcc gcc-c++ wget bzip2
wget https://nodejs.org/dist/v8.9.3/node-v8.9.3.tar.gz
tar -zxf node-v8.9.3.tar.gz
cd node-v8.9.3
./configure --prefix=/usr/local/node-v8.9.3
make && make install
ln -s /usr/local/node-v8.9.3/bin/npm /usr/bin/
ln -s /usr/local/node-v8.9.3/bin/node /usr/bin/
cd /usr/local/ && git clone https://github.com/mobz/elasticsearch-head.git
mkdir -p /tmp/phantomjs/ && wget https://github.com/Medium/phantomjs/releases/download/v2.1.1/phantomjs-2.1.1-linux-x86_64.tar.bz2 -O /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
cd elasticsearch-head/ && npm install
cd /usr/local/elasticsearch-head/ && npm install grunt --save
npm install grunt-contrib-clean --registry=https://registry.npm.taobao.org
npm install grunt-contrib-concat --registry=https://registry.npm.taobao.org
npm install grunt-contrib-watch --registry=https://registry.npm.taobao.org
npm install grunt-contrib-connect --registry=https://registry.npm.taobao.org
npm install grunt-contrib-copy --registry=https://registry.npm.taobao.org
npm install grunt-contrib-jasmine --registry=https://registry.npm.taobao.org
cd /usr/local/elasticsearch-head/ && sed -i "/port: 9100/ihostname: '0.0.0.0'," Gruntfile.js
cd /usr/local/elasticsearch-head/ && /usr/local/elasticsearch-head/node_modules/grunt/bin/grunt server &

Docker

听说有些依赖在国内解决不了，已经考虑使用Docker

# docker pull salgat/elasticsearch-head
# docker run -d --name elasticsearch-head -p9100:9100 docker.io/salgat/elasticsearch-head

官方也提供了elasticsearch-head在Docker容器中使用的方法
Running with docker
for Elasticsearch 5.x: docker run -p 9100:9100 mobz/elasticsearch-head:5
for Elasticsearch 2.x: docker run -p 9100:9100 mobz/elasticsearch-head:2
for Elasticsearch 1.x: docker run -p 9100:9100 mobz/elasticsearch-head:1
for fans of alpine there is mobz/elasticsearch-head:5-alpine
open http://localhost:9100/

Q & A

如果服务器重启了，有时发现执行/usr/local/elasticsearch-head/node_modules/grunt/bin/grunt server报错Fatal error: Unable to find local grunt.，需要重新按照Question1的解决方法操作一下，也可以执行下面的脚本

[root@ELK ~]# cat check_es_head_grunt.sh 
#!/bin/bash
# Auther: yfshare
# Date:2016-12-29
eshead_dir="/usr/local/elasticsearch-head"
grunt_dir="$eshead_dir/node_modules/grunt"
grunt_bin="$grunt_dir/bin/grunt"

kill -9 `ps -ef | grep -iw 'grunt' | grep -v grep | awk '{print $2}'` &>/dev/null
[ ! -x "$grunt_bin" ] && chmod 755 "$grunt_bin"
echo ''
echo 'Please wait a moment.'
cd "$eshead_dir"
npm install grunt --save-dev &> /dev/null
cd "$eshead_dir"
"$grunt_bin" server &
[ $? -eq 0 ] && echo 'start ok.'
[root@ELK ~]#

[root@ELK ~]# sh check_es_head_grunt.sh 

Please wait a moment.
start ok.
Running "connect:server" (connect) task
Waiting forever...
Started connect web server on http://localhost:9100

[root@ELK ~]#

Question1：如果报下面的错误：

[root@ELK ~]# /usr/local/elasticsearch-head/node_modules/grunt/bin/grunt server  
grunt-cli: The grunt command line interface (v1.2.0)  
Fatal error: Unable to find local grunt.
If you're seeing this message, grunt hasn't been installed locally to
your project. For more information about installing and configuring grunt,
please see the Getting Started guide:  
http://gruntjs.com/getting-started  
[root@ELK ~]#  

Answer：  
[root@ELK ~]# cd /usr/local/elasticsearch-head/  
[root@ELK elasticsearch-head]# npm install grunt --save-dev  
再执行/usr/local/elasticsearch-head/node_modules/grunt/bin/grunt server就OK了

Question2：
之前修改ElasticSearch 5.1的network.host的IP时，不管修改成什么(注释network.host除外)，重启均报错，开始以为是不能绑定IP地址。之前有注意到日志里的报错的bootstrap checks failed. max number of threads [1024] for user [elasticsearch] is too low, increase to at least [2048]因为注释了network.host后ElasticSearch能起来，所以没意识到是它的问题

#ElasticSearch日志报错
[2016-12-28T16:57:23,190][INFO ][o.e.n.Node               ] [es-node01] starting ...
[2016-12-28T16:57:24,616][INFO ][o.e.t.TransportService   ] [es-node01] publish_address {192.168.31.100:9300}, bound_addresses {192.168.31.100:9300}
[2016-12-28T16:57:24,686][INFO ][o.e.b.BootstrapCheck     ] [es-node01] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2016-12-28T16:57:24,708][ERROR][o.e.b.Bootstrap          ] [es-node01] node validation exception
bootstrap checks failed
max number of threads [1024] for user [elasticsearch] is too low, increase to at least [2048]
[2016-12-28T16:57:24,739][INFO ][o.e.n.Node               ] [es-node01] stopping ...
[2016-12-28T16:57:25,275][INFO ][o.e.n.Node               ] [es-node01] stopped
[2016-12-28T16:57:25,277][INFO ][o.e.n.Node               ] [es-node01] closing ...
[2016-12-28T16:57:25,500][INFO ][o.e.n.Node               ] [es-node01] closed

Answer：修改/etc/security/limits.conf配置文件，添加如下参数重启ElasticSearch即可
关于ElasticSearch5.1的network.host设置
参考：https://www.elastic.co/guide/en/elasticsearch/reference/5.1/modules-network.html#common-network-settings

[root@ELK ~]# grep -v ^# /etc/security/limits.conf | grep -v ^$
elasticsearch	soft	nproc	2048
elasticsearch	hard	nproc	4096
[root@ELK ~]#

Question3：
如果执行/usr/local/elasticsearch-head/node_modules/grunt/bin/grunt server报下面的错误

[root@ELK-test ~]# /usr/local/elasticsearch-head/node_modules/grunt/bin/grunt server
grunt-cli: The grunt command line interface (v1.2.0)

Fatal error: Unable to find local grunt.

If you're seeing this message, grunt hasn't been installed locally to
your project. For more information about installing and configuring grunt,
please see the Getting Started guide:

http://gruntjs.com/getting-started
[root@ELK-test ~]#

Answer：
#进入elasticsearch-head安装目录即可，因为在别的地方找不到Gruntfile.js文件
[root@ELK-test ~]# cd /usr/local/elasticsearch-head/

Question4：
如果执行/usr/local/elasticsearch-head/node_modules/grunt/bin/grunt server报下面的错误

[root@ELK-test elasticsearch-head]# /usr/local/elasticsearch-head/node_modules/grunt/bin/grunt server
>> Local Npm module "grunt-contrib-clean" not found. Is it installed?
>> Local Npm module "grunt-contrib-concat" not found. Is it installed?
>> Local Npm module "grunt-contrib-watch" not found. Is it installed?
>> Local Npm module "grunt-contrib-connect" not found. Is it installed?
>> Local Npm module "grunt-contrib-copy" not found. Is it installed?
>> Local Npm module "grunt-contrib-jasmine" not found. Is it installed?
Warning: Task "connect:server" not found. Use --force to continue.

Aborted due to warnings.
[root@ELK-test elasticsearch-head]#

Answer：
出现以下提示，为Gruntfile.js引用的，缺少以下包
[root@ELK-test elasticsearch-head]# npm install grunt-contrib-clean --registry=https://registry.npm.taobao.org
[root@ELK-test elasticsearch-head]# npm install grunt-contrib-concat --registry=https://registry.npm.taobao.org
[root@ELK-test elasticsearch-head]# npm install grunt-contrib-watch --registry=https://registry.npm.taobao.org
[root@ELK-test elasticsearch-head]# npm install grunt-contrib-connect --registry=https://registry.npm.taobao.org
[root@ELK-test elasticsearch-head]# npm install grunt-contrib-copy --registry=https://registry.npm.taobao.org
[root@ELK-test elasticsearch-head]# npm install grunt-contrib-jasmine --registry=https://registry.npm.taobao.org

[root@ELK-test elasticsearch-head]# /usr/local/elasticsearch-head/node_modules/grunt/bin/grunt server &
[1] 22877
[root@ELK-test elasticsearch-head]#

Question5：
如果在执行npm install时报如下错误

Answer：可以先下载其所需的文件再执行npm install安装

mkdir -p /tmp/phantomjs/
wget https://github.com/Medium/phantomjs/releases/download/v2.1.1/phantomjs-2.1.1-linux-x86_64.tar.bz2 -O /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2

#npm install
npm WARN deprecated coffee-script@1.10.0: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)
npm WARN deprecated http2@3.3.7: Use the built-in module in node 9.0.0 or newer, instead

> phantomjs-prebuilt@2.1.16 install /usr/local/elasticsearch-head/node_modules/phantomjs-prebuilt
> node install.js

PhantomJS not found on PATH
Downloading https://github.com/Medium/phantomjs/releases/download/v2.1.1/phantomjs-2.1.1-linux-x86_64.tar.bz2
Saving to /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
Receiving...

Error making request.
Error: connect ETIMEDOUT 52.216.82.152:443
    at Object._errnoException (util.js:1024:11)
    at _exceptionWithHostPort (util.js:1046:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1182:14)

Please report this full log at https://github.com/Medium/phantomjs
npm WARN elasticsearch-head@0.0.0 license should be a valid SPDX license expression
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.1.3 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! phantomjs-prebuilt@2.1.16 install: `node install.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the phantomjs-prebuilt@2.1.16 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2017-12-16T02_18_46_188Z-debug.log

注：集群配置完成后，需要把Logstash里的ElasticSearch的地址修改正确
即：Logstash配置文件的output关于ElasticSearch的配置

output {
        elasticsearch {
                hosts => ["192.168.31.100:9200"]
                index => "logstash-nginx-%{+YYYY.MM.dd}"
        }

效果图

2台elasticsearch，显示一台master和一台slave才是正常的

本文参考：https://www.chinasa.net/archives/325.html
　　　　　http://zerosre.com/2016/12/20/k8s日志管理-三/
　　　　　http://hnr520.blog.51cto.com/4484939/1867033
nodejs下载地址：https://nodejs.org/en/blog/release/v0.12.16/

附件：
yum.repo
elasticsearch-head.zip
node-v0.12.16.tar.gz
phantomjs-2.1.1-linux-x86_64.tar.bz2
check_es_head_grunt.sh

---

本作品采用知识共享署名 2.5 中国大陆许可协议进行许可，欢迎转载，但转载请注明来自Jack Wang Blog，并保持转载后文章内容的完整。本人保留所有版权相关权利。

本文出自”Jack Wang Blog”：http://www.yfshare.vip/2017/12/04/ElasticSearch%E9%9B%86%E7%BE%A4-%E5%9F%BA%E7%A1%80/