本文共 6593 字，大约阅读时间需要 21 分钟。
ELK日志平台是一个完整的日志分析系统，有三个开源工具构建组成，分别是：Elasticsearch、Logstash和Kibana。Elasticsearch用于数据分析和深度搜索；Logstash作用是从其他服务器上传输和转发日志，对其集中管理，进行分析；Kibana则是提供了强大的UI展示，将数据可视化。
安装ELK日志平台
ELK基础环境需要java环境，官网要求5.x版本要大于java8。而且安装方式多样化，支持zip、tar.gz、rpm包、deb包、window环境还有docker环境。根据自己喜好选择吧。
我选择的是yum安装，简单方便，系统要求的话没有辣么严格，官网说yum安装方式不再支持centos5.x系列了，非要用centos5.x就去使用tar.gz包吧，官网有具体方法，不再复述。yum安装方式centos6.x和centos7.x都可以，但是我推荐用centos7.x安装，不知道为啥，感觉centos7.x支持更好，centos6.x装完经常会出问题。
还有一点需要说下就是，ELK各个组件版本要一致，官网要求的！
在一个就是安装顺序，为的是确保每个组件相互调用时都能正常运行：
1、Elasticsearch
-
X-Pack for Elasticsearch
Kibana
-
X-Pack for Kibana
LogstashBeatsElasticsearch Hadoop
安装Elasticsearch
1、导入Elasticsearch安装包PGP Key
| 1 2 | rpm -- import https: //artifacts .elastic.co /GPG-KEY-elasticsearch |
2、创建yum源
| 1 2 3 4 5 6 7 8 9 10 | [root@localhost ~] # cat >> /etc/yum.repos.d/elasticsearch.repo <<EOF > [elasticsearch-5.x] > name=Elasticsearch repository for 5.x packages > baseurl=https: //artifacts .elastic.co /packages/5 .x /yum > gpgcheck=1 > gpgkey=https: //artifacts .elastic.co /GPG-KEY-elasticsearch > enabled=1 > autorefresh=1 > type =rpm-md > EOF |
3、安装、启动Elasticsearch进程并开机启动
| 1 2 3 4 | [wangpeng@localhost ~]$ sudo yum install elasticsearch [wangpeng@localhost ~]$ sudo /bin/systemctl daemon-reload [wangpeng@localhost ~]$ sudo /bin/systemctl enable elasticsearch.service [wangpeng@localhost ~]$ sudo systemctl start elasticsearch.service |
4、检查Elasticsearch是否已经启动
查看9200、9300是否已经启动
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 | [wangpeng@localhost ~]$ curl http: //localhost :9200 { "name" : "F5Mw8Pp" , "cluster_name" : "elasticsearch" , "cluster_uuid" : "zVEeXtPNTaeH-TKah7Buzw" , "version" : { "number" : "5.4.0" , "build_hash" : "780f8c4" , "build_date" : "2017-04-28T17:43:27.229Z" , "build_snapshot" : false , "lucene_version" : "6.5.0" }, "tagline" : "You Know, for Search" } |
5、配置Elasticsearch
rpm包配置文件在/etc/elasticsearch下面的elasticsearch.yml
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | vim /etc/elasticsearch/elasticsearch .yml cluster.name: elasticsearch- test node.name: node-1 path.data: /var/lib/elasticsearch path.logs: /var/log/elasticsearch network.host: 0.0.0.0 [wangpeng@localhost ~]$ sudo systemctl restart elasticsearch.service [wangpeng@localhost ~]$ curl http: //localhost :9200 { "name" : "node-1" , "cluster_name" : "elasticsearch-test" , "cluster_uuid" : "zVEeXtPNTaeH-TKah7Buzw" , "version" : { "number" : "5.4.0" , "build_hash" : "780f8c4" , "build_date" : "2017-04-28T17:43:27.229Z" , "build_snapshot" : false , "lucene_version" : "6.5.0" }, "tagline" : "You Know, for Search" } |
6、将/etc/elasticsearch/配置拷贝到/usr/share/elasticsearch/config下面
| 1 2 3 4 | [wangpeng@localhost ~]$ sudo mkdir /usr/share/elasticsearch/config [wangpeng@localhost ~]$ sudo ln -sf /etc/elasticsearch/ * /usr/share/elasticsearch/config/ [wangpeng@localhost ~]$ sudo chown -R elasticsearch:elasticsearch /usr/share/elasticsearch [wangpeng@localhost ~]$ sudo systemctl restart elasticsearch.service |
注意：这一点好多人不会注意，因为你不修复也不会启动失败，但是就是写不进数据进去，这个坑好久才发现，看下日志会报错，但是却能启动，我也是服了！~
7、装个head插件
这个插件5.X官网不再支持了，插件命令没有了，因为它有自己x-pack插件了，但是我装了x-pack发现着实让人吐血，有安全认证方面的问题，导致elk各种问题出现，目前还没研究明白，时间不充裕。
这个head插件我是直接抄的网上大神制作，略有改动。
7.1、下载并配置nodejs
由于head插件本质上还是一个nodejs的工程，因此需要安装node，使用npm来安装依赖的包。（npm可以理解为maven）
去官网下载nodejs，
| 1 2 3 4 5 6 | wget https: //nodejs .org /dist/v8 .1.1 /node-v8 .1.1-linux-x64. tar .xz tar xf node-v8.1.1-linux-x64. tar .xz mv node-v8.1.1-linux-x64 /usr/local/node chown -R elasticsearch:elasticsearch /usr/local/node ln -sf /usr/local/node/bin/node /usr/bin/node ln -sf /usr/local/node/bin/npm /usr/bin/npm |
7.2、安装grunt
| 1 2 | npm install -g grunt-cli ln -sf /usr/local/node/bin/grunt /usr/bin/grunt cd /var/lib/elasticsearch |
7.3、下载、安装并配置head
| 1 2 3 4 5 6 | yum -y install git cd /var/lib/elasticsearch git clone git: //github .com /mobz/elasticsearch-head .git chown -R elasticsearch:elasticsearch elasticsearch- head / cd elasticsearch- head / npm install |
7.4、配置head文件
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 | [root@elk-node1 ~] # cd /var/lib/elasticsearch/elasticsearch-head/ vim Gruntfile.js connect: { server: { options: { port: 9100, hostname : "0.0.0.0" , base: '.' , keepalive: true } } } [root@elk-node1 elasticsearch- head ] # cd _site/ [root@elk-node1 _site] # vim app.js |
把localhost修改成你es的服务器地址:
| 1 | this.base_uri = this.config.base_uri || this.prefs.get( "app-base_uri" ) || "http://10.10.10.10:9200" ; |
7.5、启动head插件
| 1 | grunt server & |
安装Kibana
1、yum安装Kibana
| 1 | [wangpeng@localhost ~]$ sudo yum install kibana |
rpm包配置文件在/etc/kibana下面的kibana.yml
/etc/kibana/kibana.yml
2、配置Kibana文件
| 1 2 3 4 | [wangpeng@localhost ~]$ vim /etc/kibana/kibana .yml server.port: 5601 server.host: "0.0.0.0" elasticsearch.url: "http://localhost:9200" |
3、启动并设置开机启动
| 1 2 3 | [wangpeng@localhost ~]$ sudo systemctl enable kibana.service Created symlink from /etc/systemd/system/multi-user .target.wants /kibana .service to /etc/systemd/system/kibana .service [wangpeng@localhost ~]$ sudo systemctl start kibana.service |
安装Logstash
1、yum安装Logstash
| 1 2 3 4 5 | [wangpeng@localhost ~]$ sudo yum -y install logstash [wangpeng@localhost ~]$ sudo systemctl start logstash.service [wangpeng@localhost ~]$ sudo ln -s /usr/share/logstash/bin/logstash /usr/bin/logstash [wangpeng@localhost ~]$ sudo -u logstash sh -c 'mkdir -pv /usr/share/logstash/config' [wangpeng@localhost ~]$ sudo -u logstash sh -c 'ln -s /etc/logstash/* /usr/share/logstash/config/' |
2、测试Logstash是否能正常运行
| 1 2 3 4 5 6 7 8 9 10 11 12 13 | [wangpeng@localhost ~]$ sudo logstash -e 'input {stdin{}}output { stdout{}}' hello world 2017-06-02T07:14:13.130Z localhost hello world [wangpeng@localhost ~]$ sudo logstash -e 'input {stdin{}}output { stdout{codec=>rubydebug}}' hello world The stdin plugin is now waiting for input: { "@timestamp" => 2017-06-02T07:17:44.053Z, "@version" => "1" , "host" => "localhost" , "message" => "hello world" } |
3、写个测试文件，测试一下es是否能够接受数据
| 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | [wangpeng@localhost ~]$ vim /etc/logstash/conf .d /test .conf input{ stdin{} } output{ elasticsearch{ hosts => "127.0.0.1:9200" index => "test-messages-%{+YYYY.MM.dd}" } } [wangpeng@localhost ~]$ logstash -f /etc/logstash/conf .d /test .conf -t Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties Configuration OK [wangpeng@localhost ~]$ logstash -f /etc/logstash/conf .d /test .conf Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties The stdin plugin is now waiting for input: hello world this is test message study logstash |
4、Kibana里添加该索引（测试），只要es里面能产生索引，Kibana就能加在上去
安装调试先到这里。