Elasticsearch部署

Elasticsearch mounting and deploying

3 分钟阅读

概要说明

Elasticsearch 是一个实时的分布式搜索分析引擎，它能让你以前所未有的速度和规模去探索数据，被用作全文检索、结构化搜索、分析以及这三个功能的组合。

安装配置

操作系统要求

Linux系统

中间件版本要求

JDK版本：1.8 Elasticsearch版本：Elasticsearch 7.12~7.15

安装配置

安装

//下载
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.15.2-linux-x86_64.tar.gz 
//解压
tar -zxvf elasticsearch-7.15.2-linux-x86_64.tar.gz -C /data
//创建运行用户
useradd elasticsearch
//更改文件夹名称
mv elasticsearch-7.15.2/ elasticsearch
//给目录授予权限
chown -R elasticsearch.elasticsearch /data/elasticsearch
//修改系统参数
cat >> /etc/security/limits.conf << EOF
# 配置进程 和线程数
* soft nproc 65536
* hard nproc 65536
#配置 文件句柄数
* soft nofile 65536
* hard nofile 65536
#配置 内存锁定交换
* soft memlock unlimited
* hard memlock unlimited
EOF
cat >> /etc/sysctl.conf << EOF
#禁用内存与磁盘交换
#vm.swappiness=1
#设置虚拟机内存大小
vm.max_map_count=262144
EOF
//加载
sysctl -p
//修改es使用内存
echo '-Xms8g' >> /data/elasticsearch/config/jvm.options
echo '-Xmx8g' >> /data/elasticsearch/config/jvm.options

//创建启动项
cat >> /etc/systemd/system/es.service << EOF
[Unit]
Description=es.service
After=network.target
[Service]
LimitNOFILE=infinity
User=elasticsearch
Group=elasticsearch
Type=forking
ExecStart=/data/elasticsearch/bin/elasticsearch -d
PrivateTmp=True
[Install]
WantedBy=multi-user.target
EOF
systemctl daemon-reload
systemctl start es.service
systemctl enable es.service

安装分词器

//安装ik分词器
cd /data/elasticsearch/plugins/
mkdir ik && cd ik
wget https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.15.2/elasticsearch-analysis-ik-7.15.2.zip
 
unzip elasticsearch-analysis-ik-7.15.2.zip
rm -rf  elasticsearch-analysis-ik-7.15.2.zip

—-配置说明—–»»>

cluster.name: testuser-es ---->>>集群名称，相同名称为一个集群

node.name: node-1 ---->>>节点名称，集群模式下每个节点名称唯一

node.master: true ---->>>当前节点是否可以被选举为master节点，是：true、否：false

node.data: true ---->>>当前节点是否用于存储数据，是：true、否：false

path.data: ---->>>索引数据存放的位置

path.logs: ---->>>日志文件存放的位置

bootstrap.memory_lock ---->>>>需求锁住物理内存，是：true、否：false

bootstrap.system_call_filter ---->>>>SecComp检测，是：true、否：false

network.host ---->>>>监听地址，用于访问该es

network.publish_host ---->>>>可设置成内网ip，用于集群内各机器间通信

http.port ---->>>>es对外提供的http端口，默认 9200

discovery.seed_hosts ---->>>>es7.x 之后新增的配置，写入候选主节点的设备地址，在开启服务后可以被选为主节点

cluster.initial_master_nodes ---->>>>es7.x 之后新增的配置，初始化一个新的集群时需要此配置来选举master

http.cors.enabled ---->>>>是否支持跨域，是：true，在使用head插件时需要此配置

http.cors.allow-origin ---->>>>"*" 表示支持所有域名

xpack.security.enabled: true ---->>>> 开启访问认证

xpack.security.transport.ssl.enabled: true ---->>>>开启ssl访问认证

单机配置

按需部署es模式（单机&集群），分别参照如下配置

注:按配置说明修改实际信息

# vim /opt/elasticsearch-7.15.2/config/elasticsearch.yml
cluster.name: es
node.name: node-1
path.data: /opt/elasticsearch-7.15.2/data
path.logs: /opt/elasticsearch-7.15.2/logs
discovery.seed_hosts: ["x.x.x.x"]
cluster.initial_master_nodes: ["node-1"]
network.host: x.x.x.x
http.port: 9200
transport.tcp.port: 9300
http.cors.enabled: true
http.cors.allow-origin: "*"
xpack.security.enabled: true
xpack.security.transport.ssl.enabled: true

集群配置

节点1

cat >> /data/elasticsearch/config/elasticsearch.yml << EOF
#配置es的集群名称
#集群名称
cluster.name: cluster-es
#节点名称，每个节点的名称不能重复
node.name: node-1
#ip 地址，每个节点的地址不能重复
network.host: 0.0.0.0
#是不是有资格主节点
http.port: 9200
# head 插件需要这打开这两个配置
http.cors.allow-origin: "*"
http.cors.enabled: true
http.max_content_length: 200mb
#es7.x 之后新增的配置，初始化一个新的集群时需要此配置来选举 master
cluster.initial_master_nodes: ["node-1","node-2","node-3"]
#es7.x 之后新增的配置，节点发现
discovery.seed_hosts: ["10.3.9.160","10.3.9.170","10.3.9.180"]
EOF

节点2

cat >> /data/elasticsearch/config/elasticsearch.yml << EOF
#配置es的集群名称
#集群名称
cluster.name: cluster-es
#节点名称，每个节点的名称不能重复
node.name: node-2
#ip 地址，每个节点的地址不能重复
network.host: 0.0.0.0
#是不是有资格主节点
http.port: 9200
# head 插件需要这打开这两个配置
http.cors.allow-origin: "*"
http.cors.enabled: true
http.max_content_length: 200mb
#es7.x 之后新增的配置，初始化一个新的集群时需要此配置来选举 master
cluster.initial_master_nodes: ["node-1","node-2","node-3"]
#es7.x 之后新增的配置，节点发现
discovery.seed_hosts: ["10.3.9.160","10.3.9.170","10.3.9.180"]
EOF

节点3

cat >> /data/elasticsearch/config/elasticsearch.yml << EOF
#配置es的集群名称
#集群名称
cluster.name: cluster-es
#节点名称，每个节点的名称不能重复
node.name: node-3
#ip 地址，每个节点的地址不能重复
network.host: 0.0.0.0
#是不是有资格主节点
http.port: 9200
# head 插件需要这打开这两个配置
http.cors.allow-origin: "*"
http.cors.enabled: true
http.max_content_length: 200mb
#es7.x 之后新增的配置，初始化一个新的集群时需要此配置来选举 master
cluster.initial_master_nodes: ["node-1","node-2","node-3"]
#es7.x 之后新增的配置，节点发现
discovery.seed_hosts: ["10.3.9.160","10.3.9.170","10.3.9.180"]
EOF

启动服务

在线人数	堆内存配置
<500	-Xmx16G
500-1000	-Xmx24G （推荐集群）
1000-2000	-Xmx32G（推荐集群）

按需修改启动jvm大小

vim /opt/elasticsearch-7.15.2/config/jvm.options

-Xms24g

-Xmx24g

后台启动elasticsearch,三台机器上执行（elasticsearch用户）并设置密码

# su -  elasticsearch
$ cd /opt/elasticsearch-7.15.2/bin/
$ ./elasticsearch -d
##设置密码
$ ./elasticsearch-setup-passwords interactive

[elasticsearch@hadoop163 elasticsearch-7.15.2]$ ./bin/elasticsearch-setup-passwords interactive

warning: usage of JAVA_HOME is deprecated, use ES_JAVA_HOME

Future versions of Elasticsearch will require Java 11; your Java version from [/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.272.b10-1.el7_9.x86_64/jre] does not meet this requirement. Consider switching to a distribution of Elasticsearch with a bundled JDK. If you are already using a distribution with a bundled JDK, ensure the JAVA_HOME environment variable is not set.

Initiating the setup of passwords for reserved users elastic,apm_system,kibana,kibana_system,logstash_system,beats_system,remote_monitoring_user.

You will be prompted to enter passwords as the process progresses.

Please confirm that you would like to continue [y/N]y

Enter password for [elastic]:

Reenter password for [elastic]:

Enter password for [apm_system]:

Reenter password for [apm_system]:

Enter password for [kibana_system]:

Reenter password for [kibana_system]:

Enter password for [logstash_system]:

Reenter password for [logstash_system]:

Enter password for [beats_system]:

Reenter password for [beats_system]:

Enter password for [remote_monitoring_user]:

Reenter password for [remote_monitoring_user]:

Changed password for user [apm_system]

Changed password for user [kibana_system]

Changed password for user [kibana]

Changed password for user [logstash_system]

Changed password for user [beats_system]

Changed password for user [remote_monitoring_user]

Changed password for user [elastic]

查看集群状态

curl http://ip:9200/_cat/nodes

如下返回代表部署成功

x.x.x.x 41 7 9 0.66 0.31 0.14 ****** * node-1

最后修改 August 16, 2024: 修改目录名称 (ead47ae)