mirror of
https://github.com/didi/KnowStreaming.git
synced 2025-12-24 03:42:07 +08:00
文档更新 & 问题修复
This commit is contained in:
zengqiao
@@ -1,171 +1,196 @@
|
||||
## 前言
|
||||
## 2.1、单机部署
|
||||
|
||||
- 本文以 Centos7 系统为例，系统基础配置要求：4 核 8G
|
||||
- 按照本文可以快速部署一套单机模式的 KnowStreaming 环境
|
||||
- 本文以 v3.0.0-bete 版本为例进行部署，如需其他版本请关注[官网](https://knowstreaming.com/)
|
||||
- 部署完成后可以通过浏览器输入 IP:PORT 进行访问，默认用户名密码: admin/admin2022\_
|
||||
- KnowStreaming 同样支持分布式集群模式，如需部署高可用集群，[请联系我们](https://knowstreaming.com/support-center)
|
||||
**风险提示**
|
||||
|
||||
## 1.1、软件版本及依赖
|
||||
⚠️ 脚本全自动安装，会将所部署机器上的 MySQL、JDK、ES 等进行删除重装，请注意原有服务丢失风险。
|
||||
|
||||
| 软件名 | 版本要求 | 默认端口 |
|
||||
| ------------- | -------- | -------- |
|
||||
| Mysql | v5.7+ | 3306 |
|
||||
| Elasticsearch | v6+ | 8060 |
|
||||
| JDK | v8+ | - |
|
||||
| Centos | v6+ | - |
|
||||
| Ubantu | v16+ | - |
|
||||
### 2.1.1、安装说明
|
||||
|
||||
## 1.2、部署方式选择
|
||||
- 以 `v3.0.0-bete` 版本为例进行部署；
|
||||
- 以 CentOS-7 为例，系统基础配置要求 4C-8G；
|
||||
- 部署完成后，可通过浏览器：`IP:PORT` 进行访问，默认端口是 `8080`，系统默认账号密码: `admin` / `admin2022_`；
|
||||
- 本文为单机部署，如需分布式部署，[请联系我们](https://knowstreaming.com/support-center)
|
||||
|
||||
- Shell 部署（单机版本）
|
||||
**软件依赖**
|
||||
|
||||
- 容器化部署（需准备 K8S 环境）
|
||||
| 软件名 | 版本要求 | 默认端口 |
|
||||
| ------------- | ------------ | -------- |
|
||||
| MySQL | v5.7 或 v8.0 | 3306 |
|
||||
| ElasticSearch | v7.6+ | 8060 |
|
||||
| JDK | v8+ | - |
|
||||
| CentOS | v6+ | - |
|
||||
| Ubantu | v16+ | - |
|
||||
|
||||
- 根据操作手册进行手动部署
|
||||
|
||||
|
||||
## 1.3、Shell 部署
|
||||
### 2.1.2、脚本部署
|
||||
|
||||
### 1.3.1、在线方式安装
|
||||
**在线安装**
|
||||
|
||||
#在服务器中下载安装脚本,脚本中会重新安装Mysql
|
||||
wget https://s3-gzpu.didistatic.com/pub/knowstreaming/deploy_KnowStreaming.sh
|
||||
```bash
|
||||
# 在服务器中下载安装脚本, 该脚本中会在当前目录下，重新安装MySQL。重装后的mysql密码存放在当前目录的mysql.password文件中。
|
||||
wget https://s3-gzpu.didistatic.com/pub/knowstreaming/deploy_KnowStreaming.sh
|
||||
|
||||
#执行脚本
|
||||
sh deploy_KnowStreaming.sh
|
||||
# 执行脚本
|
||||
sh deploy_KnowStreaming.sh
|
||||
|
||||
#访问测试
|
||||
127.0.0.1:8080
|
||||
# 访问地址
|
||||
127.0.0.1:8080
|
||||
```
|
||||
|
||||
### 1.3.2、离线方式安装
|
||||
**离线安装**
|
||||
|
||||
#将安装包下载到本地且传输到目标服务器
|
||||
wget https://s3-gzpu.didistatic.com/pub/knowstreaming/KnowStreaming-3.0.0-beta—offline.tar.gz
|
||||
```bash
|
||||
# 将安装包下载到本地且传输到目标服务器
|
||||
wget https://s3-gzpu.didistatic.com/pub/knowstreaming/KnowStreaming-3.0.0-beta—offline.tar.gz
|
||||
|
||||
#解压安装包
|
||||
tar -zxf KnowStreaming-3.0.0-beta—offline.tar.gz
|
||||
# 解压安装包
|
||||
tar -zxf KnowStreaming-3.0.0-beta—offline.tar.gz
|
||||
|
||||
#执行安装脚本
|
||||
sh deploy_KnowStreaming-offline.sh
|
||||
# 执行安装脚本
|
||||
sh deploy_KnowStreaming-offline.sh
|
||||
|
||||
#访问测试
|
||||
127.0.0.1:8080
|
||||
# 访问地址
|
||||
127.0.0.1:8080
|
||||
```
|
||||
|
||||
## 1.4、容器化部署
|
||||
|
||||
|
||||
### 1.4.1、环境依赖及版本要求
|
||||
### 2.1.3、容器部署
|
||||
|
||||
**环境依赖**
|
||||
|
||||
- Kubernetes >= 1.14 ，Helm >= 2.17.0
|
||||
|
||||
- 默认配置为全部安装（elasticsearch + mysql + knowstreaming）
|
||||
- 默认配置为全部安装（ ElasticSearch + MySQL + KnowStreaming）
|
||||
|
||||
- 如果使用已有的 elasticsearch(7.6.x) 和 mysql(5.7) 只需调整 values.yaml 部分参数即可
|
||||
- 如果使用已有的 ElasticSearch(7.6.x) 和 MySQL(5.7) 只需调整 values.yaml 部分参数即可
|
||||
|
||||
### 1.4.2、安装方式
|
||||
**安装命令**
|
||||
|
||||
#下载安装包
|
||||
wget https://s3-gzpu.didistatic.com/pub/knowstreaming/knowstreaming-3.0.0-hlem.tgz
|
||||
```bash
|
||||
# 下载安装包
|
||||
wget https://s3-gzpu.didistatic.com/pub/knowstreaming/knowstreaming-3.0.0-hlem.tgz
|
||||
|
||||
#解压安装包
|
||||
tar -zxf knowstreaming-3.0.0-hlem.tgz
|
||||
# 解压安装包
|
||||
tar -zxf knowstreaming-3.0.0-hlem.tgz
|
||||
|
||||
#执行命令(NAMESPACE需要更改为已存在的)
|
||||
helm install -n [NAMESPACE] knowstreaming knowstreaming-manager/
|
||||
# 执行命令(NAMESPACE需要更改为已存在的)
|
||||
helm install -n [NAMESPACE] knowstreaming knowstreaming-manager/
|
||||
|
||||
#获取KnowStreaming前端ui的service. 默认nodeport方式.(http://nodeIP:nodeport，默认用户名密码：admin/admin2022_)
|
||||
# 获取KnowStreaming前端ui的service. 默认nodeport方式.
|
||||
# (http://nodeIP:nodeport，默认用户名密码：admin/admin2022_)
|
||||
```
|
||||
|
||||
## 1.5、手动部署
|
||||
|
||||
|
||||
### 1.5.1、部署流程
|
||||
### 2.1.4、手动部署
|
||||
|
||||
基础依赖服务部署 ——> KnowStreaming 模块
|
||||
**部署流程**
|
||||
|
||||
### 1.5.2、基础依赖服务部署
|
||||
1. 安装 `JDK-11`、`MySQL`、`ElasticSearch` 等依赖服务
|
||||
2. 安装 KnowStreaming
|
||||
|
||||
#### 如现有环境中已经有相关服务，可跳过对其的安装
|
||||
|
||||
|
||||
#### 基础依赖：JAVA11、Mysql、Elasticsearch
|
||||
#### 2.1.4.1、安装 MySQL 服务
|
||||
|
||||
#### 1.5.2.1、安装 Mysql 服务
|
||||
**yum 方式安装**
|
||||
|
||||
##### 1.5.2.1.1 yum 方式安装
|
||||
```bash
|
||||
# 配置yum源
|
||||
wget https://dev.mysql.com/get/mysql57-community-release-el7-9.noarch.rpm
|
||||
rpm -ivh mysql57-community-release-el7-9.noarch.rpm
|
||||
|
||||
#配置yum源
|
||||
wget https://dev.mysql.com/get/mysql57-community-release-el7-9.noarch.rpm
|
||||
rpm -ivh mysql57-community-release-el7-9.noarch.rpm
|
||||
# 执行安装
|
||||
yum -y install mysql-server mysql-client
|
||||
|
||||
#执行安装
|
||||
yum -y install mysql-server mysql-client
|
||||
# 服务启动
|
||||
systemctl start mysqld
|
||||
|
||||
#服务启动
|
||||
systemctl start mysqld
|
||||
# 获取初始密码并修改
|
||||
old_pass=`grep 'temporary password' /var/log/mysqld.log | awk '{print $NF}' | tail -n 1`
|
||||
|
||||
#获取初始密码并修改
|
||||
old_pass=`grep 'temporary password' /var/log/mysqld.log | awk '{print $NF}' | tail -n 1`
|
||||
mysql -NBe "alter user USER() identified by 'Didi_km_678';" --connect-expired-password -uroot -p$old_pass
|
||||
```
|
||||
|
||||
mysql -NBe "alter user USER() identified by 'Didi_km_678';" --connect-expired-password -uroot -p$old_pass
|
||||
**rpm 方式安装**
|
||||
|
||||
##### 1.5.2.1.2、rpm 包方式安装
|
||||
```bash
|
||||
# 下载安装包
|
||||
wget https://s3-gzpu.didistatic.com/knowsearch/mysql5.7.tar.gz
|
||||
|
||||
#下载安装包
|
||||
wget https://s3-gzpu.didistatic.com/knowsearch/mysql5.7.tar.gz
|
||||
# 解压到指定目录
|
||||
tar -zxf mysql5.7.tar.gz -C /tmp/
|
||||
|
||||
#解压到指定目录
|
||||
tar -zxf mysql5.7.tar.gz -C /tmp/
|
||||
# 执行安装
|
||||
yum -y localinstall /tmp/libaio-*.rpm /tmp/mysql-*.rpm
|
||||
|
||||
#执行安装
|
||||
yum -y localinstall /tmp/libaio-*.rpm /tmp/mysql-*.rpm
|
||||
|
||||
#服务启动
|
||||
systemctl start mysqld
|
||||
# 服务启动
|
||||
systemctl start mysqld
|
||||
|
||||
|
||||
#获取初始密码并修改
|
||||
old_pass=`grep 'temporary password' /var/log/mysqld.log | awk '{print $NF}' | tail -n 1`
|
||||
# 获取初始密码并修改
|
||||
old_pass=`grep 'temporary password' /var/log/mysqld.log | awk '{print $NF}' | tail -n 1`
|
||||
|
||||
mysql -NBe "alter user USER() identified by 'Didi_km_678';" --connect-expired-pa
|
||||
ssword -uroot -p$old_pass
|
||||
mysql -NBe "alter user USER() identified by 'Didi_km_678';" --connect-expired-password -uroot -p$old_pass
|
||||
|
||||
#### 1.5.2.2、配置 JAVA 环境
|
||||
```
|
||||
|
||||
#下载安装包
|
||||
wget https://s3-gzpu.didistatic.com/pub/jdk11.tar.gz #解压到指定目录
|
||||
tar -zxf jdk11.tar.gz -C /usr/local/ #更改目录名
|
||||
mv /usr/local/jdk-11.0.2 /usr/local/java11 #添加到环境变量
|
||||
|
||||
|
||||
#### 2.1.4.2、配置 JDK 环境
|
||||
|
||||
```bash
|
||||
# 下载安装包
|
||||
wget https://s3-gzpu.didistatic.com/pub/jdk11.tar.gz
|
||||
|
||||
# 解压到指定目录
|
||||
tar -zxf jdk11.tar.gz -C /usr/local/
|
||||
|
||||
# 更改目录名
|
||||
mv /usr/local/jdk-11.0.2 /usr/local/java11
|
||||
|
||||
# 添加到环境变量
|
||||
echo "export JAVA_HOME=/usr/local/java11" >> ~/.bashrc
|
||||
echo "export CLASSPATH=/usr/java/java11/lib" >> ~/.bashrc
|
||||
echo "export PATH=\$JAVA_HOME/bin:\$PATH:\$HOME/bin" >> ~/.bashrc
|
||||
echo "export PATH=$JAVA_HOME/bin:$PATH:$HOME/bin" >> ~/.bashrc
|
||||
|
||||
source ~/.bashrc
|
||||
|
||||
#### 1.5.2.3、Elasticsearch 实例搭建
|
||||
```
|
||||
|
||||
#### Elasticsearch 元数据集群来支持平台核心指标数据的存储，如集群维度指标、节点维度指标等
|
||||
|
||||
|
||||
#### 以下安装示例为单节点模式，如需集群部署可以参考[Elasticsearch 官方文档](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/elasticsearch-intro.html)
|
||||
#### 2.1.4.3、ElasticSearch 实例搭建
|
||||
|
||||
#下载安装包
|
||||
wget https://s3-gzpu.didistatic.com/pub/elasticsearch.tar.gz
|
||||
- ElasticSearch 用于存储平台采集的 Kafka 指标；
|
||||
- 以下安装示例为单节点模式，如需集群部署可以参考：[Elasticsearch 官方文档](https://www.elastic.co/guide/en/elasticsearch/reference/7.6/elasticsearch-intro.html)
|
||||
|
||||
#创建ES数据存储目录
|
||||
mkdir -p /data/es_data
|
||||
```bash
|
||||
# 下载安装包
|
||||
wget https://s3-gzpu.didistatic.com/pub/elasticsearch.tar.gz
|
||||
|
||||
#创建ES所属用户
|
||||
useradd arius
|
||||
# 创建ES数据存储目录
|
||||
mkdir -p /data/es_data
|
||||
|
||||
#配置用户的打开文件数
|
||||
echo "arius soft nofile 655350" >>/etc/security/limits.conf
|
||||
echo "arius hard nofile 655350" >>/etc/security/limits.conf
|
||||
echo "vm.max_map_count = 655360" >>/etc/sysctl.conf
|
||||
sysctl -p
|
||||
# 创建ES所属用户
|
||||
useradd arius
|
||||
|
||||
#解压安装包
|
||||
tar -zxf elasticsearch.tar.gz -C /data/
|
||||
# 配置用户的打开文件数
|
||||
echo "arius soft nofile 655350" >> /etc/security/limits.conf
|
||||
echo "arius hard nofile 655350" >> /etc/security/limits.conf
|
||||
echo "vm.max_map_count = 655360" >> /etc/sysctl.conf
|
||||
sysctl -p
|
||||
|
||||
#更改目录所属组
|
||||
chown -R arius:arius /data/
|
||||
# 解压安装包
|
||||
tar -zxf elasticsearch.tar.gz -C /data/
|
||||
|
||||
#修改配置文件(参考以下配置)
|
||||
vim /data/elasticsearch/config/elasticsearch.yml
|
||||
# 更改目录所属组
|
||||
chown -R arius:arius /data/
|
||||
|
||||
# 修改配置文件(参考以下配置)
|
||||
vim /data/elasticsearch/config/elasticsearch.yml
|
||||
cluster.name: km_es
|
||||
node.name: es-node1
|
||||
node.master: true
|
||||
@@ -174,64 +199,64 @@ source ~/.bashrc
|
||||
http.port: 8060
|
||||
discovery.seed_hosts: ["127.0.0.1:9300"]
|
||||
|
||||
#修改内存配置
|
||||
vim /data/elasticsearch/config/jvm.options
|
||||
# 修改内存配置
|
||||
vim /data/elasticsearch/config/jvm.options
|
||||
-Xms2g
|
||||
-Xmx2g
|
||||
|
||||
#启动服务
|
||||
su - arius
|
||||
export JAVA_HOME=/usr/local/java11
|
||||
sh /data/elasticsearch/control.sh start
|
||||
# 启动服务
|
||||
su - arius
|
||||
export JAVA_HOME=/usr/local/java11
|
||||
sh /data/elasticsearch/control.sh start
|
||||
|
||||
#确认状态
|
||||
sh /data/elasticsearch/control.sh status
|
||||
# 确认状态
|
||||
sh /data/elasticsearch/control.sh status
|
||||
```
|
||||
|
||||
### 1.5.3、KnowStreaming 服务部署
|
||||
|
||||
|
||||
#### 以 KnowStreaming 为例
|
||||
#### 2.1.4.4、KnowStreaming 实例搭建
|
||||
|
||||
#下载安装包
|
||||
wget wget https://s3-gzpu.didistatic.com/pub/knowstreaming/KnowStreaming-3.0.0-beta.tar.gz
|
||||
```bash
|
||||
# 下载安装包
|
||||
wget https://s3-gzpu.didistatic.com/pub/knowstreaming/KnowStreaming-3.0.0-beta.tar.gz
|
||||
|
||||
#解压安装包到指定目录
|
||||
tar -zxf KnowStreaming-3.0.0-beta.tar.gz -C /data/
|
||||
# 解压安装包到指定目录
|
||||
tar -zxf KnowStreaming-3.0.0-beta.tar.gz -C /data/
|
||||
|
||||
#修改启动脚本并加入systemd管理
|
||||
cd /data/KnowStreaming/
|
||||
# 修改启动脚本并加入systemd管理
|
||||
cd /data/KnowStreaming/
|
||||
|
||||
#创建相应的库和导入初始化数据
|
||||
mysql -uroot -pDidi_km_678 -e "create database know_streaming;"
|
||||
mysql -uroot -pDidi_km_678 know_streaming < ./init/sql/ddl-ks-km.sql
|
||||
mysql -uroot -pDidi_km_678 know_streaming < ./init/sql/ddl-logi-job.sql
|
||||
mysql -uroot -pDidi_km_678 know_streaming < ./init/sql/ddl-logi-security.sql
|
||||
mysql -uroot -pDidi_km_678 know_streaming < ./init/sql/dml-ks-km.sql
|
||||
mysql -uroot -pDidi_km_678 know_streaming < ./init/sql/dml-logi.sql
|
||||
# 创建相应的库和导入初始化数据
|
||||
mysql -uroot -pDidi_km_678 -e "create database know_streaming;"
|
||||
mysql -uroot -pDidi_km_678 know_streaming < ./init/sql/ddl-ks-km.sql
|
||||
mysql -uroot -pDidi_km_678 know_streaming < ./init/sql/ddl-logi-job.sql
|
||||
mysql -uroot -pDidi_km_678 know_streaming < ./init/sql/ddl-logi-security.sql
|
||||
mysql -uroot -pDidi_km_678 know_streaming < ./init/sql/dml-ks-km.sql
|
||||
mysql -uroot -pDidi_km_678 know_streaming < ./init/sql/dml-logi.sql
|
||||
|
||||
#创建elasticsearch初始化数据
|
||||
sh ./init/template/template.sh
|
||||
# 创建elasticsearch初始化数据
|
||||
sh ./init/template/template.sh
|
||||
|
||||
#修改配置文件
|
||||
vim ./conf/application.yml
|
||||
# 修改配置文件
|
||||
vim ./conf/application.yml
|
||||
|
||||
#监听端口
|
||||
server:
|
||||
# 监听端口
|
||||
server:
|
||||
port: 8080 # web 服务端口
|
||||
tomcat:
|
||||
accept-count: 1000
|
||||
max-connections: 10000
|
||||
|
||||
port: 8080 # web 服务端口
|
||||
tomcat:
|
||||
accept-count: 1000
|
||||
max-connections: 10000
|
||||
# ES地址
|
||||
es.client.address: 127.0.0.1:8060
|
||||
|
||||
#elasticsearch地址
|
||||
es.client.address: 127.0.0.1:8060
|
||||
# 数据库配置（一共三处地方，修改正确的mysql地址和数据库名称以及用户名密码）
|
||||
jdbc-url: jdbc:mariadb://127.0.0.1:3306/know_streaming?.....
|
||||
username: root
|
||||
password: Didi_km_678
|
||||
|
||||
#数据库配置（一共三处地方，修改正确的mysql地址和数据库名称以及用户名密码）
|
||||
jdbc-url: jdbc:mariadb://127.0.0.1:3306/know-streaming?.....
|
||||
username: root
|
||||
password: Didi_km_678
|
||||
|
||||
#启动服务
|
||||
cd /data/KnowStreaming/bin/
|
||||
sh startup.sh
|
||||
|
||||
#### 打开浏览器输入 IP 地址+端口测试(默认端口 8080)，用户名密码: admin/admin2022\_
|
||||
# 启动服务
|
||||
cd /data/KnowStreaming/bin/
|
||||
sh startup.sh
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user