Compare commits

...

213 Commits

Author SHA1 Message Date
EricZeng
045f65204b Merge pull request #633 from didi/master
合并主分支
2022-09-29 13:09:19 +08:00
EricZeng
f414b47a78 1、补充升级至v3.0.0信息;2、增加v3.0.0变更内容;(#636)
1、补充升级至v3.0.0信息;2、增加v3.0.0变更内容;(#)
2022-09-29 13:08:32 +08:00
EricZeng
44f4e2f0f9 Merge pull request #635 from didi/dev
合并前端调整的内容
2022-09-29 11:50:25 +08:00
zengqiao
2361008bdf Merge branch 'dev' of github.com:didi/KnowStreaming into dev 2022-09-29 11:49:00 +08:00
zengqiao
7377ef3ec5 增加v3.0.0变更内容 2022-09-29 11:45:29 +08:00
lucasun
a28d064b7a Merge pull request #634 from GraceWalk/dev
前端 bug 修复 & 问题优化
2022-09-29 11:23:25 +08:00
GraceWalk
e2e57e8575 fix: 依赖版本更新 2022-09-29 11:15:47 +08:00
zengqiao
9d90bd2835 补充升级至v3.0.0信息 2022-09-29 11:04:49 +08:00
EricZeng
7445e68df4 Merge pull request #632 from didi/master
合并主分支
2022-09-29 10:53:54 +08:00
GraceWalk
ab42625ad2 fix: 数字展示格式化 2022-09-29 10:52:31 +08:00
GraceWalk
18789a0a53 fix: IconFont 组件改为从独立包引入 2022-09-29 10:51:52 +08:00
zengqiao
68a37bb56a Merge branch 'master' of github.com:didi/KnowStreaming 2022-09-29 10:49:46 +08:00
GraceWalk
3b33652c47 fix: Rebalance 卡片 icon 颜色调整 2022-09-29 10:48:52 +08:00
GraceWalk
1e0c4c3904 feat: Topic 详情消息 Value 列支持复制 2022-09-29 10:48:09 +08:00
zengqiao
04e223de16 修改协议文案 2022-09-29 10:48:00 +08:00
GraceWalk
c4a691aa8a fix: 多集群列表兼容集群无 ZK 情况 2022-09-29 10:44:28 +08:00
GraceWalk
ff9dde163a feat: 图表支持存储拖拽排序 & 补点逻辑优化 2022-09-29 10:42:44 +08:00
EricZeng
eb7efbd1a5 增加字段校验注解(#631)
增加字段校验注解(#631)
2022-09-29 09:59:02 +08:00
zengqiao
8c8c362c54 Merge branch 'dev' of github.com:didi/KnowStreaming into dev 2022-09-28 20:19:35 +08:00
zengqiao
66e119ad5d 增加字段校验注解 2022-09-28 20:16:06 +08:00
EricZeng
6dedc04a05 Merge pull request #630 from didi/dev
合并开发分支
2022-09-28 20:14:51 +08:00
EricZeng
0cf8bad0df Merge pull request #629 from didi/master
合并主分支
2022-09-28 20:06:26 +08:00
zengqiao
95c9582d8b 优化消费组详情指标为实时获取 2022-09-28 20:03:23 +08:00
EricZeng
7815126ff5 1、修复Group指标防重复不生效问题;2、修复自动创建ES索引模版失败问题; (#628)
* 修复自动创建ES索引模版失败问题

* 修复Group指标防重复不生效问题

Co-authored-by: zengqiao <zengqiao@didiglobal.com>
2022-09-28 19:55:30 +08:00
zengqiao
a5fa9de54b 修复Group指标防重复不生效问题 2022-09-28 19:52:11 +08:00
zengqiao
95f1a2c630 修复自动创建ES索引模版失败问题 2022-09-28 19:46:07 +08:00
zengqiao
1e256ae1fd 修复自动创建ES索引模版失败问题 2022-09-28 19:44:33 +08:00
zengqiao
9fc9c54fa1 bump version to 3.0.0 2022-09-28 11:20:16 +08:00
zengqiao
1b362b1e02 Merge branch 'master' of github.com:didi/KnowStreaming 2022-09-28 11:16:54 +08:00
EricZeng
04e3172cca [ISSUE-624]过滤掉不存在的Topic(#625)
[ISSUE-624]过滤掉不存在的Topic(#625)
2022-09-28 11:13:15 +08:00
EricZeng
1caab7f3f7 [ISSUE-624]过滤掉不存在的Topic(#624)
[ISSUE-624]过滤掉不存在的Topic(#624)
2022-09-28 10:41:39 +08:00
zengqiao
9d33c725ad [ISSUE-624]过滤掉不存在的Topic(#624)
同步Group元信息时,如果Topic已经不存在了,则过滤掉该Group+Topic信息
2022-09-28 10:39:33 +08:00
EricZeng
6ed1d38106 [ISSUE-598]Fix start_time not set when create reassign job in MySQL-8 (#623)
[ISSUE-598]Fix start_time not set when create reassign job in MySQL-8 (#623 )
2022-09-28 10:26:56 +08:00
zengqiao
0f07ddedaf [ISSUE-598]Fix start_time not set when create reassign job in MySQL-8 2022-09-28 10:24:32 +08:00
EricZeng
289945b471 Merge pull request #622 from didi/dev
后端补充Kafka集群运行模式字段信息
2022-09-28 10:08:17 +08:00
zengqiao
f331a6d144 后端补充Kafka集群运行模式字段信息 2022-09-27 18:43:22 +08:00
EricZeng
0c8c12a651 Merge pull request #621 from didi/dev
指标发送ES类按照指标类别拆分
2022-09-27 18:38:05 +08:00
zengqiao
028c3bb2fa 指标发送ES类按照指标类别拆分 2022-09-27 10:19:18 +08:00
EricZeng
d7a5a0d405 健康巡检任务按照类型进行拆分
健康巡检任务按照类型进行拆分
2022-09-27 10:17:12 +08:00
zengqiao
5ef5f6e531 健康巡检任务按照类型进行拆分 2022-09-26 20:10:49 +08:00
EricZeng
1d205734b3 Merge pull request #619 from didi/dev
集群信息中,补充ZK配置字段
2022-09-26 19:50:26 +08:00
Peng
5edd43884f Update README.md 2022-09-26 18:43:25 +08:00
zengqiao
c1992373bc 集群信息中,补充ZK配置字段 2022-09-26 11:10:38 +08:00
EricZeng
ed562f9c8a Merge pull request #618 from didi/dev
DB中Group信息的更新方式,由replace调整为insert或update
2022-09-26 10:02:24 +08:00
zengqiao
b4d44ef8c7 DB中Group信息的更新方式,由replace调整为insert或update 2022-09-23 17:02:25 +08:00
EricZeng
ad0c16a1b4 升级Helm版本及增加Docker相关文件
升级Helm版本及增加Docker相关文件
2022-09-23 16:17:00 +08:00
wangdongfang-aden
7eabe66853 Merge pull request #616 from wangdongfang-aden/dev
添加docker-compose部署和更新helm
2022-09-23 14:50:22 +08:00
wangdongfang-aden
3983d73695 Update Chart.yaml 2022-09-23 14:47:40 +08:00
wangdongfang-aden
161d4c4562 Update 单机部署手册.md 2022-09-23 14:46:27 +08:00
wangdongfang-aden
9a1e89564e Update 单机部署手册.md 2022-09-23 14:44:49 +08:00
wangdongfang-aden
0c18c5b4f6 Update 单机部署手册.md 2022-09-23 14:43:23 +08:00
wangdongfang-aden
3e12ba34f7 Update docker-compose.yml 2022-09-23 14:33:05 +08:00
wangdongfang-aden
e71e29391b Delete ks-start.sh 2022-09-23 14:26:24 +08:00
wangdongfang-aden
9b7b9a7af0 Delete es_template_create.sh 2022-09-23 14:26:16 +08:00
wangdongfang-aden
a23819c308 Create ks-start.sh 2022-09-23 14:19:35 +08:00
wangdongfang-aden
6cb1825d96 Create es_template_create.sh 2022-09-23 14:19:10 +08:00
wangdongfang-aden
77b8c758dc Create initsql 2022-09-23 14:18:17 +08:00
wangdongfang-aden
e5a582cfad Create my.cnf 2022-09-23 14:17:25 +08:00
wangdongfang-aden
ec83db267e Create init.sh 2022-09-23 14:17:02 +08:00
wangdongfang-aden
bfd026cae7 Create dockerfile 2022-09-23 14:16:28 +08:00
wangdongfang-aden
35f1dd8082 Create dockerfile 2022-09-23 14:14:47 +08:00
wangdongfang-aden
7ed0e7dd23 Create dockerfile 2022-09-23 14:14:02 +08:00
wangdongfang-aden
1a3cbf7a9d Create knowstreaming.conf 2022-09-23 14:07:04 +08:00
wangdongfang-aden
d9e4abc3de Create ks-start.sh 2022-09-23 14:05:59 +08:00
wangdongfang-aden
a4186085d3 Create es_template_create.sh 2022-09-23 14:05:05 +08:00
wangdongfang-aden
26b1846bb4 Create docker-compose.yml 2022-09-23 14:03:14 +08:00
wangdongfang-aden
1aa89527a6 helm update 3.0.0-beta.3 2022-09-23 11:36:46 +08:00
wangdongfang-aden
eac76d7ad0 helm update 3.0.0-beta.3 2022-09-23 11:36:01 +08:00
wangdongfang-aden
cea0cd56f6 Merge pull request #607 from haoqi123/dev
[单机部署手册.md]docker-compose部署方式添加注释描述
2022-09-23 10:27:04 +08:00
EricZeng
c4b897f282 bump version to 3.0.0-beta.4
bump version to 3.0.0-beta.4
2022-09-23 10:24:52 +08:00
zengqiao
47389dbabb bump version to 3.0.0-beta.4 2022-09-23 10:17:58 +08:00
haoqi
a2f8b1a851 1. [单机部署手册.md]docker-compose部署方式添加注释描述 2022-09-22 19:46:21 +08:00
EricZeng
feac0a058f Merge pull request #613 from didi/dev
补充v3.0.0-beta.2变更信息
2022-09-22 17:30:35 +08:00
zengqiao
27eeac9fd4 补充v3.0.0-beta.2变更信息 2022-09-22 17:28:51 +08:00
EricZeng
a14db4b194 Merge pull request #612 from didi/dev
合并开发分支
2022-09-22 17:28:09 +08:00
lucasun
54ee271a47 Merge pull request #611 from GraceWalk/dev
修复前端bug和体验问题
2022-09-22 15:51:46 +08:00
GraceWalk
a3a9be4f7f fix: 更正前端本地环境接口代理地址 2022-09-22 15:37:24 +08:00
GraceWalk
d4f0a832f3 fix: 样式更新 2022-09-22 15:31:52 +08:00
GraceWalk
7dc533372c fix: 更正文件引用路径 2022-09-22 15:31:34 +08:00
GraceWalk
1737d87713 fix: 修复配置无法删除的问题 2022-09-22 15:31:13 +08:00
GraceWalk
dbb98dea11 fix: 更新登录页图片 2022-09-22 15:21:04 +08:00
GraceWalk
802b382b36 fix: Topic Messages 详情提示优化 2022-09-22 15:20:31 +08:00
GraceWalk
fc82999d45 fix: 消费测试 Message 限制最大值 2022-09-22 15:19:56 +08:00
GraceWalk
08aa000c07 refactor: 接入/编辑集群优化 2022-09-22 15:19:03 +08:00
GraceWalk
39015b5100 feat: 多集群管理列表页增加手动刷新功能 2022-09-22 15:18:13 +08:00
GraceWalk
0d635ad419 refactor: webpack 配置结构调整 2022-09-22 15:13:25 +08:00
EricZeng
9133205915 Merge pull request #610 from didi/dev
合并开发分支
2022-09-22 14:51:23 +08:00
zengqiao
725ac10c3d 1、调整KafkaZKDao位置;2、offset信息获取时,过滤掉无leader分区;3、调整验证ZK是否合法时的session超时时间 2022-09-22 11:30:46 +08:00
zengqiao
2b76358c8f Overview页面,后端增加排序信息 2022-09-22 11:24:13 +08:00
zengqiao
833c360698 bump oshi-core version to 5.6.1 2022-09-22 11:17:59 +08:00
zengqiao
7da1e67b01 FAQ补充权限识别失败问题说明 2022-09-22 11:13:54 +08:00
GraceWalk
7eb86a47dd fix: 部分依赖更新 2022-09-21 16:22:45 +08:00
GraceWalk
d67e383c28 feat: 系统管理列表增加手动刷新功能 2022-09-21 16:21:57 +08:00
GraceWalk
8749d3e1f5 fix: config 子应用 axios 配置错误兼容 2022-09-21 16:21:07 +08:00
GraceWalk
30fba21c48 fix: 生产测试单词发送消息数限制为 0~1000 2022-09-21 16:15:19 +08:00
GraceWalk
d83d35aee9 fix: 样式 & 文案优化 2022-09-21 16:12:13 +08:00
GraceWalk
1d3caeea7d feat: Cluster 图表去掉放大功能 2022-09-21 16:11:14 +08:00
haoqi
26916f6632 1. [单机部署手册.md]docker-compose部署方式添加注释描述
2. 更改docker-compose中ui对外访问port为80
2022-09-21 12:55:43 +08:00
EricZeng
fbfa0d2d2a Merge pull request #600 from haoqi123/dev
docker-compose addition
2022-09-21 10:49:08 +08:00
haoqi
e626b99090 1. 删除km-dist/docker文件夹,以[单机部署手册.md]为准 2022-09-20 19:30:20 +08:00
haoqi123
203859b71b Merge branch 'didi:dev' into dev 2022-09-20 19:25:12 +08:00
haoqi
9a25c22f3a 1. 调整docker-compose.yml中各个服务的镜像
2. 经过@wangdongfang-aden大哥的调试将helm与docker镜像合二为一,于是删减掉各个镜像的Dockerfile与启动脚本,后续也不需要额外维护
2022-09-20 19:23:18 +08:00
zengqiao
0a03f41a7c 后端增加指标摆放顺序功能 2022-09-20 14:42:22 +08:00
zengqiao
56191939c8 Merge branch 'dev' of github.com:didi/KnowStreaming into dev 2022-09-20 14:23:09 +08:00
zengqiao
beb754aaaa 修复JMX连接被关闭,抛出IOException后,未进行连接重建的问题 2022-09-20 14:22:06 +08:00
EricZeng
f234f740ca Merge pull request #603 from didi/dev
合并开发分支
2022-09-20 10:51:39 +08:00
EricZeng
e14679694c Merge pull request #602 from f1558/dev
fix issue
2022-09-20 10:31:16 +08:00
zengqiao
e06712397e 修复因DB中Broker信息不存在导致TotalLogSize指标获取时抛空指针问题 2022-09-20 10:27:30 +08:00
Richard
b6c6df7ffc fix issue
* SQL specification comments to avoid direct operation failure
2022-09-20 09:42:42 +08:00
zengqiao
375c6f56c9 修改GroupOffsetResetEnum类名为OffsetTypeEnum 2022-09-19 13:55:59 +08:00
EricZeng
0bf85c97b5 Merge pull request #555 from superspeedone/dev
Dev
2022-09-19 11:18:28 +08:00
EricZeng
630e582321 Merge pull request #593 from Mengqi777/mengqi-dev
fix: adjust os judgment method with uname
2022-09-19 10:34:16 +08:00
EricZeng
a89fe23bdd Merge pull request #597 from WYAOBO/dev
文档更新
2022-09-19 10:15:38 +08:00
haoqi
a7a5fa9a31 1. 调整docker-compose.yml中networks配置
2. ks-manager添加健康检查
3. 更新单机部署手册
2022-09-18 19:10:22 +08:00
_haoqi
c73a7eee2f 1. 调整docker-compose服务,容器名称 2022-09-16 20:03:58 +08:00
_haoqi
121f8468d5 1. 调整文件格式LF
2. 调整docker-compose服务,容器名称
2022-09-16 17:33:19 +08:00
haoqi
7b0b6936e0 1. 调整docker-compose.yml中容器名称 2022-09-16 15:54:34 +08:00
Peng
597ea04a96 Update README.md 2022-09-16 15:20:04 +08:00
Peng
f7f90aeaaa Update README.md 2022-09-16 15:18:29 +08:00
_haoqi
227479f695 1. 修改dockerfile
2. 删除无用配置文件
2022-09-16 15:13:18 +08:00
WYAOBO
6477fb3fe0 Merge branch 'didi:dev' into dev 2022-09-16 14:50:13 +08:00
wangdongfang-aden
4223f4f3c4 Merge pull request #596 from wangdongfang-aden/dev
helm update 3.0.0-beta.2
2022-09-16 14:45:43 +08:00
wangdongfang-aden
7288874d72 helm update 3.0.0-beta.2 2022-09-16 14:44:14 +08:00
wangdongfang-aden
68f76f2daf helm update 3.0.0-beta.2 2022-09-16 14:42:34 +08:00
wyb
fe6ddebc49 文档更新 2022-09-16 14:41:45 +08:00
wangdongfang-aden
12b5acd073 helm update 3.0.0-beta.2 2022-09-16 14:41:40 +08:00
wangdongfang-aden
a6f1fe07b3 helm update 3.0.0-beta.2 2022-09-16 14:41:02 +08:00
wangdongfang-aden
85e3f2a946 helm update 3.0.0-beta.2 2022-09-16 14:40:34 +08:00
pokemeng
d4f416de14 fix: adjust os judgment method with uname 2022-09-16 11:34:03 +08:00
haoqi
0d9a6702c1 1. 更改es初始化脚本输出追加为重定向 2022-09-15 17:13:58 +08:00
haoqi
d11285cdbf Merge branch 'master' into dev
# Conflicts:
#	km-dist/init/sql/ddl-logi-security.sql
2022-09-15 17:01:39 +08:00
EricZeng
5f1f33d2b9 Merge pull request #591 from didi/master
合并主分支
2022-09-15 16:59:11 +08:00
zengqiao
474daf752d bump version to 3.0.0-beta.3 2022-09-15 16:54:52 +08:00
haoqi
27d1b92690 1. 添加init容器,只用于初始化es索引 2022-09-15 16:22:51 +08:00
zengqiao
993afa4c19 默认用户名密码调整说明 2022-09-15 16:20:13 +08:00
EricZeng
028d891c32 Merge pull request #588 from didi/dev_v3.0.0-beta.2
合并v3.0.0 beta.2
2022-09-15 15:46:58 +08:00
zengqiao
0df55ec22d 更新3.0.0-beta.2升级手册 2022-09-15 15:23:29 +08:00
zengqiao
579f64774d 更新3.0.0-beta.2变更说明 2022-09-15 15:20:50 +08:00
haoqi
792f8d939d 1. 更改Dockerfile 2022-09-15 15:06:19 +08:00
EricZeng
e4fb02fcda Merge pull request #587 from didi/dev
合并开发分支
2022-09-15 14:35:00 +08:00
haoqi
0c14c641d0 1. 添加docker-compose部署方式
2. 更改manage服务初始化方式
3. 更改es初始化方式
2022-09-15 14:26:45 +08:00
EricZeng
dba671fd1e Merge pull request #586 from GraceWalk/dev
Dev
2022-09-15 13:49:04 +08:00
GraceWalk
80d1693722 fix: 修复单集群详情引导步骤定位错误的问题 2022-09-15 13:39:09 +08:00
GraceWalk
26014a11b2 feat: 补充前端打包构建部分文档说明 2022-09-15 13:36:09 +08:00
GraceWalk
848fddd55a fix: 切换依赖安装源为 taobao 镜像 2022-09-15 13:34:15 +08:00
EricZeng
97f5f05f1a Merge pull request #585 from didi/dev
更新单机部署文档
2022-09-15 13:05:14 +08:00
zengqiao
25b82810f2 更新单机部署文档 2022-09-15 13:01:33 +08:00
EricZeng
9b1e506fa7 Merge pull request #584 from didi/dev
修复日志表字段过短问题
2022-09-15 12:56:49 +08:00
zengqiao
7a42996e97 修复日志表字段过短问题 2022-09-15 12:55:06 +08:00
EricZeng
dbfcebcf67 Merge pull request #583 from didi/dev
合并开发分支
2022-09-15 12:38:11 +08:00
zengqiao
37c3f69a28 修复类型转化失败问题 2022-09-15 11:32:44 +08:00
zengqiao
5d412890b4 调整超时时间配置 2022-09-15 11:31:25 +08:00
zengqiao
1e318a4c40 修改默认的用户名密码 2022-09-15 11:31:03 +08:00
EricZeng
d4549176ec Merge pull request #566 from lomodays207/master
解决 java.lang.NumberFormatException: For input string: "{"value":0,"relation":"eq"}" 问题
2022-09-15 10:05:26 +08:00
haoqi
61efdf492f 添加docker-compose部署方式 2022-09-13 23:20:41 +08:00
lucasun
67ea4d44c8 Merge pull request #575 from GraceWalk/dev
同步前端代码
2022-09-13 15:13:02 +08:00
GraceWalk
fdae05a4aa fix: 登录页文案修改 2022-09-13 14:46:42 +08:00
GraceWalk
5efb837ee8 fix: 单集群详情样式优化 2022-09-13 14:46:29 +08:00
GraceWalk
584b626d93 fix: 修复 Broker Card 返回数据后依旧展示加载态的问题 2022-09-13 14:45:56 +08:00
GraceWalk
de25a4ed8e fix: 修复 Broker Card 返回数据后依旧展示加载态的问题 2022-09-13 14:45:27 +08:00
GraceWalk
2e852e5ca6 fix: 修复用户登出后回退还可以访问系统的问题 2022-09-13 14:44:18 +08:00
GraceWalk
b11000715a 修复 Topic Config 编辑表单不能正确回显当前值的问题 2022-09-13 14:43:35 +08:00
GraceWalk
b3f8b46f0f fix: 修复扩缩/迁移副本无法选中默认 Topic 的问题 & 迁移副本 Topic 迁移时间单位支持分钟粒度 2022-09-13 14:42:21 +08:00
GraceWalk
8d22a0664a fix: Broker 列表标识当前 Controller 2022-09-13 14:37:20 +08:00
GraceWalk
20756a3453 fix: 重置 Offset 部分 partationId 修改为 Select & Offset 数值限制 2022-09-13 14:35:23 +08:00
GraceWalk
c9b4d45a64 fix: 修复 Job 扩缩副本任务明细错误的问题 2022-09-13 14:31:45 +08:00
GraceWalk
83f7f5468b fix: 均衡历史列表样式重构 & 周期均衡场景化 & 立即均衡默认带入周期均衡参数 2022-09-13 14:30:03 +08:00
GraceWalk
59c042ad67 fix: Topic 列表趋势图优化 & 相关文案调整 2022-09-13 14:26:12 +08:00
GraceWalk
d550fc5068 fix: 修复 Consume 点击 Stop 后未停止请求发送的问题 2022-09-13 14:24:30 +08:00
GraceWalk
6effba69a0 feat: 补充 ReBalance 和 Topic 部分权限项 2022-09-13 14:22:50 +08:00
GraceWalk
9b46956259 fix: Topic 详情 Partition Tab 卡片模式展示优化 2022-09-13 14:18:17 +08:00
GraceWalk
b5a4a732da fix: 健康分设置问题修复 2022-09-13 14:15:15 +08:00
GraceWalk
487862367e feat: 多集群列表支持编辑 & 代码结构优化 2022-09-13 14:14:15 +08:00
GraceWalk
5b63b9ce67 feat: 左侧栏内容调整 2022-09-13 14:12:34 +08:00
GraceWalk
afbcd3e1df fix: Broker/Topic 图表详情 bugfix & 体验优化 2022-09-13 14:09:57 +08:00
GraceWalk
12b82c1395 fix: 图表展示 bugifx & 优化 2022-09-13 14:09:03 +08:00
GraceWalk
863b765e0d feat: 新增 RenderEmpty 组件 2022-09-13 14:04:55 +08:00
GraceWalk
731429c51c fix: 系统管理子应用补充返回 code 码拦截逻辑 2022-09-13 11:44:47 +08:00
GraceWalk
66f3bc61fe fix: 创建/编辑角色优化 2022-09-13 11:44:08 +08:00
GraceWalk
4efe35dd51 fix: 项目打包构建流程优化 & 补充说明 2022-09-13 11:43:30 +08:00
EricZeng
c92461ef93 Merge pull request #565 from didi/dev
合并开发分支
2022-09-12 05:53:34 +08:00
superspeedone
405e6e0c1d Topic消息查询支持Timestamp排序,接口支持按指定日期查询 2022-09-09 18:56:45 +08:00
superspeedone
0d227aef49 Topic消息查询支持Timestamp排序,接口支持按指定日期查询 2022-09-09 17:29:22 +08:00
superspeedone
0e49002f42 Topic消息查询支持Timestamp排序,接口支持按指定日期查询 2022-09-09 15:45:31 +08:00
wangdongfang-aden
2e016800e0 Merge pull request #568 from wangdongfang-aden/dev
使用3.0.0-beta.1镜像
2022-09-09 15:18:36 +08:00
wangdongfang-aden
09f317b991 使用3.0.0-beta.1镜像 2022-09-09 15:17:02 +08:00
wangdongfang-aden
5a48cb1547 使用3.0.0-beta.1镜像 2022-09-09 15:16:33 +08:00
wangdongfang-aden
f632febf33 Update Chart.yaml 2022-09-09 15:15:56 +08:00
wangdongfang-aden
3c53467943 使用3.0.0-beta.1镜像 2022-09-09 15:15:24 +08:00
qiubo
d358c0f4f7 修复ES total 查询转换异常问题 2022-09-09 10:22:26 +08:00
zengqiao
de977a5b32 加快添加集群后的信息获取的速度 2022-09-08 14:21:26 +08:00
zengqiao
703d685d59 Task任务分为metrics,common,metaddata三类,每一类任务的执行对应一个线程池,减少对Job模块线程池的依赖 2022-09-08 14:17:15 +08:00
zengqiao
31a5f17408 修复旧副本数为NULL的问题 2022-09-08 13:53:41 +08:00
zengqiao
c40ae3c455 增加副本变更任务结束后,进行优先副本选举的操作 2022-09-08 13:52:51 +08:00
zengqiao
b71a34279e 调整默认的权限 2022-09-08 13:50:08 +08:00
zengqiao
8f8c0c4eda 删除无效文件 2022-09-08 13:49:07 +08:00
zengqiao
3a384f0e34 优化重置Offset时的错误信息 2022-09-08 13:47:21 +08:00
zengqiao
cf7bc11cbd 增加登录系统对接文档 2022-09-08 13:46:45 +08:00
EricZeng
be60ae8399 Merge pull request #560 from didi/dev
合并开发分支
2022-09-07 14:20:04 +08:00
superspeedone
8e50d145d5 Topic消息查询支持Timestamp排序,支持查询最新消息或最早消息 #534 2022-09-07 11:17:59 +08:00
zengqiao
7a3d15525c 支持Ldap登录认证 2022-09-06 15:25:27 +08:00
zengqiao
64f32d8b24 bump logi-security version to 2.10.13 and logi-elasticsearch-client version to 1.0.24 2022-09-06 15:24:05 +08:00
zengqiao
949d6ba605 集群Broker列表,增加Controller角色信息 2022-09-06 15:22:57 +08:00
zengqiao
ceb8db09f4 优化查询Topic信息,Topic不存在时的错误提示 2022-09-06 15:21:53 +08:00
zengqiao
ed05a0ebb8 修复集群Group列表搜索反馈结果错误问题 2022-09-06 15:20:50 +08:00
zengqiao
a7cbb76655 修复Offset单位错误问题 2022-09-06 15:19:29 +08:00
zengqiao
93cbfa0b1f 后端增加页面权限点 2022-09-06 15:18:54 +08:00
zengqiao
6120613a98 Merge branch 'dev' of github.com:didi/KnowStreaming into dev 2022-09-06 15:15:14 +08:00
EricZeng
dbd00db159 Merge pull request #559 from didi/master
合并主分支
2022-09-06 15:14:18 +08:00
zengqiao
befde952f5 补充KS连接特定JMX IP的说明 2022-09-06 15:13:03 +08:00
zengqiao
1aa759e5be bump version to 3.0.0-beta.2 2022-09-06 10:25:14 +08:00
superspeedone
0f35427645 Merge branch 'dev' of https://github.com/superspeedone/KnowStreaming into dev 2022-09-05 15:41:08 +08:00
yanweiwen
fa7ad64140 Topic消息查询支持Timestamp排序,支持查询最新消息或最早消息 #534 2022-09-05 14:46:40 +08:00
264 changed files with 44984 additions and 3690 deletions

View File

@@ -13,7 +13,7 @@ Before sending pull request to this project, please read and follow guidelines b
Add device mode, API version, related log, screenshots and other related information in your pull request if possible. Add device mode, API version, related log, screenshots and other related information in your pull request if possible.
NOTE: We assume all your contribution can be licensed under the [Apache License 2.0](LICENSE). NOTE: We assume all your contribution can be licensed under the [AGPL-3.0](LICENSE).
## Issues ## Issues

View File

@@ -51,16 +51,16 @@
- 无需侵入改造 `Apache Kafka` ,一键便能纳管 `0.10.x` ~ `3.x.x` 众多版本的Kafka包括 `ZK``Raft` 运行模式的版本,同时在兼容架构上具备良好的扩展性,帮助您提升集群管理水平; - 无需侵入改造 `Apache Kafka` ,一键便能纳管 `0.10.x` ~ `3.x.x` 众多版本的Kafka包括 `ZK``Raft` 运行模式的版本,同时在兼容架构上具备良好的扩展性,帮助您提升集群管理水平;
- 🌪️ &nbsp;**零成本、界面化** - 🌪️ &nbsp;**零成本、界面化**
- 提炼高频 CLI 能力,设计合理的产品路径,提供清新美观的 GUI 界面,支持 Cluster、Broker、Topic、Group、Message、ACL 等组件 GUI 管理普通用户5分钟即可上手 - 提炼高频 CLI 能力,设计合理的产品路径,提供清新美观的 GUI 界面,支持 Cluster、Broker、Zookeeper、Topic、ConsumerGroup、Message、ACL、Connect 等组件 GUI 管理普通用户5分钟即可上手
- 👏 &nbsp;**云原生、插件化** - 👏 &nbsp;**云原生、插件化**
- 基于云原生构建,具备水平扩展能力,只需要增加节点即可获取更强的采集及对外服务能力,提供众多可热插拔的企业级特性,覆盖可观测性生态整合、资源治理、多活容灾等核心场景; - 基于云原生构建,具备水平扩展能力,只需要增加节点即可获取更强的采集及对外服务能力,提供众多可热插拔的企业级特性,覆盖可观测性生态整合、资源治理、多活容灾等核心场景;
- 🚀 &nbsp;**专业能力** - 🚀 &nbsp;**专业能力**
- 集群管理:支持集群一键纳管,健康分析、核心组件观测 等功能; - 集群管理:支持一键纳管,健康分析、核心组件观测 等功能;
- 观测提升:多维度指标观测大盘、观测指标最佳实践 等功能; - 观测提升:多维度指标观测大盘、观测指标最佳实践 等功能;
- 异常巡检:集群多维度健康巡检、集群多维度健康分 等功能; - 异常巡检:集群多维度健康巡检、集群多维度健康分 等功能;
- 能力增强Topic扩缩副本、Topic副本迁移 等功能; - 能力增强:集群负载均衡、Topic扩缩副本、Topic副本迁移 等功能;
&nbsp; &nbsp;
@@ -133,6 +133,8 @@ PS: 提问请尽量把问题一次性描述清楚,并告知环境信息情况
**`2、微信群`** **`2、微信群`**
微信加群:添加`mike_zhangliang``PenceXie`的微信号备注KnowStreaming加群。 微信加群:添加`mike_zhangliang``PenceXie`的微信号备注KnowStreaming加群。
<br/>
<img width="116" alt="wx" src="https://user-images.githubusercontent.com/71620349/192257217-c4ebc16c-3ad9-485d-a914-5911d3a4f46b.png">
## Star History ## Star History

View File

@@ -1,4 +1,120 @@
## v3.0.0
**Bug修复**
- 修复 Group 指标防重复采集不生效问题
- 修复自动创建 ES 索引模版失败问题
- 修复 Group+Topic 列表中存在已删除Topic的问题
- 修复使用 MySQL-8 ,因兼容问题, start_time 信息为 NULL 时,会导致创建任务失败的问题
- 修复 Group 信息表更新时,出现死锁的问题
- 修复图表补点逻辑与图表时间范围不适配的问题
**体验优化**
- 按照资源类别,拆分健康巡检任务
- 优化 Group 详情页的指标为实时获取
- 图表拖拽排序支持用户级存储
- 多集群列表 ZK 信息展示兼容无 ZK 情况
- Topic 详情消息预览支持复制功能
- 部分内容大数字支持千位分割符展示
**新增**
- 集群信息中,新增 Zookeeper 客户端配置字段
- 集群信息中,新增 Kafka 集群运行模式字段
- 新增 docker-compose 的部署方式
## v3.0.0-beta.3
**文档**
- FAQ 补充权限识别失败问题的说明
- 同步更新文档,保持与官网一致
**Bug修复**
- Offset 信息获取时,过滤掉无 Leader 的分区
- 升级 oshi-core 版本至 5.6.1 版本,修复 Windows 系统获取系统指标失败问题
- 修复 JMX 连接被关闭后,未进行重建的问题
- 修复因 DB 中 Broker 信息不存在导致 TotalLogSize 指标获取时抛空指针问题
- 修复 dml-logi.sql 中SQL 注释错误的问题
- 修复 startup.sh 中,识别操作系统类型错误的问题
- 修复配置管理页面删除配置失败的问题
- 修复系统管理应用文件引用路径
- 修复 Topic Messages 详情提示信息点击跳转 404 的问题
- 修复扩副本时,当前副本数不显示问题
**体验优化**
- Topic-Messages 页面增加返回数据的排序以及按照Earliest/Latest的获取方式
- 优化 GroupOffsetResetEnum 类名为 OffsetTypeEnum使得类名含义更准确
- 移动 KafkaZKDAO 类,及 Kafka Znode 实体类的位置,使得 Kafka Zookeeper DAO 更加内聚及便于识别
- 后端补充 Overview 页面指标排序的功能
- 前端 Webpack 配置优化
- Cluster Overview 图表取消放大展示功能
- 列表页增加手动刷新功能
- 接入/编辑集群,优化 JMX-PORTVersion 信息的回显优化JMX信息的展示
- 提高登录页面图片展示清晰度
- 部分样式和文案优化
---
## v3.0.0-beta.2
**文档**
- 新增登录系统对接文档
- 优化前端工程打包构建部分文档说明
- FAQ补充KnowStreaming连接特定JMX IP的说明
**Bug修复**
- 修复logi_security_oplog表字段过短导致删除Topic等操作无法记录的问题
- 修复ES查询时抛java.lang.NumberFormatException: For input string: "{"value":0,"relation":"eq"}" 问题
- 修复LogStartOffset和LogEndOffset指标单位错误问题
- 修复进行副本变更时旧副本数为NULL的问题
- 修复集群Group列表在第二页搜索时搜索时返回的分页信息错误问题
- 修复重置Offset时返回的错误信息提示不一致的问题
- 修复集群查看系统查看LoadRebalance等页面权限点缺失问题
- 修复查询不存在的Topic时错误信息提示不明显的问题
- 修复Windows用户打包前端工程报错的问题
- package-lock.json锁定前端依赖版本号修复因依赖自动升级导致打包失败等问题
- 系统管理子应用补充后端返回的Code码拦截解决后端接口返回报错不展示的问题
- 修复用户登出后,依旧可以访问系统的问题
- 修复巡检任务配置时,数值显示错误的问题
- 修复Broker/Topic Overview 图表和图表详情问题
- 修复Job扩缩副本任务明细数据错误的问题
- 修复重置Offset时分区IDOffset数值无限制问题
- 修复扩缩/迁移副本时无法选中Kafka系统Topic的问题
- 修复Topic的Config页面编辑表单时不能正确回显当前值的问题
- 修复Broker Card返回数据后依旧展示加载态的问题
**体验优化**
- 优化默认用户密码为 admin/admin
- 缩短新增集群后,集群信息加载的耗时
- 集群Broker列表增加Controller角色信息
- 副本变更任务结束后,增加进行优先副本选举的操作
- Task模块任务分为Metrics、Common、Metadata三类任务每类任务配备独立线程池减少对Job模块的线程池以及不同类任务之间的相互影响
- 删除代码中存在的多余无用文件
- 自动新增ES索引模版及近7天索引减少用户搭建时需要做的事项
- 优化前端工程打包流程
- 优化登录页文案页面左侧栏内容单集群详情样式Topic列表趋势图等
- 首次进入Broker/Topic图表详情时进行预缓存数据从而优化体验
- 优化Topic详情Partition Tab的展示
- 多集群列表页增加编辑功能
- 优化副本变更时,迁移时间支持分钟级别粒度
- logi-security版本升级至2.10.13
- logi-elasticsearch-client版本升级至1.0.24
**能力提升**
- 支持Ldap登录认证
---
## v3.0.0-beta.1 ## v3.0.0-beta.1
**文档** **文档**
@@ -35,6 +151,7 @@
- 增加周期任务用于主动创建缺少的ES模版及索引的能力减少额外的脚本操作 - 增加周期任务用于主动创建缺少的ES模版及索引的能力减少额外的脚本操作
- 增加JMX连接的Broker地址可选择的能力 - 增加JMX连接的Broker地址可选择的能力
---
## v3.0.0-beta.0 ## v3.0.0-beta.0

View File

@@ -9,7 +9,7 @@ error_exit ()
[ ! -e "$JAVA_HOME/bin/java" ] && unset JAVA_HOME [ ! -e "$JAVA_HOME/bin/java" ] && unset JAVA_HOME
if [ -z "$JAVA_HOME" ]; then if [ -z "$JAVA_HOME" ]; then
if $darwin; then if [ "Darwin" = "$(uname -s)" ]; then
if [ -x '/usr/libexec/java_home' ] ; then if [ -x '/usr/libexec/java_home' ] ; then
export JAVA_HOME=`/usr/libexec/java_home` export JAVA_HOME=`/usr/libexec/java_home`

View File

@@ -84,7 +84,7 @@ IDEA 更多具体的配置如下图所示:
`Know Streaming` 启动之后,可以访问一些信息,包括: `Know Streaming` 启动之后,可以访问一些信息,包括:
- 产品页面http://localhost:8080 ,默认账号密码:`admin` / `admin2022_` 进行登录。 - 产品页面http://localhost:8080 ,默认账号密码:`admin` / `admin2022_` 进行登录。`v3.0.0-beta.2`版本开始,默认账号密码为`admin` / `admin`
- 接口地址http://localhost:8080/swagger-ui.html 查看后端提供的相关接口。 - 接口地址http://localhost:8080/swagger-ui.html 查看后端提供的相关接口。
更多信息,详见:[KnowStreaming 官网](https://knowstreaming.com/) 更多信息,详见:[KnowStreaming 官网](https://knowstreaming.com/)

View File

@@ -0,0 +1,199 @@
![Logo](https://user-images.githubusercontent.com/71620349/185368586-aed82d30-1534-453d-86ff-ecfa9d0f35bd.png)
## 登录系统对接
[KnowStreaming](https://github.com/didi/KnowStreaming)以下简称KS 除了实现基于本地MySQL的用户登录认证方式外还已经实现了基于Ldap的登录认证。
但是登录认证系统并非仅此两种。因此为了具有更好的拓展性KS具有自定义登陆认证逻辑快速对接已有系统的特性。
在KS中我们将登陆认证相关的一些文件放在[km-extends](https://github.com/didi/KnowStreaming/tree/master/km-extends)模块下的[km-account](https://github.com/didi/KnowStreaming/tree/master/km-extends/km-account)模块里。
本文将介绍KS如何快速对接自有的用户登录认证系统。
### 对接步骤
- 创建一个登陆认证类,实现[LogiCommon](https://github.com/didi/LogiCommon)的LoginExtend接口
- 将[application.yml](https://github.com/didi/KnowStreaming/blob/master/km-rest/src/main/resources/application.yml)中的spring.logi-security.login-extend-bean-name字段改为登陆认证类的bean名称
```Java
//LoginExtend 接口
public interface LoginExtend {
/**
* 验证登录信息,同时记住登录状态
*/
UserBriefVO verifyLogin(AccountLoginDTO var1, HttpServletRequest var2, HttpServletResponse var3) throws LogiSecurityException;
/**
* 登出接口,清楚登录状态
*/
Result<Boolean> logout(HttpServletRequest var1, HttpServletResponse var2);
/**
* 检查是否已经登录
*/
boolean interceptorCheck(HttpServletRequest var1, HttpServletResponse var2, String var3, List<String> var4) throws IOException;
}
```
### 对接例子
我们以Ldap对接为例说明KS如何对接登录认证系统。
+ 编写[LdapLoginServiceImpl](https://github.com/didi/KnowStreaming/blob/master/km-extends/km-account/src/main/java/com/xiaojukeji/know/streaming/km/account/login/ldap/LdapLoginServiceImpl.java)类实现LoginExtend接口。
+ 设置[application.yml](https://github.com/didi/KnowStreaming/blob/master/km-rest/src/main/resources/application.yml)中的spring.logi-security.login-extend-bean-name=ksLdapLoginService。
完成上述两步即可实现KS对接Ldap认证登陆。
```Java
@Service("ksLdapLoginService")
public class LdapLoginServiceImpl implements LoginExtend {
@Override
public UserBriefVO verifyLogin(AccountLoginDTO loginDTO,
HttpServletRequest request,
HttpServletResponse response) throws LogiSecurityException {
String decodePasswd = AESUtils.decrypt(loginDTO.getPw());
// 去LDAP验证账密
LdapPrincipal ldapAttrsInfo = ldapAuthentication.authenticate(loginDTO.getUserName(), decodePasswd);
if (ldapAttrsInfo == null) {
// 用户不存在,正常来说上如果有问题,上一步会直接抛出异常
throw new LogiSecurityException(ResultCode.USER_NOT_EXISTS);
}
// 进行业务相关操作
// 记录登录状态Ldap因为无法记录登录状态因此有KnowStreaming进行记录
initLoginContext(request, response, loginDTO.getUserName(), user.getId());
return CopyBeanUtil.copy(user, UserBriefVO.class);
}
@Override
public Result<Boolean> logout(HttpServletRequest request, HttpServletResponse response) {
//清理cookie和session
return Result.buildSucc(Boolean.TRUE);
}
@Override
public boolean interceptorCheck(HttpServletRequest request, HttpServletResponse response, String requestMappingValue, List<String> whiteMappingValues) throws IOException {
// 检查是否已经登录
String userName = HttpRequestUtil.getOperator(request);
if (StringUtils.isEmpty(userName)) {
// 未登录,则进行登出
logout(request, response);
return Boolean.FALSE;
}
return Boolean.TRUE;
}
}
```
### 实现原理
因为登陆和登出整体实现逻辑是一致的,所以我们以登陆逻辑为例进行介绍。
+ 登陆原理
登陆走的是[LogiCommon](https://github.com/didi/LogiCommon)自带的LoginController。
```java
@RestController
public class LoginController {
//登陆接口
@PostMapping({"/login"})
public Result<UserBriefVO> login(HttpServletRequest request, HttpServletResponse response, @RequestBody AccountLoginDTO loginDTO) {
try {
//登陆认证
UserBriefVO userBriefVO = this.loginService.verifyLogin(loginDTO, request, response);
return Result.success(userBriefVO);
} catch (LogiSecurityException var5) {
return Result.fail(var5);
}
}
}
```
而登陆操作是调用LoginServiceImpl类来实现但是具体由哪个登陆认证类来执行登陆操作却由loginExtendBeanTool来指定。
```java
//LoginServiceImpl类
@Service
public class LoginServiceImpl implements LoginService {
//实现登陆操作但是具体哪个登陆类由loginExtendBeanTool来管理
public UserBriefVO verifyLogin(AccountLoginDTO loginDTO, HttpServletRequest request, HttpServletResponse response) throws LogiSecurityException {
return this.loginExtendBeanTool.getLoginExtendImpl().verifyLogin(loginDTO, request, response);
}
}
```
而loginExtendBeanTool类会优先去查找用户指定的登陆认证类如果失败则调用默认的登陆认证函数。
```java
//LoginExtendBeanTool类
@Component("logiSecurityLoginExtendBeanTool")
public class LoginExtendBeanTool {
public LoginExtend getLoginExtendImpl() {
LoginExtend loginExtend;
//先调用用户指定登陆类,如果失败则调用系统默认登陆认证
try {
//调用的类由spring.logi-security.login-extend-bean-name指定
loginExtend = this.getCustomLoginExtendImplBean();
} catch (UnsupportedOperationException var3) {
loginExtend = this.getDefaultLoginExtendImplBean();
}
return loginExtend;
}
}
```
+ 认证原理
认证的实现则比较简单向Spring中注册我们的拦截器PermissionInterceptor。
拦截器会调用LoginServiceImpl类的拦截方法LoginServiceImpl后续处理逻辑就和前面登陆是一致的。
```java
public class PermissionInterceptor implements HandlerInterceptor {
/**
* 拦截预处理
* @return boolean false:拦截, 不向下执行, true:放行
*/
@Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) throws Exception {
//免登录相关校验,如果验证通过,提前返回
//走拦截函数,进行普通用户验证
return loginService.interceptorCheck(request, response, classRequestMappingValue, whiteMappingValues);
}
}
```

View File

@@ -1,19 +1,14 @@
![Logo](https://user-images.githubusercontent.com/71620349/185368586-aed82d30-1534-453d-86ff-ecfa9d0f35bd.png) ![Logo](https://user-images.githubusercontent.com/71620349/185368586-aed82d30-1534-453d-86ff-ecfa9d0f35bd.png)
## JMX-连接失败问题解决 ## JMX-连接失败问题解决
- [JMX-连接失败问题解决](#jmx-连接失败问题解决) 集群正常接入`KnowStreaming`之后即可以看到集群的Broker列表此时如果查看不了Topic的实时流量或者是Broker的实时流量信息时那么大概率就是`JMX`连接的问题了。
- [1、问题&说明](#1问题说明)
- [2、解决方法](#2解决方法)
- [3、解决方法 —— 认证的JMX](#3解决方法--认证的jmx)
集群正常接入Logi-KafkaManager之后即可以看到集群的Broker列表此时如果查看不了Topic的实时流量或者是Broker的实时流量信息时那么大概率就是JMX连接的问题了。
下面我们按照步骤来一步一步的检查。 下面我们按照步骤来一步一步的检查。
### 1、问题&说明 ### 1、问题说明
**类型一JMX配置未开启** **类型一JMX配置未开启**
@@ -43,6 +38,26 @@ java.rmi.ConnectException: Connection refused to host: 192.168.0.1; nested excep
java.rmi.ConnectException: Connection refused to host: 127.0.0.1;; nested exception is: java.rmi.ConnectException: Connection refused to host: 127.0.0.1;; nested exception is:
``` ```
**类型三连接特定IP**
Broker 配置了内外网而JMX在配置时可能配置了内网IP或者外网IP此时 `KnowStreaming` 需要连接到特定网络的IP才可以进行访问。
比如:
Broker在ZK的存储结构如下所示我们期望连接到 `endpoints` 中标记为 `INTERNAL` 的地址,但是 `KnowStreaming` 却连接了 `EXTERNAL` 的地址,此时可以看 `4、解决方法 —— JMX连接特定网络` 进行解决。
```json
{
"listener_security_protocol_map": {"EXTERNAL":"SASL_PLAINTEXT","INTERNAL":"SASL_PLAINTEXT"},
"endpoints": ["EXTERNAL://192.168.0.1:7092","INTERNAL://192.168.0.2:7093"],
"jmx_port": 8099,
"host": "192.168.0.1",
"timestamp": "1627289710439",
"port": -1,
"version": 4
}
```
### 2、解决方法 ### 2、解决方法
这里仅介绍一下比较通用的解决方式,如若有更好的方式,欢迎大家指导告知一下。 这里仅介绍一下比较通用的解决方式,如若有更好的方式,欢迎大家指导告知一下。
@@ -76,26 +91,36 @@ fi
如果您是直接看的这个部分,建议先看一下上一节:`2、解决方法`以确保`JMX`的配置没有问题了。 如果您是直接看的这个部分,建议先看一下上一节:`2、解决方法`以确保`JMX`的配置没有问题了。
在JMX的配置等都没有问题的情况下如果是因为认证的原因导致连接不了的此时可以使用下面介绍的方法进行解决 `JMX`的配置等都没有问题的情况下,如果是因为认证的原因导致连接不了的,可以在集群接入界面配置你的`JMX`认证信息
**当前这块后端刚刚开发完成,可能还不够完善,有问题随时沟通。** <img src='http://img-ys011.didistatic.com/static/dc2img/do1_EUU352qMEX1Jdp7pxizp' width=350>
`Logi-KafkaManager 2.2.0+`之后的版本后端已经支持`JMX`认证方式的连接,但是还没有界面,此时我们可以往`cluster`表的`jmx_properties`字段写入`JMX`的认证信息。
这个数据是`json`格式的字符串,例子如下所示:
### 4、解决方法 —— JMX连接特定网络
可以手动往`ks_km_physical_cluster`表的`jmx_properties`字段增加一个`useWhichEndpoint`字段,从而控制 `KnowStreaming` 连接到特定的JMX IP及PORT。
`jmx_properties`格式:
```json ```json
{ {
"maxConn": 10, # KM对单台Broker的最大JMX连接数 "maxConn": 100, # KM对单台Broker的最大JMX连接数
"username": "xxxxx", # 用户名 "username": "xxxxx", # 用户名,可以不填写
"password": "xxxx", # 密码 "password": "xxxx", # 密码,可以不填写
"openSSL": true, # 开启SSL, true表示开启ssl, false表示关闭 "openSSL": true, # 开启SSL, true表示开启ssl, false表示关闭
"useWhichEndpoint": "EXTERNAL" #指定要连接的网络名称填写EXTERNAL就是连接endpoints里面的EXTERNAL地址
} }
``` ```
&nbsp; &nbsp;
SQL例子: SQL例子
```sql ```sql
UPDATE cluster SET jmx_properties='{ "maxConn": 10, "username": "xxxxx", "password": "xxxx", "openSSL": false }' where id={xxx}; UPDATE ks_km_physical_cluster SET jmx_properties='{ "maxConn": 10, "username": "xxxxx", "password": "xxxx", "openSSL": false , "useWhichEndpoint": "xxx"}' where id={xxx};
``` ```
注意:
+ 目前此功能只支持采用 `ZK` 做分布式协调的kafka集群。

View File

@@ -6,9 +6,10 @@
### 2.1.1、安装说明 ### 2.1.1、安装说明
-`v3.0.0-bete` 版本为例进行部署; -`v3.0.0-beta.1` 版本为例进行部署;
- 以 CentOS-7 为例,系统基础配置要求 4C-8G - 以 CentOS-7 为例,系统基础配置要求 4C-8G
- 部署完成后,可通过浏览器:`IP:PORT` 进行访问,默认端口是 `8080`,系统默认账号密码: `admin` / `admin2022_` - 部署完成后,可通过浏览器:`IP:PORT` 进行访问,默认端口是 `8080`,系统默认账号密码: `admin` / `admin2022_`
- `v3.0.0-beta.2`版本开始,默认账号密码为`admin` / `admin`
- 本文为单机部署,如需分布式部署,[请联系我们](https://knowstreaming.com/support-center) - 本文为单机部署,如需分布式部署,[请联系我们](https://knowstreaming.com/support-center)
**软件依赖** **软件依赖**
@@ -19,7 +20,7 @@
| ElasticSearch | v7.6+ | 8060 | | ElasticSearch | v7.6+ | 8060 |
| JDK | v8+ | - | | JDK | v8+ | - |
| CentOS | v6+ | - | | CentOS | v6+ | - |
| Ubantu | v16+ | - | | Ubuntu | v16+ | - |
&nbsp; &nbsp;
@@ -29,7 +30,7 @@
```bash ```bash
# 在服务器中下载安装脚本, 该脚本中会在当前目录下重新安装MySQL。重装后的mysql密码存放在当前目录的mysql.password文件中。 # 在服务器中下载安装脚本, 该脚本中会在当前目录下重新安装MySQL。重装后的mysql密码存放在当前目录的mysql.password文件中。
wget https://s3-gzpu.didistatic.com/pub/knowstreaming/deploy_KnowStreaming.sh wget https://s3-gzpu.didistatic.com/pub/knowstreaming/deploy_KnowStreaming-3.0.0-beta.1.sh
# 执行脚本 # 执行脚本
sh deploy_KnowStreaming.sh sh deploy_KnowStreaming.sh
@@ -42,10 +43,10 @@ sh deploy_KnowStreaming.sh
```bash ```bash
# 将安装包下载到本地且传输到目标服务器 # 将安装包下载到本地且传输到目标服务器
wget https://s3-gzpu.didistatic.com/pub/knowstreaming/KnowStreaming-3.0.0-betaoffline.tar.gz wget https://s3-gzpu.didistatic.com/pub/knowstreaming/KnowStreaming-3.0.0-beta.1-offline.tar.gz
# 解压安装包 # 解压安装包
tar -zxf KnowStreaming-3.0.0-betaoffline.tar.gz tar -zxf KnowStreaming-3.0.0-beta.1-offline.tar.gz
# 执行安装脚本 # 执行安装脚本
sh deploy_KnowStreaming-offline.sh sh deploy_KnowStreaming-offline.sh
@@ -58,28 +59,182 @@ sh deploy_KnowStreaming-offline.sh
### 2.1.3、容器部署 ### 2.1.3、容器部署
#### 2.1.3.1、Helm
**环境依赖** **环境依赖**
- Kubernetes >= 1.14 Helm >= 2.17.0 - Kubernetes >= 1.14 Helm >= 2.17.0
- 默认配置为全部安装 ElasticSearch + MySQL + KnowStreaming - 默认依赖全部安装ElasticSearch3 节点集群模式) + MySQL(单机) + KnowStreaming-manager + KnowStreaming-ui
- 如果使用已有的 ElasticSearch(7.6.x) 和 MySQL(5.7) 只需调整 values.yaml 部分参数即可 - 使用已有的 ElasticSearch(7.6.x) 和 MySQL(5.7) 只需调整 values.yaml 部分参数即可
**安装命令** **安装命令**
```bash ```bash
# 下载安装包 # 相关镜像在Docker Hub都可以下载
wget https://s3-gzpu.didistatic.com/pub/knowstreaming/knowstreaming-3.0.0-hlem.tgz # 快速安装(NAMESPACE需要更改为已存在的安装启动需要几分钟初始化请稍等~)
helm install -n [NAMESPACE] [NAME] http://download.knowstreaming.com/charts/knowstreaming-manager-0.1.5.tgz
# 解压安装包
tar -zxf knowstreaming-3.0.0-hlem.tgz
# 执行命令(NAMESPACE需要更改为已存在的)
helm install -n [NAMESPACE] knowstreaming knowstreaming-manager/
# 获取KnowStreaming前端ui的service. 默认nodeport方式. # 获取KnowStreaming前端ui的service. 默认nodeport方式.
# (http://nodeIP:nodeport默认用户名密码admin/admin2022_) # (http://nodeIP:nodeport默认用户名密码admin/admin2022_)
# `v3.0.0-beta.2`版本开始helm chart包版本0.1.4开始),默认账号密码为`admin` / `admin`
# 添加仓库
helm repo add knowstreaming http://download.knowstreaming.com/charts
# 拉取最新版本
helm pull knowstreaming/knowstreaming-manager
```
&nbsp;
#### 2.1.3.2、Docker Compose
**环境依赖**
- [Docker](https://docs.docker.com/engine/install/)
- [Docker Compose](https://docs.docker.com/compose/install/)
**安装命令**
```bash
# `v3.0.0-beta.2`版本开始(docker镜像为0.2.0版本开始),默认账号密码为`admin` / `admin`
# https://hub.docker.com/u/knowstreaming 在此处寻找最新镜像版本
# mysql与es可以使用自己搭建的服务,调整对应配置即可
# 复制docker-compose.yml到指定位置后执行下方命令即可启动
docker-compose up -d
```
**验证安装**
```shell
docker-compose ps
# 验证启动 - 状态为 UP 则表示成功
Name Command State Ports
----------------------------------------------------------------------------------------------------
elasticsearch-single /usr/local/bin/docker-entr ... Up 9200/tcp, 9300/tcp
knowstreaming-init /bin/bash /es_template_cre ... Up
knowstreaming-manager /bin/sh /ks-start.sh Up 80/tcp
knowstreaming-mysql /entrypoint.sh mysqld Up (health: starting) 3306/tcp, 33060/tcp
knowstreaming-ui /docker-entrypoint.sh ngin ... Up 0.0.0.0:80->80/tcp
# 稍等一分钟左右 knowstreaming-init 会退出表示es初始化完成可以访问页面
Name Command State Ports
-------------------------------------------------------------------------------------------
knowstreaming-init /bin/bash /es_template_cre ... Exit 0
knowstreaming-mysql /entrypoint.sh mysqld Up (healthy) 3306/tcp, 33060/tcp
```
**访问**
```http request
http://127.0.0.1:80/
```
**docker-compose.yml**
```yml
version: "2"
services:
# *不要调整knowstreaming-manager服务名称ui中会用到
knowstreaming-manager:
image: knowstreaming/knowstreaming-manager:latest
container_name: knowstreaming-manager
privileged: true
restart: always
depends_on:
- elasticsearch-single
- knowstreaming-mysql
expose:
- 80
command:
- /bin/sh
- /ks-start.sh
environment:
TZ: Asia/Shanghai
# mysql服务地址
SERVER_MYSQL_ADDRESS: knowstreaming-mysql:3306
# mysql数据库名
SERVER_MYSQL_DB: know_streaming
# mysql用户名
SERVER_MYSQL_USER: root
# mysql用户密码
SERVER_MYSQL_PASSWORD: admin2022_
# es服务地址
SERVER_ES_ADDRESS: elasticsearch-single:9200
# 服务JVM参数
JAVA_OPTS: -Xmx1g -Xms1g
# 对于kafka中ADVERTISED_LISTENERS填写的hostname可以通过该方式完成
# extra_hosts:
# - "hostname:x.x.x.x"
# 服务日志路径
# volumes:
# - /ks/manage/log:/logs
knowstreaming-ui:
image: knowstreaming/knowstreaming-ui:latest
container_name: knowstreaming-ui
restart: always
ports:
- '80:80'
environment:
TZ: Asia/Shanghai
depends_on:
- knowstreaming-manager
# extra_hosts:
# - "hostname:x.x.x.x"
elasticsearch-single:
image: docker.io/library/elasticsearch:7.6.2
container_name: elasticsearch-single
restart: always
expose:
- 9200
- 9300
# ports:
# - '9200:9200'
# - '9300:9300'
environment:
TZ: Asia/Shanghai
# es的JVM参数
ES_JAVA_OPTS: -Xms512m -Xmx512m
# 单节点配置,多节点集群参考 https://www.elastic.co/guide/en/elasticsearch/reference/7.6/docker.html#docker-compose-file
discovery.type: single-node
# 数据持久化路径
# volumes:
# - /ks/es/data:/usr/share/elasticsearch/data
# es初始化服务与manager使用同一镜像
# 首次启动es需初始化模版和索引,后续会自动创建
knowstreaming-init:
image: knowstreaming/knowstreaming-manager:latest
container_name: knowstreaming-init
depends_on:
- elasticsearch-single
command:
- /bin/bash
- /es_template_create.sh
environment:
TZ: Asia/Shanghai
# es服务地址
SERVER_ES_ADDRESS: elasticsearch-single:9200
knowstreaming-mysql:
image: knowstreaming/knowstreaming-mysql:latest
container_name: knowstreaming-mysql
restart: always
environment:
TZ: Asia/Shanghai
# root 用户密码
MYSQL_ROOT_PASSWORD: admin2022_
# 初始化时创建的数据库名称
MYSQL_DATABASE: know_streaming
# 通配所有host,可以访问远程
MYSQL_ROOT_HOST: '%'
expose:
- 3306
# ports:
# - '3306:3306'
# 数据持久化路径
# volumes:
# - /ks/mysql/data:/data/mysql
``` ```
&nbsp; &nbsp;
@@ -219,10 +374,10 @@ sh /data/elasticsearch/control.sh status
```bash ```bash
# 下载安装包 # 下载安装包
wget https://s3-gzpu.didistatic.com/pub/knowstreaming/KnowStreaming-3.0.0-beta.tar.gz wget https://s3-gzpu.didistatic.com/pub/knowstreaming/KnowStreaming-3.0.0-beta.1.tar.gz
# 解压安装包到指定目录 # 解压安装包到指定目录
tar -zxf KnowStreaming-3.0.0-beta.tar.gz -C /data/ tar -zxf KnowStreaming-3.0.0-beta.1.tar.gz -C /data/
# 修改启动脚本并加入systemd管理 # 修改启动脚本并加入systemd管理
cd /data/KnowStreaming/ cd /data/KnowStreaming/
@@ -236,7 +391,7 @@ mysql -uroot -pDidi_km_678 know_streaming < ./init/sql/dml-ks-km.sql
mysql -uroot -pDidi_km_678 know_streaming < ./init/sql/dml-logi.sql mysql -uroot -pDidi_km_678 know_streaming < ./init/sql/dml-logi.sql
# 创建elasticsearch初始化数据 # 创建elasticsearch初始化数据
sh ./init/template/template.sh sh ./bin/init_es_template.sh
# 修改配置文件 # 修改配置文件
vim ./conf/application.yml vim ./conf/application.yml

View File

@@ -1,7 +1,5 @@
![Logo](https://user-images.githubusercontent.com/71620349/185368586-aed82d30-1534-453d-86ff-ecfa9d0f35bd.png) ![Logo](https://user-images.githubusercontent.com/71620349/185368586-aed82d30-1534-453d-86ff-ecfa9d0f35bd.png)
# `Know Streaming` 源码编译打包手册 # `Know Streaming` 源码编译打包手册
## 1、环境信息 ## 1、环境信息
@@ -25,27 +23,23 @@
具体见下面描述。 具体见下面描述。
### 2.1、前后端合并打包 ### 2.1、前后端合并打包
1. 下载源码; 1. 下载源码;
2. 进入 `KS-KM` 工程目录,执行 `mvn -Prelease-package -Dmaven.test.skip=true clean install -U` 命令; 2. 进入 `KS-KM` 工程目录,执行 `mvn -Prelease-package -Dmaven.test.skip=true clean install -U` 命令;
3. 打包命令执行完成后,会在 `km-dist/target` 目录下面生成一个 `KnowStreaming-*.tar.gz` 的安装包。 3. 打包命令执行完成后,会在 `km-dist/target` 目录下面生成一个 `KnowStreaming-*.tar.gz` 的安装包。
### 2.2、前端单独打包 ### 2.2、前端单独打包
1. 下载源码; 1. 下载源码;
2. 进入 `KS-KM/km-console` 工程目录; 2. 跳转到 [前端打包构建文档](https://github.com/didi/KnowStreaming/blob/master/km-console/README.md) 按步骤进行。打包成功后,会在 `km-rest/src/main/resources` 目录下生成名为 `templates` 的前端静态资源包;
3. 执行 `npm run build`命令,会在 `KS-KM/km-console` 目录下生成一个名为 `pub` 的前端静态资源包; 3. 如果上一步过程中报错,请查看 [FAQ](https://github.com/didi/KnowStreaming/blob/master/docs/user_guide/faq.md) 第 8.10 条;
### 2.3、后端单独打包 ### 2.3、后端单独打包
1. 下载源码; 1. 下载源码;
2. 修改顶层 `pom.xml` ,去掉其中的 `km-console` 模块,如下所示; 2. 修改顶层 `pom.xml` ,去掉其中的 `km-console` 模块,如下所示;
```xml ```xml
<modules> <modules>
<!-- <module>km-console</module>--> <!-- <module>km-console</module>-->
@@ -62,10 +56,7 @@
<module>km-rest</module> <module>km-rest</module>
<module>km-dist</module> <module>km-dist</module>
</modules> </modules>
``` ```
3. 执行 `mvn -U clean package -Dmaven.test.skip=true`命令; 3. 执行 `mvn -U clean package -Dmaven.test.skip=true`命令;
4. 执行完成之后会在 `KS-KM/km-rest/target` 目录下面生成一个 `ks-km.jar` 即为KS的后端部署的Jar包也可以执行 `mvn -Prelease-package -Dmaven.test.skip=true clean install -U` 生成的tar包也仅有后端服务的功能 4. 执行完成之后会在 `KS-KM/km-rest/target` 目录下面生成一个 `ks-km.jar` 即为 KS 的后端部署的 Jar 包,也可以执行 `mvn -Prelease-package -Dmaven.test.skip=true clean install -U` 生成的 tar 包也仅有后端服务的功能;

View File

@@ -1,21 +1,105 @@
## 6.2、版本升级手册 ## 6.2、版本升级手册
注意:如果想升级至具体版本,需要将你当前版本至你期望使用版本的变更统统执行一遍,然后才能正常使用。 注意:
- 如果想升级至具体版本,需要将你当前版本至你期望使用版本的变更统统执行一遍,然后才能正常使用。
- 如果中间某个版本没有升级信息,则表示该版本直接替换安装包即可从前一个版本升级至当前版本。
### 6.2.0、升级至 `master` 版本 ### 6.2.0、升级至 `master` 版本
暂无 暂无
### 6.2.1、升级至 `v3.0.0` 版本
**SQL 变更**
```sql
ALTER TABLE `ks_km_physical_cluster`
ADD COLUMN `zk_properties` TEXT NULL COMMENT 'ZK配置' AFTER `jmx_properties`;
```
--- ---
### 6.2.1、升级至 `v3.0.0-beta.1`版本
### 6.2.2、升级至 `v3.0.0-beta.2`版本
**配置变更**
```yaml
# 新增配置
spring:
logi-security: # know-streaming 依赖的 logi-security 模块的数据库的配置,默认与 know-streaming 的数据库配置保持一致即可
login-extend-bean-name: logiSecurityDefaultLoginExtendImpl # 使用的登录系统Service的Bean名称无需修改
# 线程池大小相关配置在task模块中新增了三类线程池
# 从而减少不同类型任务之间的相互影响以及减少对logi-job内的线程池的影响
thread-pool:
task: # 任务模块的配置
metrics: # metrics采集任务配置
thread-num: 18 # metrics采集任务线程池核心线程数
queue-size: 180 # metrics采集任务线程池队列大小
metadata: # metadata同步任务配置
thread-num: 27 # metadata同步任务线程池核心线程数
queue-size: 270 # metadata同步任务线程池队列大小
common: # 剩余其他任务配置
thread-num: 15 # 剩余其他任务线程池核心线程数
queue-size: 150 # 剩余其他任务线程池队列大小
# 删除配置,下列配置将不再使用
thread-pool:
task: # 任务模块的配置
heaven: # 采集任务配置
thread-num: 20 # 采集任务线程池核心线程数
queue-size: 1000 # 采集任务线程池队列大小
```
**SQL 变更**
```sql
-- 多集群管理权限2022-09-06新增
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('2000', '多集群管理查看', '1593', '1', '2', '多集群管理查看', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('2002', 'Topic-迁移副本', '1593', '1', '2', 'Topic-迁移副本', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('2004', 'Topic-扩缩副本', '1593', '1', '2', 'Topic-扩缩副本', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('2006', 'Cluster-LoadReBalance-周期均衡', '1593', '1', '2', 'Cluster-LoadReBalance-周期均衡', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('2008', 'Cluster-LoadReBalance-立即均衡', '1593', '1', '2', 'Cluster-LoadReBalance-立即均衡', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('2010', 'Cluster-LoadReBalance-设置集群规格', '1593', '1', '2', 'Cluster-LoadReBalance-设置集群规格', '0', 'know-streaming');
**SQL变更** -- 系统管理权限2022-09-06新增
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('3000', '系统管理查看', '1595', '1', '2', '系统管理查看', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '2000', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '2002', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '2004', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '2006', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '2008', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '2010', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '3000', '0', 'know-streaming');
-- 修改字段长度
ALTER TABLE `logi_security_oplog`
CHANGE COLUMN `operator_ip` `operator_ip` VARCHAR(64) NOT NULL COMMENT '操作者ip' ,
CHANGE COLUMN `operator` `operator` VARCHAR(64) NULL DEFAULT NULL COMMENT '操作者账号' ,
CHANGE COLUMN `operate_page` `operate_page` VARCHAR(64) NOT NULL DEFAULT '' COMMENT '操作页面' ,
CHANGE COLUMN `operate_type` `operate_type` VARCHAR(64) NOT NULL COMMENT '操作类型' ,
CHANGE COLUMN `target_type` `target_type` VARCHAR(64) NOT NULL COMMENT '对象分类' ,
CHANGE COLUMN `target` `target` VARCHAR(1024) NOT NULL COMMENT '操作对象' ,
CHANGE COLUMN `operation_methods` `operation_methods` VARCHAR(64) NOT NULL DEFAULT '' COMMENT '操作方式' ;
```
---
### 6.2.3、升级至 `v3.0.0-beta.1`版本
**SQL 变更**
1、在`ks_km_broker`表增加了一个监听信息字段。 1、在`ks_km_broker`表增加了一个监听信息字段。
2、为`logi_security_oplog`表operation_methods字段设置默认值''。 2、为`logi_security_oplog` operation_methods 字段设置默认值''。
因此需要执行下面的sql对数据库表进行更新。 因此需要执行下面的 sql 对数据库表进行更新。
```sql ```sql
ALTER TABLE `ks_km_broker` ALTER TABLE `ks_km_broker`
@@ -28,8 +112,7 @@ ALTER COLUMN `operation_methods` set default '';
--- ---
### 6.2.4、`2.x`版本 升级至 `v3.0.0-beta.0`版本
### 6.2.2、`2.x`版本 升级至 `v3.0.0-beta.0`版本
**升级步骤:** **升级步骤:**
@@ -53,14 +136,14 @@ ALTER COLUMN `operation_methods` set default '';
UPDATE ks_km_topic UPDATE ks_km_topic
INNER JOIN INNER JOIN
(SELECT (SELECT
topic.cluster_id AS cluster_id, topic.cluster_id AS cluster_id,
topic.topic_name AS topic_name, topic.topic_name AS topic_name,
topic.description AS description topic.description AS description
FROM topic WHERE description != '' FROM topic WHERE description != ''
) AS t ) AS t
ON ks_km_topic.cluster_phy_id = t.cluster_id ON ks_km_topic.cluster_phy_id = t.cluster_id
AND ks_km_topic.topic_name = t.topic_name AND ks_km_topic.topic_name = t.topic_name
AND ks_km_topic.id > 0 AND ks_km_topic.id > 0
SET ks_km_topic.description = t.description; SET ks_km_topic.description = t.description;
``` ```

View File

@@ -1,4 +1,3 @@
# FAQ # FAQ
## 8.1、支持哪些 Kafka 版本? ## 8.1、支持哪些 Kafka 版本?
@@ -110,20 +109,76 @@ SECURITY.TRICK_USERS
但是还有一点需要注意,绕过的用户仅能调用他有权限的接口,比如一个普通用户,那么他就只能调用普通的接口,不能去调用运维人员的接口。 但是还有一点需要注意,绕过的用户仅能调用他有权限的接口,比如一个普通用户,那么他就只能调用普通的接口,不能去调用运维人员的接口。
## 8.8、Specified key was too long; max key length is 767 bytes ## 8.8、Specified key was too long; max key length is 767 bytes
**原因:**不同版本的InoDB引擎参数innodb_large_prefix默认值不同即在5.6默认值为OFF5.7默认值为ON。 **原因:** 不同版本的 InoDB 引擎参数innodb_large_prefix默认值不同即在 5.6 默认值为 OFF5.7 默认值为 ON。
对于引擎为InnoDBinnodb_large_prefix=OFF且行格式为Antelope即支持REDUNDANTCOMPACT时索引键前缀长度最大为 767 字节。innodb_large_prefix=ON且行格式为Barracuda即支持DYNAMICCOMPRESSED时索引键前缀长度最大为3072字节。 对于引擎为 InnoDBinnodb_large_prefix=OFF且行格式为 Antelope 即支持 REDUNDANTCOMPACT 时,索引键前缀长度最大为 767 字节。innodb_large_prefix=ON且行格式为 Barracuda 即支持 DYNAMICCOMPRESSED 时,索引键前缀长度最大为 3072 字节。
**解决方案:** **解决方案:**
- 减少varchar字符大小低于767/4=191。 - 减少 varchar 字符大小低于 767/4=191。
- 将字符集改为latin1一个字符=一个字节)。 - 将字符集改为 latin1一个字符=一个字节)。
- 开启innodb_large_prefix修改默认行格式innodb_file_format为Barracuda并设置row_format=dynamic。 - 开启innodb_large_prefix修改默认行格式innodb_file_format Barracuda并设置 row_format=dynamic。
## 8.9、出现ESIndexNotFoundEXception报错 ## 8.9、出现 ESIndexNotFoundEXception 报错
**原因 **没有创建ES索引模版 **原因 **没有创建 ES 索引模版
**解决方案:**执行init_es_template.sh脚本创建ES索引模版即可。 **解决方案:**执行 init_es_template.sh 脚本,创建 ES 索引模版即可。
## 8.10、km-console 打包构建失败
首先,**请确保您正在使用最新版本**,版本列表见 [Tags](https://github.com/didi/KnowStreaming/tags)。如果不是最新版本,请升级后再尝试有无问题。
常见的原因是由于工程依赖没有正常安装,导致在打包过程中缺少依赖,造成打包失败。您可以检查是否有以下文件夹,且文件夹内是否有内容
```
KnowStreaming/km-console/node_modules
KnowStreaming/km-console/packages/layout-clusters-fe/node_modules
KnowStreaming/km-console/packages/config-manager-fe/node_modules
```
如果发现没有对应的 `node_modules` 目录或着目录内容为空,说明依赖没有安装成功。请按以下步骤操作,
1. 手动删除上述三个文件夹(如果有)
2. 如果之前是通过 `mvn install` 打包 `km-console`请到项目根目录KnowStreaming下重新输入该指令进行打包。观察打包过程有无报错。如有报错请见步骤 4。
3. 如果是通过本地独立构建前端工程的方式(指直接执行 `npm run build`),请进入 `KnowStreaming/km-console` 目录,执行下述步骤(注意:执行时请确保您在使用 `node v12` 版本)
a. 执行 `npm run i`。如有报错,请见步骤 4。
b. 执行 `npm run build`。如有报错,请见步骤 4。
4. 麻烦联系我们协助解决。推荐提供以下信息,方面我们快速定位问题,示例如下。
```
操作系统: Mac
命令行终端bash
Node 版本: v12.22.12
复现步骤: 1. -> 2.
错误截图:
```
## 8.11、在 `km-console` 目录下执行 `npm run start` 时看不到应用构建和热加载过程?如何启动单个应用?
需要到具体的应用中执行 `npm run start`,例如 `cd packages/layout-clusters-fe` 后,执行 `npm run start`
应用启动后需要到基座应用中查看(需要启动基座应用,即 layout-clusters-fe
## 8.12、权限识别失败问题
1、使用admin账号登陆KnowStreaming时点击系统管理-用户管理-角色管理-新增角色,查看页面是否正常。
<img src="http://img-ys011.didistatic.com/static/dc2img/do1_gwGfjN9N92UxzHU8dfzr" width = "400" >
2、查看'/logi-security/api/v1/permission/tree'接口返回值,出现如下图所示乱码现象。
![接口返回值](http://img-ys011.didistatic.com/static/dc2img/do1_jTxBkwNGU9vZuYQQbdNw)
3、查看logi_security_permission表看看是否出现了中文乱码现象。
根据以上几点,我们可以确定是由于数据库乱码造成的权限识别失败问题。
+ 原因:由于数据库编码和我们提供的脚本不一致,数据库里的数据发生了乱码,因此出现权限识别失败问题。
+ 解决方案清空数据库数据将数据库字符集调整为utf8最后重新执行[dml-logi.sql](https://github.com/didi/KnowStreaming/blob/master/km-dist/init/sql/dml-logi.sql)脚本导入数据即可。

View File

@@ -11,7 +11,7 @@
下面是用户第一次使用我们产品的典型体验路径: 下面是用户第一次使用我们产品的典型体验路径:
![text](http://img-ys011.didistatic.com/static/dc2img/do1_YehqxqmsVaqU5gf3XphI) ![text](http://img-ys011.didistatic.com/static/dc2img/do1_qgqPsAY46sZeBaPUCwXY)
## 5.3、常用功能 ## 5.3、常用功能

View File

@@ -14,6 +14,7 @@ import com.xiaojukeji.know.streaming.km.common.bean.entity.topic.Topic;
import com.xiaojukeji.know.streaming.km.common.bean.vo.cluster.res.ClusterBrokersOverviewVO; import com.xiaojukeji.know.streaming.km.common.bean.vo.cluster.res.ClusterBrokersOverviewVO;
import com.xiaojukeji.know.streaming.km.common.bean.vo.cluster.res.ClusterBrokersStateVO; import com.xiaojukeji.know.streaming.km.common.bean.vo.cluster.res.ClusterBrokersStateVO;
import com.xiaojukeji.know.streaming.km.common.bean.vo.kafkacontroller.KafkaControllerVO; import com.xiaojukeji.know.streaming.km.common.bean.vo.kafkacontroller.KafkaControllerVO;
import com.xiaojukeji.know.streaming.km.common.constant.KafkaConstant;
import com.xiaojukeji.know.streaming.km.common.enums.SortTypeEnum; import com.xiaojukeji.know.streaming.km.common.enums.SortTypeEnum;
import com.xiaojukeji.know.streaming.km.common.utils.PaginationMetricsUtil; import com.xiaojukeji.know.streaming.km.common.utils.PaginationMetricsUtil;
import com.xiaojukeji.know.streaming.km.common.utils.PaginationUtil; import com.xiaojukeji.know.streaming.km.common.utils.PaginationUtil;
@@ -71,6 +72,9 @@ public class ClusterBrokersManagerImpl implements ClusterBrokersManager {
Topic groupTopic = topicService.getTopic(clusterPhyId, org.apache.kafka.common.internals.Topic.GROUP_METADATA_TOPIC_NAME); Topic groupTopic = topicService.getTopic(clusterPhyId, org.apache.kafka.common.internals.Topic.GROUP_METADATA_TOPIC_NAME);
Topic transactionTopic = topicService.getTopic(clusterPhyId, org.apache.kafka.common.internals.Topic.TRANSACTION_STATE_TOPIC_NAME); Topic transactionTopic = topicService.getTopic(clusterPhyId, org.apache.kafka.common.internals.Topic.TRANSACTION_STATE_TOPIC_NAME);
//获取controller信息
KafkaController kafkaController = kafkaControllerService.getKafkaControllerFromDB(clusterPhyId);
// 格式转换 // 格式转换
return PaginationResult.buildSuc( return PaginationResult.buildSuc(
this.convert2ClusterBrokersOverviewVOList( this.convert2ClusterBrokersOverviewVOList(
@@ -78,7 +82,8 @@ public class ClusterBrokersManagerImpl implements ClusterBrokersManager {
brokerList, brokerList,
metricsResult.getData(), metricsResult.getData(),
groupTopic, groupTopic,
transactionTopic transactionTopic,
kafkaController
), ),
paginationResult paginationResult
); );
@@ -159,7 +164,8 @@ public class ClusterBrokersManagerImpl implements ClusterBrokersManager {
List<Broker> brokerList, List<Broker> brokerList,
List<BrokerMetrics> metricsList, List<BrokerMetrics> metricsList,
Topic groupTopic, Topic groupTopic,
Topic transactionTopic) { Topic transactionTopic,
KafkaController kafkaController) {
Map<Integer, BrokerMetrics> metricsMap = metricsList == null? new HashMap<>(): metricsList.stream().collect(Collectors.toMap(BrokerMetrics::getBrokerId, Function.identity())); Map<Integer, BrokerMetrics> metricsMap = metricsList == null? new HashMap<>(): metricsList.stream().collect(Collectors.toMap(BrokerMetrics::getBrokerId, Function.identity()));
Map<Integer, Broker> brokerMap = brokerList == null? new HashMap<>(): brokerList.stream().collect(Collectors.toMap(Broker::getBrokerId, Function.identity())); Map<Integer, Broker> brokerMap = brokerList == null? new HashMap<>(): brokerList.stream().collect(Collectors.toMap(Broker::getBrokerId, Function.identity()));
@@ -169,12 +175,12 @@ public class ClusterBrokersManagerImpl implements ClusterBrokersManager {
Broker broker = brokerMap.get(brokerId); Broker broker = brokerMap.get(brokerId);
BrokerMetrics brokerMetrics = metricsMap.get(brokerId); BrokerMetrics brokerMetrics = metricsMap.get(brokerId);
voList.add(this.convert2ClusterBrokersOverviewVO(brokerId, broker, brokerMetrics, groupTopic, transactionTopic)); voList.add(this.convert2ClusterBrokersOverviewVO(brokerId, broker, brokerMetrics, groupTopic, transactionTopic, kafkaController));
} }
return voList; return voList;
} }
private ClusterBrokersOverviewVO convert2ClusterBrokersOverviewVO(Integer brokerId, Broker broker, BrokerMetrics brokerMetrics, Topic groupTopic, Topic transactionTopic) { private ClusterBrokersOverviewVO convert2ClusterBrokersOverviewVO(Integer brokerId, Broker broker, BrokerMetrics brokerMetrics, Topic groupTopic, Topic transactionTopic, KafkaController kafkaController) {
ClusterBrokersOverviewVO clusterBrokersOverviewVO = new ClusterBrokersOverviewVO(); ClusterBrokersOverviewVO clusterBrokersOverviewVO = new ClusterBrokersOverviewVO();
clusterBrokersOverviewVO.setBrokerId(brokerId); clusterBrokersOverviewVO.setBrokerId(brokerId);
if (broker != null) { if (broker != null) {
@@ -192,6 +198,9 @@ public class ClusterBrokersManagerImpl implements ClusterBrokersManager {
if (transactionTopic != null && transactionTopic.getBrokerIdSet().contains(brokerId)) { if (transactionTopic != null && transactionTopic.getBrokerIdSet().contains(brokerId)) {
clusterBrokersOverviewVO.getKafkaRoleList().add(transactionTopic.getTopicName()); clusterBrokersOverviewVO.getKafkaRoleList().add(transactionTopic.getTopicName());
} }
if (kafkaController != null && kafkaController.getBrokerId().equals(brokerId)) {
clusterBrokersOverviewVO.getKafkaRoleList().add(KafkaConstant.CONTROLLER_ROLE);
}
clusterBrokersOverviewVO.setLatestMetrics(brokerMetrics); clusterBrokersOverviewVO.setLatestMetrics(brokerMetrics);
return clusterBrokersOverviewVO; return clusterBrokersOverviewVO;

View File

@@ -19,7 +19,8 @@ import com.xiaojukeji.know.streaming.km.common.bean.vo.group.GroupTopicConsumedD
import com.xiaojukeji.know.streaming.km.common.bean.vo.group.GroupTopicOverviewVO; import com.xiaojukeji.know.streaming.km.common.bean.vo.group.GroupTopicOverviewVO;
import com.xiaojukeji.know.streaming.km.common.constant.MsgConstant; import com.xiaojukeji.know.streaming.km.common.constant.MsgConstant;
import com.xiaojukeji.know.streaming.km.common.enums.AggTypeEnum; import com.xiaojukeji.know.streaming.km.common.enums.AggTypeEnum;
import com.xiaojukeji.know.streaming.km.common.enums.GroupOffsetResetEnum; import com.xiaojukeji.know.streaming.km.common.enums.OffsetTypeEnum;
import com.xiaojukeji.know.streaming.km.common.enums.group.GroupStateEnum;
import com.xiaojukeji.know.streaming.km.common.exception.AdminOperateException; import com.xiaojukeji.know.streaming.km.common.exception.AdminOperateException;
import com.xiaojukeji.know.streaming.km.common.exception.NotExistException; import com.xiaojukeji.know.streaming.km.common.exception.NotExistException;
import com.xiaojukeji.know.streaming.km.common.utils.ConvertUtil; import com.xiaojukeji.know.streaming.km.common.utils.ConvertUtil;
@@ -75,7 +76,7 @@ public class GroupManagerImpl implements GroupManager {
} }
if (!paginationResult.hasData()) { if (!paginationResult.hasData()) {
return PaginationResult.buildSuc(dto); return PaginationResult.buildSuc(new ArrayList<>(), paginationResult);
} }
// 获取指标 // 获取指标
@@ -171,7 +172,7 @@ public class GroupManagerImpl implements GroupManager {
} }
if (!ConsumerGroupState.EMPTY.equals(description.state()) && !ConsumerGroupState.DEAD.equals(description.state())) { if (!ConsumerGroupState.EMPTY.equals(description.state()) && !ConsumerGroupState.DEAD.equals(description.state())) {
return Result.buildFromRSAndMsg(ResultStatus.KAFKA_OPERATE_FAILED, String.format("group处于%s, 重置失败(仅Empty情况可重置)", description.state().name())); return Result.buildFromRSAndMsg(ResultStatus.KAFKA_OPERATE_FAILED, String.format("group处于%s, 重置失败(仅Empty情况可重置)", GroupStateEnum.getByRawState(description.state()).getState()));
} }
// 获取offset // 获取offset
@@ -198,12 +199,12 @@ public class GroupManagerImpl implements GroupManager {
return Result.buildFromRSAndMsg(ResultStatus.NOT_EXIST, MsgConstant.getTopicNotExist(dto.getClusterId(), dto.getTopicName())); return Result.buildFromRSAndMsg(ResultStatus.NOT_EXIST, MsgConstant.getTopicNotExist(dto.getClusterId(), dto.getTopicName()));
} }
if (GroupOffsetResetEnum.PRECISE_OFFSET.getResetType() == dto.getResetType() if (OffsetTypeEnum.PRECISE_OFFSET.getResetType() == dto.getResetType()
&& ValidateUtils.isEmptyList(dto.getOffsetList())) { && ValidateUtils.isEmptyList(dto.getOffsetList())) {
return Result.buildFromRSAndMsg(ResultStatus.PARAM_ILLEGAL, "参数错误指定offset重置需传offset信息"); return Result.buildFromRSAndMsg(ResultStatus.PARAM_ILLEGAL, "参数错误指定offset重置需传offset信息");
} }
if (GroupOffsetResetEnum.PRECISE_TIMESTAMP.getResetType() == dto.getResetType() if (OffsetTypeEnum.PRECISE_TIMESTAMP.getResetType() == dto.getResetType()
&& ValidateUtils.isNull(dto.getTimestamp())) { && ValidateUtils.isNull(dto.getTimestamp())) {
return Result.buildFromRSAndMsg(ResultStatus.PARAM_ILLEGAL, "参数错误,指定时间重置需传时间信息"); return Result.buildFromRSAndMsg(ResultStatus.PARAM_ILLEGAL, "参数错误,指定时间重置需传时间信息");
} }
@@ -212,7 +213,7 @@ public class GroupManagerImpl implements GroupManager {
} }
private Result<Map<TopicPartition, Long>> getPartitionOffset(GroupOffsetResetDTO dto) { private Result<Map<TopicPartition, Long>> getPartitionOffset(GroupOffsetResetDTO dto) {
if (GroupOffsetResetEnum.PRECISE_OFFSET.getResetType() == dto.getResetType()) { if (OffsetTypeEnum.PRECISE_OFFSET.getResetType() == dto.getResetType()) {
return Result.buildSuc(dto.getOffsetList().stream().collect(Collectors.toMap( return Result.buildSuc(dto.getOffsetList().stream().collect(Collectors.toMap(
elem -> new TopicPartition(dto.getTopicName(), elem.getPartitionId()), elem -> new TopicPartition(dto.getTopicName(), elem.getPartitionId()),
PartitionOffsetDTO::getOffset, PartitionOffsetDTO::getOffset,
@@ -221,9 +222,9 @@ public class GroupManagerImpl implements GroupManager {
} }
OffsetSpec offsetSpec = null; OffsetSpec offsetSpec = null;
if (GroupOffsetResetEnum.PRECISE_TIMESTAMP.getResetType() == dto.getResetType()) { if (OffsetTypeEnum.PRECISE_TIMESTAMP.getResetType() == dto.getResetType()) {
offsetSpec = OffsetSpec.forTimestamp(dto.getTimestamp()); offsetSpec = OffsetSpec.forTimestamp(dto.getTimestamp());
} else if (GroupOffsetResetEnum.EARLIEST.getResetType() == dto.getResetType()) { } else if (OffsetTypeEnum.EARLIEST.getResetType() == dto.getResetType()) {
offsetSpec = OffsetSpec.earliest(); offsetSpec = OffsetSpec.earliest();
} else { } else {
offsetSpec = OffsetSpec.latest(); offsetSpec = OffsetSpec.latest();
@@ -271,15 +272,11 @@ public class GroupManagerImpl implements GroupManager {
// 获取Group指标信息 // 获取Group指标信息
Result<List<GroupMetrics>> groupMetricsResult = groupMetricService.listPartitionLatestMetricsFromES( Result<List<GroupMetrics>> groupMetricsResult = groupMetricService.collectGroupMetricsFromKafka(clusterPhyId, groupName, latestMetricNames == null ? Arrays.asList() : latestMetricNames);
clusterPhyId,
groupName,
topicName,
latestMetricNames == null? Arrays.asList(): latestMetricNames
);
// 转换Group指标 // 转换Group指标
List<GroupMetrics> esGroupMetricsList = groupMetricsResult.hasData()? groupMetricsResult.getData(): new ArrayList<>(); List<GroupMetrics> esGroupMetricsList = groupMetricsResult.hasData() ? groupMetricsResult.getData().stream().filter(elem -> topicName.equals(elem.getTopic())).collect(Collectors.toList()) : new ArrayList<>();
Map<Integer, GroupMetrics> esMetricsMap = new HashMap<>(); Map<Integer, GroupMetrics> esMetricsMap = new HashMap<>();
for (GroupMetrics groupMetrics: esGroupMetricsList) { for (GroupMetrics groupMetrics: esGroupMetricsList) {
esMetricsMap.put(groupMetrics.getPartitionId(), groupMetrics); esMetricsMap.put(groupMetrics.getPartitionId(), groupMetrics);

View File

@@ -1,5 +1,6 @@
package com.xiaojukeji.know.streaming.km.biz.topic; package com.xiaojukeji.know.streaming.km.biz.topic;
import com.xiaojukeji.know.streaming.km.common.bean.dto.pagination.PaginationSortDTO;
import com.xiaojukeji.know.streaming.km.common.bean.dto.topic.TopicRecordDTO; import com.xiaojukeji.know.streaming.km.common.bean.dto.topic.TopicRecordDTO;
import com.xiaojukeji.know.streaming.km.common.bean.entity.result.Result; import com.xiaojukeji.know.streaming.km.common.bean.entity.result.Result;
import com.xiaojukeji.know.streaming.km.common.bean.vo.topic.TopicBrokersPartitionsSummaryVO; import com.xiaojukeji.know.streaming.km.common.bean.vo.topic.TopicBrokersPartitionsSummaryVO;

View File

@@ -22,25 +22,26 @@ import com.xiaojukeji.know.streaming.km.common.bean.vo.topic.partition.TopicPart
import com.xiaojukeji.know.streaming.km.common.constant.Constant; import com.xiaojukeji.know.streaming.km.common.constant.Constant;
import com.xiaojukeji.know.streaming.km.common.constant.KafkaConstant; import com.xiaojukeji.know.streaming.km.common.constant.KafkaConstant;
import com.xiaojukeji.know.streaming.km.common.constant.MsgConstant; import com.xiaojukeji.know.streaming.km.common.constant.MsgConstant;
import com.xiaojukeji.know.streaming.km.common.converter.PartitionConverter;
import com.xiaojukeji.know.streaming.km.common.converter.TopicVOConverter; import com.xiaojukeji.know.streaming.km.common.converter.TopicVOConverter;
import com.xiaojukeji.know.streaming.km.common.enums.OffsetTypeEnum;
import com.xiaojukeji.know.streaming.km.common.enums.SortTypeEnum;
import com.xiaojukeji.know.streaming.km.common.exception.AdminOperateException; import com.xiaojukeji.know.streaming.km.common.exception.AdminOperateException;
import com.xiaojukeji.know.streaming.km.common.exception.NotExistException; import com.xiaojukeji.know.streaming.km.common.exception.NotExistException;
import com.xiaojukeji.know.streaming.km.common.utils.ConvertUtil; import com.xiaojukeji.know.streaming.km.common.utils.ConvertUtil;
import com.xiaojukeji.know.streaming.km.common.utils.PaginationUtil;
import com.xiaojukeji.know.streaming.km.common.utils.ValidateUtils; import com.xiaojukeji.know.streaming.km.common.utils.ValidateUtils;
import com.xiaojukeji.know.streaming.km.core.service.broker.BrokerService; import com.xiaojukeji.know.streaming.km.core.service.broker.BrokerService;
import com.xiaojukeji.know.streaming.km.core.service.cluster.ClusterPhyService; import com.xiaojukeji.know.streaming.km.core.service.cluster.ClusterPhyService;
import com.xiaojukeji.know.streaming.km.core.service.partition.PartitionMetricService; import com.xiaojukeji.know.streaming.km.core.service.partition.PartitionMetricService;
import com.xiaojukeji.know.streaming.km.core.service.topic.TopicConfigService;
import com.xiaojukeji.know.streaming.km.core.service.partition.PartitionService; import com.xiaojukeji.know.streaming.km.core.service.partition.PartitionService;
import com.xiaojukeji.know.streaming.km.core.service.topic.TopicConfigService;
import com.xiaojukeji.know.streaming.km.core.service.topic.TopicMetricService; import com.xiaojukeji.know.streaming.km.core.service.topic.TopicMetricService;
import com.xiaojukeji.know.streaming.km.core.service.topic.TopicService; import com.xiaojukeji.know.streaming.km.core.service.topic.TopicService;
import com.xiaojukeji.know.streaming.km.core.service.version.metrics.TopicMetricVersionItems; import com.xiaojukeji.know.streaming.km.core.service.version.metrics.TopicMetricVersionItems;
import org.apache.commons.lang3.ObjectUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.kafka.clients.admin.OffsetSpec; import org.apache.kafka.clients.admin.OffsetSpec;
import org.apache.kafka.clients.consumer.ConsumerConfig; import org.apache.kafka.clients.consumer.*;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.common.TopicPartition; import org.apache.kafka.common.TopicPartition;
import org.apache.kafka.common.config.TopicConfig; import org.apache.kafka.common.config.TopicConfig;
import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Autowired;
@@ -160,8 +161,31 @@ public class TopicStateManagerImpl implements TopicStateManager {
} }
maxMessage = Math.min(maxMessage, dto.getMaxRecords()); maxMessage = Math.min(maxMessage, dto.getMaxRecords());
kafkaConsumer.assign(partitionList); kafkaConsumer.assign(partitionList);
Map<TopicPartition, OffsetAndTimestamp> partitionOffsetAndTimestampMap = new HashMap<>();
// 获取指定时间每个分区的offset按指定开始时间查询消息时
if (OffsetTypeEnum.PRECISE_TIMESTAMP.getResetType() == dto.getFilterOffsetReset()) {
Map<TopicPartition, Long> timestampsToSearch = new HashMap<>();
partitionList.forEach(topicPartition -> {
timestampsToSearch.put(topicPartition, dto.getStartTimestampUnitMs());
});
partitionOffsetAndTimestampMap = kafkaConsumer.offsetsForTimes(timestampsToSearch);
}
for (TopicPartition partition : partitionList) { for (TopicPartition partition : partitionList) {
kafkaConsumer.seek(partition, Math.max(beginOffsetsMapResult.getData().get(partition), endOffsetsMapResult.getData().get(partition) - dto.getMaxRecords())); if (OffsetTypeEnum.EARLIEST.getResetType() == dto.getFilterOffsetReset()) {
// 重置到最旧
kafkaConsumer.seek(partition, beginOffsetsMapResult.getData().get(partition));
} else if (OffsetTypeEnum.PRECISE_TIMESTAMP.getResetType() == dto.getFilterOffsetReset()) {
// 重置到指定时间
kafkaConsumer.seek(partition, partitionOffsetAndTimestampMap.get(partition).offset());
} else if (OffsetTypeEnum.PRECISE_OFFSET.getResetType() == dto.getFilterOffsetReset()) {
// 重置到指定位置
} else {
// 默认,重置到最新
kafkaConsumer.seek(partition, Math.max(beginOffsetsMapResult.getData().get(partition), endOffsetsMapResult.getData().get(partition) - dto.getMaxRecords()));
}
} }
// 这里需要减去 KafkaConstant.POLL_ONCE_TIMEOUT_UNIT_MS 是因为poll一次需要耗时如果这里不减去则可能会导致poll之后超过要求的时间 // 这里需要减去 KafkaConstant.POLL_ONCE_TIMEOUT_UNIT_MS 是因为poll一次需要耗时如果这里不减去则可能会导致poll之后超过要求的时间
@@ -185,6 +209,15 @@ public class TopicStateManagerImpl implements TopicStateManager {
} }
} }
// 排序
if (ObjectUtils.isNotEmpty(voList)) {
// 默认按时间倒序排序
if (StringUtils.isBlank(dto.getSortType())) {
dto.setSortType(SortTypeEnum.DESC.getSortType());
}
PaginationUtil.pageBySort(voList, dto.getSortField(), dto.getSortType());
}
return Result.buildSuc(voList.subList(0, Math.min(dto.getMaxRecords(), voList.size()))); return Result.buildSuc(voList.subList(0, Math.min(dto.getMaxRecords(), voList.size())));
} catch (Exception e) { } catch (Exception e) {
log.error("method=getTopicMessages||clusterPhyId={}||topicName={}||param={}||errMsg=exception", clusterPhyId, topicName, dto, e); log.error("method=getTopicMessages||clusterPhyId={}||topicName={}||param={}||errMsg=exception", clusterPhyId, topicName, dto, e);

View File

@@ -7,12 +7,14 @@ import com.didiglobal.logi.log.LogFactory;
import com.didiglobal.logi.security.common.dto.config.ConfigDTO; import com.didiglobal.logi.security.common.dto.config.ConfigDTO;
import com.didiglobal.logi.security.service.ConfigService; import com.didiglobal.logi.security.service.ConfigService;
import com.xiaojukeji.know.streaming.km.biz.version.VersionControlManager; import com.xiaojukeji.know.streaming.km.biz.version.VersionControlManager;
import com.xiaojukeji.know.streaming.km.common.bean.dto.metrices.MetricDetailDTO;
import com.xiaojukeji.know.streaming.km.common.bean.dto.metrices.UserMetricConfigDTO; import com.xiaojukeji.know.streaming.km.common.bean.dto.metrices.UserMetricConfigDTO;
import com.xiaojukeji.know.streaming.km.common.bean.entity.config.metric.UserMetricConfig; import com.xiaojukeji.know.streaming.km.common.bean.entity.config.metric.UserMetricConfig;
import com.xiaojukeji.know.streaming.km.common.bean.entity.result.Result; import com.xiaojukeji.know.streaming.km.common.bean.entity.result.Result;
import com.xiaojukeji.know.streaming.km.common.bean.entity.version.VersionControlItem; import com.xiaojukeji.know.streaming.km.common.bean.entity.version.VersionControlItem;
import com.xiaojukeji.know.streaming.km.common.bean.vo.config.metric.UserMetricConfigVO; import com.xiaojukeji.know.streaming.km.common.bean.vo.config.metric.UserMetricConfigVO;
import com.xiaojukeji.know.streaming.km.common.bean.vo.version.VersionItemVO; import com.xiaojukeji.know.streaming.km.common.bean.vo.version.VersionItemVO;
import com.xiaojukeji.know.streaming.km.common.constant.Constant;
import com.xiaojukeji.know.streaming.km.common.enums.version.VersionEnum; import com.xiaojukeji.know.streaming.km.common.enums.version.VersionEnum;
import com.xiaojukeji.know.streaming.km.common.utils.ConvertUtil; import com.xiaojukeji.know.streaming.km.common.utils.ConvertUtil;
import com.xiaojukeji.know.streaming.km.common.utils.VersionUtil; import com.xiaojukeji.know.streaming.km.common.utils.VersionUtil;
@@ -47,29 +49,29 @@ public class VersionControlManagerImpl implements VersionControlManager {
@PostConstruct @PostConstruct
public void init(){ public void init(){
defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_HEALTH_SCORE, true)); defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_HEALTH_SCORE, true));
defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_TOTAL_PRODUCE_REQUESTS, true));
defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_FAILED_FETCH_REQ, true)); defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_FAILED_FETCH_REQ, true));
defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_FAILED_PRODUCE_REQ, true)); defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_FAILED_PRODUCE_REQ, true));
defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_MESSAGE_IN, true));
defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_UNDER_REPLICA_PARTITIONS, true)); defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_UNDER_REPLICA_PARTITIONS, true));
defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_TOTAL_PRODUCE_REQUESTS, true));
defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_BYTES_IN, true)); defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_BYTES_IN, true));
defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_BYTES_OUT, true)); defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_BYTES_OUT, true));
defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_BYTES_REJECTED, true)); defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_BYTES_REJECTED, true));
defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_MESSAGE_IN, true));
defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_HEALTH_SCORE, true)); defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_HEALTH_SCORE, true));
defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_TOTAL_REQ_QUEUE_SIZE, true));
defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_TOTAL_RES_QUEUE_SIZE, true));
defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_ACTIVE_CONTROLLER_COUNT, true)); defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_ACTIVE_CONTROLLER_COUNT, true));
defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_TOTAL_PRODUCE_REQ, true));
defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_TOTAL_LOG_SIZE, true));
defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_CONNECTIONS, true));
defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_MESSAGES_IN, true));
defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_BYTES_IN, true)); defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_BYTES_IN, true));
defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_BYTES_OUT, true)); defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_BYTES_OUT, true));
defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_GROUP_REBALANCES, true)); defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_CONNECTIONS, true));
defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_JOB_RUNNING, true)); defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_MESSAGES_IN, true));
defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_PARTITIONS_NO_LEADER, true)); defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_PARTITIONS_NO_LEADER, true));
defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_PARTITION_URP, true)); defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_PARTITION_URP, true));
defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_TOTAL_LOG_SIZE, true));
defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_TOTAL_PRODUCE_REQ, true));
defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_TOTAL_REQ_QUEUE_SIZE, true));
defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_TOTAL_RES_QUEUE_SIZE, true));
defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_GROUP_REBALANCES, true));
defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_JOB_RUNNING, true));
defaultMetrics.add(new UserMetricConfig(METRIC_GROUP.getCode(), GROUP_METRIC_OFFSET_CONSUMED, true)); defaultMetrics.add(new UserMetricConfig(METRIC_GROUP.getCode(), GROUP_METRIC_OFFSET_CONSUMED, true));
defaultMetrics.add(new UserMetricConfig(METRIC_GROUP.getCode(), GROUP_METRIC_LAG, true)); defaultMetrics.add(new UserMetricConfig(METRIC_GROUP.getCode(), GROUP_METRIC_LAG, true));
@@ -77,18 +79,18 @@ public class VersionControlManagerImpl implements VersionControlManager {
defaultMetrics.add(new UserMetricConfig(METRIC_GROUP.getCode(), GROUP_METRIC_HEALTH_SCORE, true)); defaultMetrics.add(new UserMetricConfig(METRIC_GROUP.getCode(), GROUP_METRIC_HEALTH_SCORE, true));
defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_HEALTH_SCORE, true)); defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_HEALTH_SCORE, true));
defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_TOTAL_REQ_QUEUE, true)); defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_CONNECTION_COUNT, true));
defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_TOTAL_RES_QUEUE, true));
defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_MESSAGE_IN, true)); defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_MESSAGE_IN, true));
defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_TOTAL_PRODUCE_REQ, true));
defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_NETWORK_RPO_AVG_IDLE, true)); defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_NETWORK_RPO_AVG_IDLE, true));
defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_REQ_AVG_IDLE, true)); defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_REQ_AVG_IDLE, true));
defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_CONNECTION_COUNT, true)); defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_TOTAL_PRODUCE_REQ, true));
defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_BYTES_IN, true)); defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_TOTAL_REQ_QUEUE, true));
defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_BYTES_OUT, true)); defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_TOTAL_RES_QUEUE, true));
defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_PARTITIONS_SKEW, true));
defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_LEADERS_SKEW, true)); defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_LEADERS_SKEW, true));
defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_UNDER_REPLICATE_PARTITION, true)); defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_UNDER_REPLICATE_PARTITION, true));
defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_PARTITIONS_SKEW, true));
defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_BYTES_IN, true));
defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_BYTES_OUT, true));
} }
@Autowired @Autowired
@@ -159,6 +161,9 @@ public class VersionControlManagerImpl implements VersionControlManager {
UserMetricConfig umc = userMetricConfigMap.get(itemType + "@" + metric); UserMetricConfig umc = userMetricConfigMap.get(itemType + "@" + metric);
userMetricConfigVO.setSet(null != umc && umc.isSet()); userMetricConfigVO.setSet(null != umc && umc.isSet());
if (umc != null) {
userMetricConfigVO.setRank(umc.getRank());
}
userMetricConfigVO.setName(itemVO.getName()); userMetricConfigVO.setName(itemVO.getName());
userMetricConfigVO.setType(itemVO.getType()); userMetricConfigVO.setType(itemVO.getType());
userMetricConfigVO.setDesc(itemVO.getDesc()); userMetricConfigVO.setDesc(itemVO.getDesc());
@@ -178,13 +183,29 @@ public class VersionControlManagerImpl implements VersionControlManager {
@Override @Override
public Result<Void> updateUserMetricItem(Long clusterId, Integer type, UserMetricConfigDTO dto, String operator) { public Result<Void> updateUserMetricItem(Long clusterId, Integer type, UserMetricConfigDTO dto, String operator) {
Map<String, Boolean> metricsSetMap = dto.getMetricsSet(); Map<String, Boolean> metricsSetMap = dto.getMetricsSet();
if(null == metricsSetMap || metricsSetMap.isEmpty()){
//转换metricDetailDTOList
List<MetricDetailDTO> metricDetailDTOList = dto.getMetricDetailDTOList();
Map<String, MetricDetailDTO> metricDetailMap = new HashMap<>();
if (metricDetailDTOList != null && !metricDetailDTOList.isEmpty()) {
metricDetailMap = metricDetailDTOList.stream().collect(Collectors.toMap(MetricDetailDTO::getMetric, Function.identity()));
}
//转换metricsSetMap
if (metricsSetMap != null && !metricsSetMap.isEmpty()) {
for (Map.Entry<String, Boolean> metricAndShowEntry : metricsSetMap.entrySet()) {
if (metricDetailMap.containsKey(metricAndShowEntry.getKey())) continue;
metricDetailMap.put(metricAndShowEntry.getKey(), new MetricDetailDTO(metricAndShowEntry.getKey(), metricAndShowEntry.getValue(), null));
}
}
if (metricDetailMap.isEmpty()) {
return Result.buildSuc(); return Result.buildSuc();
} }
Set<UserMetricConfig> userMetricConfigs = getUserMetricConfig(operator); Set<UserMetricConfig> userMetricConfigs = getUserMetricConfig(operator);
for(Map.Entry<String, Boolean> metricAndShowEntry : metricsSetMap.entrySet()){ for (MetricDetailDTO metricDetailDTO : metricDetailMap.values()) {
UserMetricConfig userMetricConfig = new UserMetricConfig(type, metricAndShowEntry.getKey(), metricAndShowEntry.getValue()); UserMetricConfig userMetricConfig = new UserMetricConfig(type, metricDetailDTO.getMetric(), metricDetailDTO.getSet(), metricDetailDTO.getRank());
userMetricConfigs.remove(userMetricConfig); userMetricConfigs.remove(userMetricConfig);
userMetricConfigs.add(userMetricConfig); userMetricConfigs.add(userMetricConfig);
} }
@@ -228,7 +249,7 @@ public class VersionControlManagerImpl implements VersionControlManager {
return defaultMetrics; return defaultMetrics;
} }
return JSON.parseObject(value, new TypeReference<Set<UserMetricConfig>>(){}); return JSON.parseObject(value, new TypeReference<Set<UserMetricConfig>>() {});
} }
public static void main(String[] args){ public static void main(String[] args){

View File

@@ -1,121 +0,0 @@
package com.xiaojukeji.know.streaming.km.collector.metric;
import com.didiglobal.logi.log.ILog;
import com.didiglobal.logi.log.LogFactory;
import com.xiaojukeji.know.streaming.km.common.bean.event.metric.*;
import com.xiaojukeji.know.streaming.km.common.bean.po.BaseESPO;
import com.xiaojukeji.know.streaming.km.common.bean.po.metrice.*;
import com.xiaojukeji.know.streaming.km.common.utils.ConvertUtil;
import com.xiaojukeji.know.streaming.km.common.utils.EnvUtil;
import com.xiaojukeji.know.streaming.km.common.utils.NamedThreadFactory;
import com.xiaojukeji.know.streaming.km.persistence.es.dao.BaseMetricESDAO;
import org.apache.commons.collections.CollectionUtils;
import org.springframework.context.ApplicationListener;
import org.springframework.stereotype.Component;
import javax.annotation.PostConstruct;
import java.util.List;
import java.util.Objects;
import java.util.concurrent.LinkedBlockingDeque;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import static com.xiaojukeji.know.streaming.km.common.constant.ESIndexConstant.*;
@Component
public class MetricESSender implements ApplicationListener<BaseMetricEvent> {
protected static final ILog LOGGER = LogFactory.getLog("METRIC_LOGGER");
private static final int THRESHOLD = 100;
private ThreadPoolExecutor esExecutor = new ThreadPoolExecutor(10, 20, 6000, TimeUnit.MILLISECONDS,
new LinkedBlockingDeque<>(1000),
new NamedThreadFactory("KM-Collect-MetricESSender-ES"),
(r, e) -> LOGGER.warn("class=MetricESSender||msg=KM-Collect-MetricESSender-ES Deque is blocked, taskCount:{}" + e.getTaskCount()));
@PostConstruct
public void init(){
LOGGER.info("class=MetricESSender||method=init||msg=init finished");
}
@Override
public void onApplicationEvent(BaseMetricEvent event) {
if(event instanceof BrokerMetricEvent) {
BrokerMetricEvent brokerMetricEvent = (BrokerMetricEvent)event;
send2es(BROKER_INDEX,
ConvertUtil.list2List(brokerMetricEvent.getBrokerMetrics(), BrokerMetricPO.class)
);
} else if(event instanceof ClusterMetricEvent) {
ClusterMetricEvent clusterMetricEvent = (ClusterMetricEvent)event;
send2es(CLUSTER_INDEX,
ConvertUtil.list2List(clusterMetricEvent.getClusterMetrics(), ClusterMetricPO.class)
);
} else if(event instanceof TopicMetricEvent) {
TopicMetricEvent topicMetricEvent = (TopicMetricEvent)event;
send2es(TOPIC_INDEX,
ConvertUtil.list2List(topicMetricEvent.getTopicMetrics(), TopicMetricPO.class)
);
} else if(event instanceof PartitionMetricEvent) {
PartitionMetricEvent partitionMetricEvent = (PartitionMetricEvent)event;
send2es(PARTITION_INDEX,
ConvertUtil.list2List(partitionMetricEvent.getPartitionMetrics(), PartitionMetricPO.class)
);
} else if(event instanceof GroupMetricEvent) {
GroupMetricEvent groupMetricEvent = (GroupMetricEvent)event;
send2es(GROUP_INDEX,
ConvertUtil.list2List(groupMetricEvent.getGroupMetrics(), GroupMetricPO.class)
);
} else if(event instanceof ReplicaMetricEvent) {
ReplicaMetricEvent replicaMetricEvent = (ReplicaMetricEvent)event;
send2es(REPLICATION_INDEX,
ConvertUtil.list2List(replicaMetricEvent.getReplicationMetrics(), ReplicationMetricPO.class)
);
}
}
/**
* 根据不同监控维度来发送
*/
private boolean send2es(String index, List<? extends BaseESPO> statsList){
if (CollectionUtils.isEmpty(statsList)) {
return true;
}
if (!EnvUtil.isOnline()) {
LOGGER.info("class=MetricESSender||method=send2es||ariusStats={}||size={}",
index, statsList.size());
}
BaseMetricESDAO baseMetricESDao = BaseMetricESDAO.getByStatsType(index);
if (Objects.isNull( baseMetricESDao )) {
LOGGER.error("class=MetricESSender||method=send2es||errMsg=fail to find {}", index);
return false;
}
int size = statsList.size();
int num = (size) % THRESHOLD == 0 ? (size / THRESHOLD) : (size / THRESHOLD + 1);
if (size < THRESHOLD) {
esExecutor.execute(
() -> baseMetricESDao.batchInsertStats(statsList)
);
return true;
}
for (int i = 1; i < num + 1; i++) {
int end = (i * THRESHOLD) > size ? size : (i * THRESHOLD);
int start = (i - 1) * THRESHOLD;
esExecutor.execute(
() -> baseMetricESDao.batchInsertStats(statsList.subList(start, end))
);
}
return true;
}
}

View File

@@ -0,0 +1,72 @@
package com.xiaojukeji.know.streaming.km.collector.sink;
import com.didiglobal.logi.log.ILog;
import com.didiglobal.logi.log.LogFactory;
import com.xiaojukeji.know.streaming.km.common.bean.po.BaseESPO;
import com.xiaojukeji.know.streaming.km.common.utils.EnvUtil;
import com.xiaojukeji.know.streaming.km.common.utils.NamedThreadFactory;
import com.xiaojukeji.know.streaming.km.persistence.es.dao.BaseMetricESDAO;
import org.apache.commons.collections.CollectionUtils;
import java.util.List;
import java.util.Objects;
import java.util.concurrent.LinkedBlockingDeque;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
public abstract class AbstractMetricESSender {
protected static final ILog LOGGER = LogFactory.getLog("METRIC_LOGGER");
private static final int THRESHOLD = 100;
private static final ThreadPoolExecutor esExecutor = new ThreadPoolExecutor(
10,
20,
6000,
TimeUnit.MILLISECONDS,
new LinkedBlockingDeque<>(1000),
new NamedThreadFactory("KM-Collect-MetricESSender-ES"),
(r, e) -> LOGGER.warn("class=MetricESSender||msg=KM-Collect-MetricESSender-ES Deque is blocked, taskCount:{}" + e.getTaskCount())
);
/**
* 根据不同监控维度来发送
*/
protected boolean send2es(String index, List<? extends BaseESPO> statsList){
if (CollectionUtils.isEmpty(statsList)) {
return true;
}
if (!EnvUtil.isOnline()) {
LOGGER.info("class=MetricESSender||method=send2es||ariusStats={}||size={}",
index, statsList.size());
}
BaseMetricESDAO baseMetricESDao = BaseMetricESDAO.getByStatsType(index);
if (Objects.isNull( baseMetricESDao )) {
LOGGER.error("class=MetricESSender||method=send2es||errMsg=fail to find {}", index);
return false;
}
int size = statsList.size();
int num = (size) % THRESHOLD == 0 ? (size / THRESHOLD) : (size / THRESHOLD + 1);
if (size < THRESHOLD) {
esExecutor.execute(
() -> baseMetricESDao.batchInsertStats(statsList)
);
return true;
}
for (int i = 1; i < num + 1; i++) {
int end = (i * THRESHOLD) > size ? size : (i * THRESHOLD);
int start = (i - 1) * THRESHOLD;
esExecutor.execute(
() -> baseMetricESDao.batchInsertStats(statsList.subList(start, end))
);
}
return true;
}
}

View File

@@ -0,0 +1,28 @@
package com.xiaojukeji.know.streaming.km.collector.sink;
import com.didiglobal.logi.log.ILog;
import com.didiglobal.logi.log.LogFactory;
import com.xiaojukeji.know.streaming.km.common.bean.event.metric.BrokerMetricEvent;
import com.xiaojukeji.know.streaming.km.common.bean.po.metrice.BrokerMetricPO;
import com.xiaojukeji.know.streaming.km.common.utils.ConvertUtil;
import org.springframework.context.ApplicationListener;
import org.springframework.stereotype.Component;
import javax.annotation.PostConstruct;
import static com.xiaojukeji.know.streaming.km.common.constant.ESIndexConstant.BROKER_INDEX;
@Component
public class BrokerMetricESSender extends AbstractMetricESSender implements ApplicationListener<BrokerMetricEvent> {
protected static final ILog LOGGER = LogFactory.getLog("METRIC_LOGGER");
@PostConstruct
public void init(){
LOGGER.info("class=BrokerMetricESSender||method=init||msg=init finished");
}
@Override
public void onApplicationEvent(BrokerMetricEvent event) {
send2es(BROKER_INDEX, ConvertUtil.list2List(event.getBrokerMetrics(), BrokerMetricPO.class));
}
}

View File

@@ -0,0 +1,29 @@
package com.xiaojukeji.know.streaming.km.collector.sink;
import com.didiglobal.logi.log.ILog;
import com.didiglobal.logi.log.LogFactory;
import com.xiaojukeji.know.streaming.km.common.bean.event.metric.ClusterMetricEvent;
import com.xiaojukeji.know.streaming.km.common.bean.po.metrice.ClusterMetricPO;
import com.xiaojukeji.know.streaming.km.common.utils.ConvertUtil;
import org.springframework.context.ApplicationListener;
import org.springframework.stereotype.Component;
import javax.annotation.PostConstruct;
import static com.xiaojukeji.know.streaming.km.common.constant.ESIndexConstant.CLUSTER_INDEX;
@Component
public class ClusterMetricESSender extends AbstractMetricESSender implements ApplicationListener<ClusterMetricEvent> {
protected static final ILog LOGGER = LogFactory.getLog("METRIC_LOGGER");
@PostConstruct
public void init(){
LOGGER.info("class=ClusterMetricESSender||method=init||msg=init finished");
}
@Override
public void onApplicationEvent(ClusterMetricEvent event) {
send2es(CLUSTER_INDEX, ConvertUtil.list2List(event.getClusterMetrics(), ClusterMetricPO.class));
}
}

View File

@@ -0,0 +1,29 @@
package com.xiaojukeji.know.streaming.km.collector.sink;
import com.didiglobal.logi.log.ILog;
import com.didiglobal.logi.log.LogFactory;
import com.xiaojukeji.know.streaming.km.common.bean.event.metric.GroupMetricEvent;
import com.xiaojukeji.know.streaming.km.common.bean.po.metrice.GroupMetricPO;
import com.xiaojukeji.know.streaming.km.common.utils.ConvertUtil;
import org.springframework.context.ApplicationListener;
import org.springframework.stereotype.Component;
import javax.annotation.PostConstruct;
import static com.xiaojukeji.know.streaming.km.common.constant.ESIndexConstant.GROUP_INDEX;
@Component
public class GroupMetricESSender extends AbstractMetricESSender implements ApplicationListener<GroupMetricEvent> {
protected static final ILog LOGGER = LogFactory.getLog("METRIC_LOGGER");
@PostConstruct
public void init(){
LOGGER.info("class=GroupMetricESSender||method=init||msg=init finished");
}
@Override
public void onApplicationEvent(GroupMetricEvent event) {
send2es(GROUP_INDEX, ConvertUtil.list2List(event.getGroupMetrics(), GroupMetricPO.class));
}
}

View File

@@ -0,0 +1,28 @@
package com.xiaojukeji.know.streaming.km.collector.sink;
import com.didiglobal.logi.log.ILog;
import com.didiglobal.logi.log.LogFactory;
import com.xiaojukeji.know.streaming.km.common.bean.event.metric.PartitionMetricEvent;
import com.xiaojukeji.know.streaming.km.common.bean.po.metrice.PartitionMetricPO;
import com.xiaojukeji.know.streaming.km.common.utils.ConvertUtil;
import org.springframework.context.ApplicationListener;
import org.springframework.stereotype.Component;
import javax.annotation.PostConstruct;
import static com.xiaojukeji.know.streaming.km.common.constant.ESIndexConstant.PARTITION_INDEX;
@Component
public class PartitionMetricESSender extends AbstractMetricESSender implements ApplicationListener<PartitionMetricEvent> {
protected static final ILog LOGGER = LogFactory.getLog("METRIC_LOGGER");
@PostConstruct
public void init(){
LOGGER.info("class=PartitionMetricESSender||method=init||msg=init finished");
}
@Override
public void onApplicationEvent(PartitionMetricEvent event) {
send2es(PARTITION_INDEX, ConvertUtil.list2List(event.getPartitionMetrics(), PartitionMetricPO.class));
}
}

View File

@@ -0,0 +1,28 @@
package com.xiaojukeji.know.streaming.km.collector.sink;
import com.didiglobal.logi.log.ILog;
import com.didiglobal.logi.log.LogFactory;
import com.xiaojukeji.know.streaming.km.common.bean.event.metric.ReplicaMetricEvent;
import com.xiaojukeji.know.streaming.km.common.bean.po.metrice.ReplicationMetricPO;
import com.xiaojukeji.know.streaming.km.common.utils.ConvertUtil;
import org.springframework.context.ApplicationListener;
import org.springframework.stereotype.Component;
import javax.annotation.PostConstruct;
import static com.xiaojukeji.know.streaming.km.common.constant.ESIndexConstant.REPLICATION_INDEX;
@Component
public class ReplicaMetricESSender extends AbstractMetricESSender implements ApplicationListener<ReplicaMetricEvent> {
protected static final ILog LOGGER = LogFactory.getLog("METRIC_LOGGER");
@PostConstruct
public void init(){
LOGGER.info("class=GroupMetricESSender||method=init||msg=init finished");
}
@Override
public void onApplicationEvent(ReplicaMetricEvent event) {
send2es(REPLICATION_INDEX, ConvertUtil.list2List(event.getReplicationMetrics(), ReplicationMetricPO.class));
}
}

View File

@@ -0,0 +1,29 @@
package com.xiaojukeji.know.streaming.km.collector.sink;
import com.didiglobal.logi.log.ILog;
import com.didiglobal.logi.log.LogFactory;
import com.xiaojukeji.know.streaming.km.common.bean.event.metric.*;
import com.xiaojukeji.know.streaming.km.common.bean.po.metrice.*;
import com.xiaojukeji.know.streaming.km.common.utils.ConvertUtil;
import org.springframework.context.ApplicationListener;
import org.springframework.stereotype.Component;
import javax.annotation.PostConstruct;
import static com.xiaojukeji.know.streaming.km.common.constant.ESIndexConstant.TOPIC_INDEX;
@Component
public class TopicMetricESSender extends AbstractMetricESSender implements ApplicationListener<TopicMetricEvent> {
protected static final ILog LOGGER = LogFactory.getLog("METRIC_LOGGER");
@PostConstruct
public void init(){
LOGGER.info("class=TopicMetricESSender||method=init||msg=init finished");
}
@Override
public void onApplicationEvent(TopicMetricEvent event) {
send2es(TOPIC_INDEX, ConvertUtil.list2List(event.getTopicMetrics(), TopicMetricPO.class));
}
}

View File

@@ -3,6 +3,7 @@ package com.xiaojukeji.know.streaming.km.common.bean.dto.cluster;
import com.fasterxml.jackson.annotation.JsonIgnoreProperties; import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.xiaojukeji.know.streaming.km.common.bean.dto.BaseDTO; import com.xiaojukeji.know.streaming.km.common.bean.dto.BaseDTO;
import com.xiaojukeji.know.streaming.km.common.bean.entity.config.JmxConfig; import com.xiaojukeji.know.streaming.km.common.bean.entity.config.JmxConfig;
import com.xiaojukeji.know.streaming.km.common.bean.entity.config.ZKConfig;
import io.swagger.annotations.ApiModel; import io.swagger.annotations.ApiModel;
import io.swagger.annotations.ApiModelProperty; import io.swagger.annotations.ApiModelProperty;
import lombok.Data; import lombok.Data;
@@ -34,4 +35,8 @@ public class ClusterPhyBaseDTO extends BaseDTO {
@NotNull(message = "jmxProperties不允许为空") @NotNull(message = "jmxProperties不允许为空")
@ApiModelProperty(value="Jmx配置") @ApiModelProperty(value="Jmx配置")
protected JmxConfig jmxProperties; protected JmxConfig jmxProperties;
// TODO 前端页面增加时,需要加一个不为空的限制
@ApiModelProperty(value="ZK配置")
protected ZKConfig zkProperties;
} }

View File

@@ -3,6 +3,7 @@ package com.xiaojukeji.know.streaming.km.common.bean.dto.group;
import com.fasterxml.jackson.annotation.JsonIgnoreProperties; import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.xiaojukeji.know.streaming.km.common.bean.dto.partition.PartitionOffsetDTO; import com.xiaojukeji.know.streaming.km.common.bean.dto.partition.PartitionOffsetDTO;
import com.xiaojukeji.know.streaming.km.common.bean.dto.topic.ClusterTopicDTO; import com.xiaojukeji.know.streaming.km.common.bean.dto.topic.ClusterTopicDTO;
import com.xiaojukeji.know.streaming.km.common.enums.OffsetTypeEnum;
import io.swagger.annotations.ApiModelProperty; import io.swagger.annotations.ApiModelProperty;
import lombok.Data; import lombok.Data;
@@ -23,7 +24,7 @@ public class GroupOffsetResetDTO extends ClusterTopicDTO {
private String groupName; private String groupName;
/** /**
* @see com.xiaojukeji.know.streaming.km.common.enums.GroupOffsetResetEnum * @see OffsetTypeEnum
*/ */
@NotNull(message = "resetType不允许为空") @NotNull(message = "resetType不允许为空")
@ApiModelProperty(value = "重置方式", example = "1") @ApiModelProperty(value = "重置方式", example = "1")

View File

@@ -0,0 +1,32 @@
package com.xiaojukeji.know.streaming.km.common.bean.dto.metrices;
import com.xiaojukeji.know.streaming.km.common.bean.dto.BaseDTO;
import io.swagger.annotations.ApiModel;
import io.swagger.annotations.ApiModelProperty;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import javax.validation.constraints.NotNull;
/**
* @author didi
*/
@Data
@NoArgsConstructor
@AllArgsConstructor
@ApiModel(description = "指标详细属性信息")
public class MetricDetailDTO extends BaseDTO {
@ApiModelProperty("指标名称")
private String metric;
@ApiModelProperty("指标是否显示")
private Boolean set;
@NotNull(message = "MetricDetailDTO的rank字段应不为空")
@ApiModelProperty("指标优先级")
private Integer rank;
}

View File

@@ -7,6 +7,8 @@ import lombok.AllArgsConstructor;
import lombok.Data; import lombok.Data;
import lombok.NoArgsConstructor; import lombok.NoArgsConstructor;
import javax.validation.Valid;
import java.util.List;
import java.util.Map; import java.util.Map;
@@ -17,4 +19,8 @@ import java.util.Map;
public class UserMetricConfigDTO extends BaseDTO { public class UserMetricConfigDTO extends BaseDTO {
@ApiModelProperty("指标展示设置项key指标名value是否展现(true展现/false不展现)") @ApiModelProperty("指标展示设置项key指标名value是否展现(true展现/false不展现)")
private Map<String, Boolean> metricsSet; private Map<String, Boolean> metricsSet;
@Valid
@ApiModelProperty("指标自定义属性列表")
private List<MetricDetailDTO> metricDetailDTOList;
} }

View File

@@ -1,7 +1,8 @@
package com.xiaojukeji.know.streaming.km.common.bean.dto.topic; package com.xiaojukeji.know.streaming.km.common.bean.dto.topic;
import com.fasterxml.jackson.annotation.JsonIgnoreProperties; import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.xiaojukeji.know.streaming.km.common.bean.dto.BaseDTO; import com.xiaojukeji.know.streaming.km.common.bean.dto.pagination.PaginationSortDTO;
import com.xiaojukeji.know.streaming.km.common.enums.OffsetTypeEnum;
import io.swagger.annotations.ApiModel; import io.swagger.annotations.ApiModel;
import io.swagger.annotations.ApiModelProperty; import io.swagger.annotations.ApiModelProperty;
import lombok.Data; import lombok.Data;
@@ -15,7 +16,7 @@ import javax.validation.constraints.NotNull;
@Data @Data
@JsonIgnoreProperties(ignoreUnknown = true) @JsonIgnoreProperties(ignoreUnknown = true)
@ApiModel(description = "Topic记录") @ApiModel(description = "Topic记录")
public class TopicRecordDTO extends BaseDTO { public class TopicRecordDTO extends PaginationSortDTO {
@NotNull(message = "truncate不允许为空") @NotNull(message = "truncate不允许为空")
@ApiModelProperty(value = "是否截断", example = "true") @ApiModelProperty(value = "是否截断", example = "true")
private Boolean truncate; private Boolean truncate;
@@ -34,4 +35,13 @@ public class TopicRecordDTO extends BaseDTO {
@ApiModelProperty(value = "预览超时时间", example = "10000") @ApiModelProperty(value = "预览超时时间", example = "10000")
private Long pullTimeoutUnitMs = 8000L; private Long pullTimeoutUnitMs = 8000L;
/**
* @see OffsetTypeEnum
*/
@ApiModelProperty(value = "offset", example = "")
private Integer filterOffsetReset = 0;
@ApiModelProperty(value = "开始日期时间戳", example = "")
private Long startTimestampUnitMs;
} }

View File

@@ -5,7 +5,6 @@ import com.alibaba.fastjson.TypeReference;
import com.xiaojukeji.know.streaming.km.common.bean.entity.common.IpPortData; import com.xiaojukeji.know.streaming.km.common.bean.entity.common.IpPortData;
import com.xiaojukeji.know.streaming.km.common.bean.po.broker.BrokerPO; import com.xiaojukeji.know.streaming.km.common.bean.po.broker.BrokerPO;
import com.xiaojukeji.know.streaming.km.common.utils.ConvertUtil; import com.xiaojukeji.know.streaming.km.common.utils.ConvertUtil;
import com.xiaojukeji.know.streaming.km.common.zookeeper.znode.brokers.BrokerMetadata;
import lombok.AllArgsConstructor; import lombok.AllArgsConstructor;
import lombok.Data; import lombok.Data;
import lombok.NoArgsConstructor; import lombok.NoArgsConstructor;
@@ -79,20 +78,6 @@ public class Broker implements Serializable {
return metadata; return metadata;
} }
public static Broker buildFrom(Long clusterPhyId, Integer brokerId, BrokerMetadata brokerMetadata) {
Broker metadata = new Broker();
metadata.setClusterPhyId(clusterPhyId);
metadata.setBrokerId(brokerId);
metadata.setHost(brokerMetadata.getHost());
metadata.setPort(brokerMetadata.getPort());
metadata.setJmxPort(brokerMetadata.getJmxPort());
metadata.setStartTimestamp(brokerMetadata.getTimestamp());
metadata.setRack(brokerMetadata.getRack());
metadata.setStatus(1);
metadata.setEndpointMap(brokerMetadata.getEndpointMap());
return metadata;
}
public static Broker buildFrom(BrokerPO brokerPO) { public static Broker buildFrom(BrokerPO brokerPO) {
Broker broker = ConvertUtil.obj2Obj(brokerPO, Broker.class); Broker broker = ConvertUtil.obj2Obj(brokerPO, Broker.class);
String endpointMapStr = brokerPO.getEndpointMap(); String endpointMapStr = brokerPO.getEndpointMap();

View File

@@ -53,9 +53,16 @@ public class ClusterPhy implements Comparable<ClusterPhy>, EntifyIdInterface {
/** /**
* jmx配置 * jmx配置
* @see com.xiaojukeji.know.streaming.km.common.bean.entity.config.JmxConfig
*/ */
private String jmxProperties; private String jmxProperties;
/**
* zk配置
* @see com.xiaojukeji.know.streaming.km.common.bean.entity.config.ZKConfig
*/
private String zkProperties;
/** /**
* 开启ACL * 开启ACL
* @see com.xiaojukeji.know.streaming.km.common.enums.cluster.ClusterAuthTypeEnum * @see com.xiaojukeji.know.streaming.km.common.enums.cluster.ClusterAuthTypeEnum

View File

@@ -0,0 +1,31 @@
package com.xiaojukeji.know.streaming.km.common.bean.entity.config;
import io.swagger.annotations.ApiModel;
import io.swagger.annotations.ApiModelProperty;
import lombok.Data;
import java.io.Serializable;
import java.util.Properties;
/**
* @author zengqiao
* @date 22/02/24
*/
@Data
@ApiModel(description = "ZK配置")
public class ZKConfig implements Serializable {
@ApiModelProperty(value="ZK的jmx配置")
private JmxConfig jmxConfig;
@ApiModelProperty(value="ZK是否开启secure", example = "false")
private Boolean openSecure = false;
@ApiModelProperty(value="ZK的Session超时时间", example = "15000")
private Long sessionTimeoutUnitMs = 15000L;
@ApiModelProperty(value="ZK的Request超时时间", example = "5000")
private Long requestTimeoutUnitMs = 5000L;
@ApiModelProperty(value="ZK的Request超时时间")
private Properties otherProps = new Properties();
}

View File

@@ -1,12 +1,12 @@
package com.xiaojukeji.know.streaming.km.common.bean.entity.config.metric; package com.xiaojukeji.know.streaming.km.common.bean.entity.config.metric;
import com.xiaojukeji.know.streaming.km.common.constant.Constant;
import lombok.AllArgsConstructor; import lombok.AllArgsConstructor;
import lombok.Data; import lombok.Data;
import lombok.NoArgsConstructor; import lombok.NoArgsConstructor;
@Data @Data
@NoArgsConstructor @NoArgsConstructor
@AllArgsConstructor
public class UserMetricConfig { public class UserMetricConfig {
private int type; private int type;
@@ -15,6 +15,22 @@ public class UserMetricConfig {
private boolean set; private boolean set;
private Integer rank;
public UserMetricConfig(int type, String metric, boolean set, Integer rank) {
this.type = type;
this.metric = metric;
this.set = set;
this.rank = rank;
}
public UserMetricConfig(int type, String metric, boolean set) {
this.type = type;
this.metric = metric;
this.set = set;
this.rank = null;
}
@Override @Override
public int hashCode(){ public int hashCode(){
return metric.hashCode() << 1 + type; return metric.hashCode() << 1 + type;

View File

@@ -0,0 +1,15 @@
package com.xiaojukeji.know.streaming.km.common.bean.event.cluster;
import lombok.Getter;
/**
* 集群新增事件
* @author zengqiao
* @date 22/02/25
*/
@Getter
public class ClusterPhyAddedEvent extends ClusterPhyBaseEvent {
public ClusterPhyAddedEvent(Object source, Long clusterPhyId) {
super(source, clusterPhyId);
}
}

View File

@@ -1,26 +0,0 @@
package com.xiaojukeji.know.streaming.km.common.bean.event.kafka.zk;
import lombok.Getter;
@Getter
public abstract class BaseKafkaZKEvent {
/**
* 触发时间
*/
protected Long eventTime;
/**
* 初始化数据的事件
*/
protected Boolean initEvent;
/**
* 集群ID
*/
protected Long clusterPhyId;
protected BaseKafkaZKEvent(Long eventTime, Long clusterPhyId) {
this.eventTime = eventTime;
this.clusterPhyId = clusterPhyId;
}
}

View File

@@ -1,10 +0,0 @@
package com.xiaojukeji.know.streaming.km.common.bean.event.kafka.zk;
import lombok.Getter;
@Getter
public class ControllerChangeEvent extends BaseKafkaZKEvent {
public ControllerChangeEvent(Long eventTime, Long clusterPhyId) {
super(eventTime, clusterPhyId);
}
}

View File

@@ -41,6 +41,11 @@ public class ClusterPhyPO extends BasePO {
*/ */
private String jmxProperties; private String jmxProperties;
/**
* zk配置
*/
private String zkProperties;
/** /**
* 认证类型 * 认证类型
* @see com.xiaojukeji.know.streaming.km.common.enums.cluster.ClusterAuthTypeEnum * @see com.xiaojukeji.know.streaming.km.common.enums.cluster.ClusterAuthTypeEnum

View File

@@ -31,9 +31,15 @@ public class ClusterPhyBaseVO extends BaseTimeVO {
@ApiModelProperty(value="Jmx配置", example = "{}") @ApiModelProperty(value="Jmx配置", example = "{}")
protected String jmxProperties; protected String jmxProperties;
@ApiModelProperty(value="ZK配置", example = "{}")
protected String zkProperties;
@ApiModelProperty(value="描述", example = "测试") @ApiModelProperty(value="描述", example = "测试")
protected String description; protected String description;
@ApiModelProperty(value="集群的kafka版本", example = "2.5.1") @ApiModelProperty(value="集群的kafka版本", example = "2.5.1")
protected String kafkaVersion; protected String kafkaVersion;
@ApiModelProperty(value="集群的运行模式", example = "2raft模式其他是ZK模式")
private Integer runState;
} }

View File

@@ -14,4 +14,7 @@ import lombok.NoArgsConstructor;
public class UserMetricConfigVO extends VersionItemVO { public class UserMetricConfigVO extends VersionItemVO {
@ApiModelProperty(value = "该指标用户是否设置展现", example = "true") @ApiModelProperty(value = "该指标用户是否设置展现", example = "true")
private Boolean set; private Boolean set;
@ApiModelProperty(value = "该指标展示优先级", example = "1")
private Integer rank;
} }

View File

@@ -42,6 +42,7 @@ public class Constant {
*/ */
public static final Integer DEFAULT_CLUSTER_HEALTH_SCORE = 90; public static final Integer DEFAULT_CLUSTER_HEALTH_SCORE = 90;
public static final String DEFAULT_USER_NAME = "know-streaming-app"; public static final String DEFAULT_USER_NAME = "know-streaming-app";
public static final int INVALID_CODE = -1; public static final int INVALID_CODE = -1;
@@ -64,4 +65,5 @@ public class Constant {
public static final Float COLLECT_METRICS_ERROR_COST_TIME = -1.0F; public static final Float COLLECT_METRICS_ERROR_COST_TIME = -1.0F;
public static final Integer DEFAULT_RETRY_TIME = 3; public static final Integer DEFAULT_RETRY_TIME = 3;
} }

View File

@@ -41,6 +41,8 @@ public class KafkaConstant {
public static final Long POLL_ONCE_TIMEOUT_UNIT_MS = 2000L; public static final Long POLL_ONCE_TIMEOUT_UNIT_MS = 2000L;
public static final String CONTROLLER_ROLE = "controller";
public static final Map<String, ConfigDef.ConfigKey> KAFKA_ALL_CONFIG_DEF_MAP = new ConcurrentHashMap<>(); public static final Map<String, ConfigDef.ConfigKey> KAFKA_ALL_CONFIG_DEF_MAP = new ConcurrentHashMap<>();
static { static {

View File

@@ -52,6 +52,10 @@ public class MsgConstant {
/**************************************************** Partition ****************************************************/ /**************************************************** Partition ****************************************************/
public static String getPartitionNoLeader(Long clusterPhyId, String topicName) {
return String.format("集群ID:[%d] Topic名称:[%s] 所有分区NoLeader", clusterPhyId, topicName);
}
public static String getPartitionNotExist(Long clusterPhyId, String topicName) { public static String getPartitionNotExist(Long clusterPhyId, String topicName) {
return String.format("集群ID:[%d] Topic名称:[%s] 存在非法的分区ID", clusterPhyId, topicName); return String.format("集群ID:[%d] Topic名称:[%s] 存在非法的分区ID", clusterPhyId, topicName);
} }

View File

@@ -19,6 +19,11 @@ public class ClusterConverter {
ClusterPhyPO clusterPhyPO = ConvertUtil.obj2Obj(dto, ClusterPhyPO.class); ClusterPhyPO clusterPhyPO = ConvertUtil.obj2Obj(dto, ClusterPhyPO.class);
clusterPhyPO.setClientProperties(ConvertUtil.obj2Json(dto.getClientProperties())); clusterPhyPO.setClientProperties(ConvertUtil.obj2Json(dto.getClientProperties()));
clusterPhyPO.setJmxProperties(ConvertUtil.obj2Json(dto.getJmxProperties())); clusterPhyPO.setJmxProperties(ConvertUtil.obj2Json(dto.getJmxProperties()));
if (ValidateUtils.isNull(dto.getZkProperties())) {
clusterPhyPO.setZkProperties("");
} else {
clusterPhyPO.setZkProperties(ConvertUtil.obj2Json(dto.getZkProperties()));
}
clusterPhyPO.setRunState( clusterPhyPO.setRunState(
ValidateUtils.isBlank(dto.getZookeeper())? ValidateUtils.isBlank(dto.getZookeeper())?
ClusterRunStateEnum.RUN_RAFT.getRunState() : ClusterRunStateEnum.RUN_RAFT.getRunState() :
@@ -32,6 +37,11 @@ public class ClusterConverter {
ClusterPhyPO clusterPhyPO = ConvertUtil.obj2Obj(dto, ClusterPhyPO.class); ClusterPhyPO clusterPhyPO = ConvertUtil.obj2Obj(dto, ClusterPhyPO.class);
clusterPhyPO.setClientProperties(ConvertUtil.obj2Json(dto.getClientProperties())); clusterPhyPO.setClientProperties(ConvertUtil.obj2Json(dto.getClientProperties()));
clusterPhyPO.setJmxProperties(ConvertUtil.obj2Json(dto.getJmxProperties())); clusterPhyPO.setJmxProperties(ConvertUtil.obj2Json(dto.getJmxProperties()));
if (ValidateUtils.isNull(dto.getZkProperties())) {
clusterPhyPO.setZkProperties("");
} else {
clusterPhyPO.setZkProperties(ConvertUtil.obj2Json(dto.getZkProperties()));
}
clusterPhyPO.setRunState( clusterPhyPO.setRunState(
ValidateUtils.isBlank(dto.getZookeeper())? ValidateUtils.isBlank(dto.getZookeeper())?
ClusterRunStateEnum.RUN_RAFT.getRunState() : ClusterRunStateEnum.RUN_RAFT.getRunState() :

View File

@@ -170,6 +170,7 @@ public class ReassignConverter {
detail.setOriginalBrokerIdList(CommonUtils.string2IntList(subJobPO.getOriginalBrokerIds())); detail.setOriginalBrokerIdList(CommonUtils.string2IntList(subJobPO.getOriginalBrokerIds()));
detail.setReassignBrokerIdList(CommonUtils.string2IntList(subJobPO.getReassignBrokerIds())); detail.setReassignBrokerIdList(CommonUtils.string2IntList(subJobPO.getReassignBrokerIds()));
detail.setStatus(subJobPO.getStatus()); detail.setStatus(subJobPO.getStatus());
detail.setOldReplicaNum(detail.getOriginalBrokerIdList().size());
ReassignSubJobExtendData extendData = ConvertUtil.str2ObjByJson(subJobPO.getExtendData(), ReassignSubJobExtendData.class); ReassignSubJobExtendData extendData = ConvertUtil.str2ObjByJson(subJobPO.getExtendData(), ReassignSubJobExtendData.class);
if (extendData != null) { if (extendData != null) {
@@ -217,6 +218,7 @@ public class ReassignConverter {
topicDetail.setPresentReplicaNum(partitionDetailList.get(0).getPresentReplicaNum()); topicDetail.setPresentReplicaNum(partitionDetailList.get(0).getPresentReplicaNum());
topicDetail.setNewReplicaNum(partitionDetailList.get(0).getNewReplicaNum()); topicDetail.setNewReplicaNum(partitionDetailList.get(0).getNewReplicaNum());
topicDetail.setOldReplicaNum(partitionDetailList.get(0).getOldReplicaNum());
topicDetail.setOriginalRetentionTimeUnitMs(partitionDetailList.get(0).getOriginalRetentionTimeUnitMs()); topicDetail.setOriginalRetentionTimeUnitMs(partitionDetailList.get(0).getOriginalRetentionTimeUnitMs());
topicDetail.setReassignRetentionTimeUnitMs(partitionDetailList.get(0).getReassignRetentionTimeUnitMs()); topicDetail.setReassignRetentionTimeUnitMs(partitionDetailList.get(0).getReassignRetentionTimeUnitMs());

View File

@@ -3,19 +3,19 @@ package com.xiaojukeji.know.streaming.km.common.enums;
import lombok.Getter; import lombok.Getter;
/** /**
* 重置offset * offset类型
* @author zengqiao * @author zengqiao
* @date 19/4/8 * @date 19/4/8
*/ */
@Getter @Getter
public enum GroupOffsetResetEnum { public enum OffsetTypeEnum {
LATEST(0, "重置到最新"), LATEST(0, "最新"),
EARLIEST(1, "重置到最旧"), EARLIEST(1, "最旧"),
PRECISE_TIMESTAMP(2, "按时间进行重置"), PRECISE_TIMESTAMP(2, "指定时间"),
PRECISE_OFFSET(3, "重置到指定位置"), PRECISE_OFFSET(3, "指定位置"),
; ;
@@ -23,7 +23,7 @@ public enum GroupOffsetResetEnum {
private final String message; private final String message;
GroupOffsetResetEnum(int resetType, String message) { OffsetTypeEnum(int resetType, String message) {
this.resetType = resetType; this.resetType = resetType;
this.message = message; this.message = message;
} }

View File

@@ -90,6 +90,8 @@ public class JmxConnectorWrap {
} }
try { try {
jmxConnector.close(); jmxConnector.close();
jmxConnector = null;
} catch (IOException e) { } catch (IOException e) {
LOGGER.warn("close JmxConnector exception, physicalClusterId:{} brokerId:{} host:{} port:{}.", physicalClusterId, brokerId, host, port, e); LOGGER.warn("close JmxConnector exception, physicalClusterId:{} brokerId:{} host:{} port:{}.", physicalClusterId, brokerId, host, port, e);
} }
@@ -105,6 +107,11 @@ public class JmxConnectorWrap {
acquire(); acquire();
MBeanServerConnection mBeanServerConnection = jmxConnector.getMBeanServerConnection(); MBeanServerConnection mBeanServerConnection = jmxConnector.getMBeanServerConnection();
return mBeanServerConnection.getAttribute(name, attribute); return mBeanServerConnection.getAttribute(name, attribute);
} catch (IOException ioe) {
// 如果是因为连接断开,则进行重新连接,并抛出异常
reInitDueIOException();
throw ioe;
} finally { } finally {
atomicInteger.incrementAndGet(); atomicInteger.incrementAndGet();
} }
@@ -120,6 +127,11 @@ public class JmxConnectorWrap {
acquire(); acquire();
MBeanServerConnection mBeanServerConnection = jmxConnector.getMBeanServerConnection(); MBeanServerConnection mBeanServerConnection = jmxConnector.getMBeanServerConnection();
return mBeanServerConnection.getAttributes(name, attributes); return mBeanServerConnection.getAttributes(name, attributes);
} catch (IOException ioe) {
// 如果是因为连接断开,则进行重新连接,并抛出异常
reInitDueIOException();
throw ioe;
} finally { } finally {
atomicInteger.incrementAndGet(); atomicInteger.incrementAndGet();
} }
@@ -131,6 +143,11 @@ public class JmxConnectorWrap {
acquire(); acquire();
MBeanServerConnection mBeanServerConnection = jmxConnector.getMBeanServerConnection(); MBeanServerConnection mBeanServerConnection = jmxConnector.getMBeanServerConnection();
return mBeanServerConnection.queryNames(name, query); return mBeanServerConnection.queryNames(name, query);
} catch (IOException ioe) {
// 如果是因为连接断开,则进行重新连接,并抛出异常
reInitDueIOException();
throw ioe;
} finally { } finally {
atomicInteger.incrementAndGet(); atomicInteger.incrementAndGet();
} }
@@ -186,4 +203,26 @@ public class JmxConnectorWrap {
} }
} }
} }
private synchronized void reInitDueIOException() {
try {
if (jmxConnector == null) {
return;
}
// 检查是否正常
jmxConnector.getConnectionId();
// 如果正常则直接返回
return;
} catch (Exception e) {
// ignore
}
// 关闭旧的
this.close();
// 重新创建
this.checkJmxConnectionAndInitIfNeed();
}
} }

View File

@@ -241,4 +241,14 @@ public class CommonUtils {
} }
return intList; return intList;
} }
public static boolean isNumeric(String str){
for (int i = 0; i < str.length(); i++){
if (!Character.isDigit(str.charAt(i))){
return false;
}
}
return true;
}
} }

View File

@@ -9,6 +9,5 @@ build/
coverage coverage
versions/ versions/
debug.log debug.log
package-lock.json
yarn.lock yarn.lock
target target

View File

@@ -1,43 +1,65 @@
## 安装项目依赖 ## 前提
- 安装 lerna 通常情况下,您可以通过 [本地源码启动手册](https://github.com/didi/KnowStreaming/blob/master/docs/dev_guide/%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%90%AF%E5%8A%A8%E6%89%8B%E5%86%8C.md) 来打包工程。如果您需要在本地独立启动或打包前端服务,请参考以下手册。
在进行以下的步骤之前,首先确保您已经安装了 `node`。如已安装,可以通过在终端执行 `node -v` 来获取到 node 版本,项目推荐使用 `node v12` 版本运行(例如 `node v12.22.12`)。
另外,`windows` 用户请在 `git bash` 下运行下面的命令。
## 一、进入 km-console 目录
在终端执行以下步骤时,需要先进入 `KnowStreaming/km-console` 目录。
## 二、安装项目依赖(必须)
1. 安装 lerna可选安装后可以直接通过 lerna 的全局指令管理项目,如果不了解 lerna 可以不安装)
``` ```
npm install -g lerna npm install -g lerna
``` ```
- 安装项目依赖 2. 安装项目依赖
``` ```
npm run i npm run i
``` ```
## 启动项目 我们默认保留了 `package-lock.json` 文件,以防止可能的依赖包自动升级导致的问题。依赖默认会通过 taobao 镜像 `https://registry.npmmirror.com/` 服务下载。
## 三、启动项目(可选,打包构建请直接看步骤三)
``` ```
npm run start npm run start
``` ```
### 环境信息 该指令会启动 `packages` 目录下的所有应用,如果需要单独启动应用,其查看下方 QA。
http://localhost:port 多集群管理应用会启动在 http://localhost:8000系统管理应用会占用 http://localhost:8001。
请确认 `8000``8001` 端口没有被其他应用占用。
## 构建项目 后端本地服务启动在 http://localhost:8080请求通过 webpack dev server 代理访问 8080 端口,需要启动后端服务后才能正常请求接口。
如果启动失败,可以参见另外一种本地启动方式 [本地源码启动手册](https://github.com/didi/KnowStreaming/blob/master/docs/dev_guide/%E6%9C%AC%E5%9C%B0%E6%BA%90%E7%A0%81%E5%90%AF%E5%8A%A8%E6%89%8B%E5%86%8C.md)
## 四、构建项目
``` ```
npm run build npm run build
``` ```
项目构建成功后,会存放到 km-rest/src/main/resources/tamplates 目录下。
## 目录结构 ## 目录结构
- packages - packages
- layout-clusters-fe: 基座应用 & 多集群管理 - layout-clusters-fe: 基座应用 & 多集群管理(其余应用启动需要首先启动该应用)
- config-manager-fe: 子应用 - 系统管理 - config-manager-fe: 子应用 - 系统管理
- tool: 启动 & 打包脚本
- ... - ...
## 常见问题 ## QA
Q: 在 `km-console` 目录下执行 `npm run start` 时看不到应用构建和热加载过程?如何启动单个应用?
Q: 执行 `npm run start` 时看不到应用构建和热加载过程?
A: 需要到具体的应用中执行 `npm run start`,例如 `cd packages/layout-clusters-fe` 后,执行 `npm run start` A: 需要到具体的应用中执行 `npm run start`,例如 `cd packages/layout-clusters-fe` 后,执行 `npm run start`
如遇到其它问题,请见 [faq](https://github.com/didi/KnowStreaming/blob/master/docs/user_guide/faq.md)。

8567
km-console/package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -17,15 +17,15 @@
"eslint-plugin-react": "7.22.0", "eslint-plugin-react": "7.22.0",
"eslint-plugin-react-hooks": "^4.2.0", "eslint-plugin-react-hooks": "^4.2.0",
"husky": "4.3.7", "husky": "4.3.7",
"lerna": "^4.0.0", "lerna": "^5.5.0",
"lint-staged": "10.5.3", "lint-staged": "10.5.3",
"prettier": "2.3.2" "prettier": "2.3.2"
}, },
"scripts": { "scripts": {
"i": "npm install && lerna bootstrap", "i": "npm install && lerna bootstrap",
"clean": "rm -rf node_modules package-lock.json packages/*/node_modules packages/*/package-lock.json", "clean": "rm -rf node_modules package-lock.json packages/*/node_modules packages/*/package-lock.json",
"start": "sh ./tool/start.sh", "start": "lerna run start",
"build": "sh ./tool/build.sh", "build": "lerna run build",
"changelog": "conventional-changelog -p angular -i CHANGELOG.md -s -r 0 && git add CHANGELOG.md", "changelog": "conventional-changelog -p angular -i CHANGELOG.md -s -r 0 && git add CHANGELOG.md",
"cm": "git add . && cz" "cm": "git add . && cz"
}, },

View File

@@ -9,5 +9,4 @@ build/
coverage coverage
versions/ versions/
debug.log debug.log
package-lock.json
yarn.lock yarn.lock

View File

@@ -1,17 +1,21 @@
## 使用说明 ## 使用说明
### 依赖安装: ### 依赖安装(如在 km-console 目录下执行 npm run i 安装过依赖,这步可以省略)
``` ```
npm install npm install
``` ```
注意,这种方式只会安装当前应用的依赖。如果您不了解,推荐在 km-console 目录下执行 npm run i 安装依赖。
### 启动: ### 启动:
``` ```
npm run start npm run start
``` ```
该应用为子应用,启动后需要到基座应用中查看(需要启动基座应用,即 layout-clusters-fe地址为 http://localhost:8000
### 构建: ### 构建:
``` ```

View File

@@ -1,205 +0,0 @@
/* eslint-disable */
const MiniCssExtractPlugin = require('mini-css-extract-plugin');
const ProgressBarPlugin = require('progress-bar-webpack-plugin');
const CaseSensitivePathsPlugin = require('case-sensitive-paths-webpack-plugin');
const StatsPlugin = require('stats-webpack-plugin');
const { CleanWebpackPlugin } = require('clean-webpack-plugin');
const TerserJSPlugin = require('terser-webpack-plugin');
const OptimizeCSSAssetsPlugin = require('optimize-css-assets-webpack-plugin');
const HappyPack = require('happypack');
const os = require('os');
const happyThreadPool = HappyPack.ThreadPool({ size: os.cpus().length });
const ReactRefreshWebpackPlugin = require('@pmmmwh/react-refresh-webpack-plugin');
const theme = require('./theme');
var cwd = process.cwd();
const path = require('path');
const isProd = process.env.NODE_ENV === 'production';
const babelOptions = {
cacheDirectory: true,
babelrc: false,
presets: [require.resolve('@babel/preset-env'), require.resolve('@babel/preset-typescript'), require.resolve('@babel/preset-react')],
plugins: [
[require.resolve('@babel/plugin-proposal-decorators'), { legacy: true }],
[require.resolve('@babel/plugin-proposal-class-properties'), { loose: true }],
[require.resolve('@babel/plugin-proposal-private-methods'), { loose: true }],
require.resolve('@babel/plugin-proposal-export-default-from'),
require.resolve('@babel/plugin-proposal-export-namespace-from'),
require.resolve('@babel/plugin-proposal-object-rest-spread'),
require.resolve('@babel/plugin-transform-runtime'),
require.resolve('@babel/plugin-proposal-optional-chaining'), //
require.resolve('@babel/plugin-proposal-nullish-coalescing-operator'), // 解决 ?? 无法转义问题
require.resolve('@babel/plugin-proposal-numeric-separator'), // 转义 1_000_000
!isProd && require.resolve('react-refresh/babel'),
]
.filter(Boolean)
.concat([
[
'babel-plugin-import',
{
libraryName: 'antd',
style: true,
},
],
'@babel/plugin-transform-object-assign',
]),
};
module.exports = () => {
const manifestName = `manifest.json`;
const cssFileName = isProd ? '[name]-[chunkhash].css' : '[name].css';
const plugins = [
new ProgressBarPlugin(),
new CaseSensitivePathsPlugin(),
new MiniCssExtractPlugin({
filename: cssFileName,
}),
new StatsPlugin(manifestName, {
chunkModules: false,
source: true,
chunks: false,
modules: false,
assets: true,
children: false,
exclude: [/node_modules/],
}),
new HappyPack({
id: 'babel',
loaders: [
'cache-loader',
{
loader: 'babel-loader',
options: babelOptions,
},
],
threadPool: happyThreadPool,
}),
!isProd &&
new ReactRefreshWebpackPlugin({
overlay: false,
}),
// new BundleAnalyzerPlugin({
// analyzerPort: 8889
// }),
].filter(Boolean);
if (isProd) {
plugins.push(new CleanWebpackPlugin());
}
return {
externals: isProd
? [
/^react$/,
/^react\/lib.*/,
/^react-dom$/,
/.*react-dom.*/,
/^single-spa$/,
/^single-spa-react$/,
/^moment$/,
/^antd$/,
/^lodash$/,
/^react-router$/,
/^react-router-dom$/,
]
: [],
resolve: {
symlinks: false,
extensions: ['.web.jsx', '.web.js', '.ts', '.tsx', '.js', '.jsx', '.json'],
alias: {
// '@pkgs': path.resolve(cwd, 'src/packages'),
'@pkgs': path.resolve(cwd, './node_modules/@didi/d1-packages'),
'@cpts': path.resolve(cwd, 'src/components'),
'@interface': path.resolve(cwd, 'src/interface'),
'@apis': path.resolve(cwd, 'src/api'),
react: path.resolve('./node_modules/react'),
actions: path.resolve(cwd, 'src/actions'),
lib: path.resolve(cwd, 'src/lib'),
constants: path.resolve(cwd, 'src/constants'),
components: path.resolve(cwd, 'src/components'),
container: path.resolve(cwd, 'src/container'),
api: path.resolve(cwd, 'src/api'),
assets: path.resolve(cwd, 'src/assets'),
mobxStore: path.resolve(cwd, 'src/mobxStore'),
},
},
plugins,
module: {
rules: [
{
parser: { system: false },
},
{
test: /\.(js|jsx|ts|tsx)$/,
exclude: /node_modules\/(?!react-intl|@didi\/dcloud-design)/,
use: [
{
loader: 'happypack/loader?id=babel',
},
],
},
{
test: /\.(png|svg|jpeg|jpg|gif|ttf|woff|woff2|eot|pdf)$/,
use: [
{
loader: 'file-loader',
options: {
name: '[name].[ext]',
outputPath: './assets/image/',
esModule: false,
},
},
],
},
{
test: /\.(css|less)$/,
use: [
{
loader: MiniCssExtractPlugin.loader,
},
'css-loader',
{
loader: 'less-loader',
options: {
javascriptEnabled: true,
modifyVars: theme,
},
},
],
},
],
},
optimization: Object.assign(
{
splitChunks: {
cacheGroups: {
vendor: {
test: /[\\/]node_modules[\\/]/,
chunks: 'all',
name: 'vendor',
priority: 10,
enforce: true,
minChunks: 1,
maxSize: 3500000,
},
},
},
},
isProd
? {
minimizer: [
new TerserJSPlugin({
cache: true,
sourceMap: true,
}),
new OptimizeCSSAssetsPlugin({}),
],
}
: {}
),
devtool: isProd ? 'cheap-module-source-map' : 'source-map',
node: {
fs: 'empty',
net: 'empty',
tls: 'empty',
},
};
};

View File

@@ -0,0 +1,132 @@
const path = require('path');
const webpack = require('webpack');
const HtmlWebpackPlugin = require('html-webpack-plugin');
const MiniCssExtractPlugin = require('mini-css-extract-plugin');
const ProgressBarPlugin = require('progress-bar-webpack-plugin');
const CaseSensitivePathsPlugin = require('case-sensitive-paths-webpack-plugin');
const StatsPlugin = require('stats-webpack-plugin');
const HappyPack = require('happypack');
const os = require('os');
const happyThreadPool = HappyPack.ThreadPool({ size: os.cpus().length });
const theme = require('./theme');
const pkgJson = require('../package');
const devMode = process.env.NODE_ENV === 'development';
const babelOptions = {
cacheDirectory: true,
babelrc: false,
presets: [require.resolve('@babel/preset-env'), require.resolve('@babel/preset-typescript'), require.resolve('@babel/preset-react')],
plugins: [
[require.resolve('@babel/plugin-proposal-decorators'), { legacy: true }],
[require.resolve('@babel/plugin-proposal-class-properties'), { loose: true }],
[require.resolve('@babel/plugin-proposal-private-methods'), { loose: true }],
[require.resolve('@babel/plugin-proposal-private-property-in-object'), { loose: true }],
require.resolve('@babel/plugin-proposal-export-default-from'),
require.resolve('@babel/plugin-proposal-export-namespace-from'),
require.resolve('@babel/plugin-proposal-object-rest-spread'),
require.resolve('@babel/plugin-transform-runtime'),
require.resolve('@babel/plugin-proposal-optional-chaining'), //
require.resolve('@babel/plugin-proposal-nullish-coalescing-operator'), // 解决 ?? 无法转义问题
require.resolve('@babel/plugin-proposal-numeric-separator'), // 转义 1_000_000
devMode && require.resolve('react-refresh/babel'),
].filter(Boolean),
};
module.exports = {
entry: {
[pkgJson.ident]: ['./src/index.tsx'],
},
resolve: {
symlinks: false,
extensions: ['.web.jsx', '.web.js', '.ts', '.tsx', '.js', '.jsx', '.json'],
alias: {
'@src': path.resolve(process.cwd(), 'src'),
},
},
plugins: [
new ProgressBarPlugin(),
new CaseSensitivePathsPlugin(),
new StatsPlugin('manifest.json', {
chunkModules: false,
source: true,
chunks: false,
modules: false,
assets: true,
children: false,
exclude: [/node_modules/],
}),
new HappyPack({
id: 'babel',
loaders: [
'cache-loader',
{
loader: 'babel-loader',
options: babelOptions,
},
],
threadPool: happyThreadPool,
}),
new webpack.DefinePlugin({
'process.env': {
NODE_ENV: JSON.stringify(process.env.NODE_ENV),
RUN_ENV: JSON.stringify(process.env.RUN_ENV),
},
}),
new HtmlWebpackPlugin({
meta: {
manifest: 'manifest.json',
},
template: './src/index.html',
inject: 'body',
}),
].filter(Boolean),
module: {
rules: [
{
parser: { system: false },
},
{
test: /\.(js|jsx|ts|tsx)$/,
exclude: /node_modules\/(?!react-intl|@didi\/dcloud-design)/,
use: [
{
loader: 'happypack/loader?id=babel',
},
],
},
{
test: /\.(png|svg|jpeg|jpg|gif|ttf|woff|woff2|eot|pdf)$/,
use: [
{
loader: 'file-loader',
options: {
name: '[name].[ext]',
outputPath: './assets/image/',
esModule: false,
},
},
],
},
{
test: /\.(css|less)$/,
use: [
MiniCssExtractPlugin.loader,
'css-loader',
{
loader: 'less-loader',
options: {
javascriptEnabled: true,
modifyVars: theme,
},
},
],
},
],
},
node: {
fs: 'empty',
net: 'empty',
tls: 'empty',
},
stats: 'errors-warnings',
};

View File

@@ -0,0 +1,35 @@
const MiniCssExtractPlugin = require('mini-css-extract-plugin');
const ReactRefreshWebpackPlugin = require('@pmmmwh/react-refresh-webpack-plugin');
const pkgJson = require('../package');
module.exports = {
mode: 'development',
plugins: [
new MiniCssExtractPlugin(),
new ReactRefreshWebpackPlugin({
overlay: false,
}),
],
devServer: {
host: '127.0.0.1',
port: pkgJson.port,
hot: true,
open: false,
publicPath: `http://localhost:${pkgJson.port}/${pkgJson.ident}/`,
inline: true,
disableHostCheck: true,
historyApiFallback: true,
headers: {
'Access-Control-Allow-Origin': '*',
},
},
output: {
path: '/',
publicPath: `http://localhost:${pkgJson.port}/${pkgJson.ident}/`,
library: pkgJson.ident,
libraryTarget: 'amd',
filename: '[name].js',
chunkFilename: '[name].js',
},
devtool: 'cheap-module-eval-source-map',
};

View File

@@ -0,0 +1,59 @@
const path = require('path');
const MiniCssExtractPlugin = require('mini-css-extract-plugin');
const { CleanWebpackPlugin } = require('clean-webpack-plugin');
const TerserJSPlugin = require('terser-webpack-plugin');
const OptimizeCSSAssetsPlugin = require('optimize-css-assets-webpack-plugin');
const pkgJson = require('../package');
module.exports = {
mode: 'production',
externals: [
/^react$/,
/^react\/lib.*/,
/^react-dom$/,
/.*react-dom.*/,
/^single-spa$/,
/^single-spa-react$/,
/^moment$/,
/^lodash$/,
/^react-router$/,
/^react-router-dom$/,
],
plugins: [
new CleanWebpackPlugin(),
new MiniCssExtractPlugin({
filename: '[name]-[chunkhash].css',
}),
],
output: {
path: path.resolve(process.cwd(), `../../../km-rest/src/main/resources/templates/${pkgJson.ident}`),
publicPath: `${process.env.PUBLIC_PATH}/${pkgJson.ident}/`,
library: pkgJson.ident,
libraryTarget: 'amd',
filename: '[name]-[chunkhash].js',
chunkFilename: '[name]-[chunkhash].js',
},
optimization: {
splitChunks: {
cacheGroups: {
vendor: {
test: /[\\/]node_modules[\\/]/,
chunks: 'all',
name: 'vendor',
priority: 10,
enforce: true,
minChunks: 1,
maxSize: 3500000,
},
},
},
minimizer: [
new TerserJSPlugin({
cache: true,
sourceMap: true,
}),
new OptimizeCSSAssetsPlugin({}),
],
},
devtool: 'none',
};

File diff suppressed because it is too large Load Diff

View File

@@ -18,12 +18,14 @@
"scripts": { "scripts": {
"test": "echo \"Error: run tests from root\" && exit 1", "test": "echo \"Error: run tests from root\" && exit 1",
"start": "cross-env NODE_ENV=development webpack-dev-server", "start": "cross-env NODE_ENV=development webpack-dev-server",
"build": "rm -rf ../../pub/layout & cross-env NODE_ENV=production webpack --max_old_space_size=8000" "build": "cross-env NODE_ENV=production webpack --max_old_space_size=8000"
}, },
"dependencies": { "dependencies": {
"@knowdesign/icons": "^1.0.0",
"babel-preset-react-app": "^10.0.0", "babel-preset-react-app": "^10.0.0",
"classnames": "^2.2.6", "classnames": "^2.2.6",
"dotenv": "^16.0.1", "dotenv": "^16.0.1",
"knowdesign": "1.3.7",
"less": "^3.9.0", "less": "^3.9.0",
"lodash": "^4.17.11", "lodash": "^4.17.11",
"mobx": "4.15.7", "mobx": "4.15.7",
@@ -36,8 +38,7 @@
"react-intl": "^3.2.1", "react-intl": "^3.2.1",
"react-router-cache-route": "^1.11.1", "react-router-cache-route": "^1.11.1",
"single-spa": "^5.8.0", "single-spa": "^5.8.0",
"single-spa-react": "^2.14.0", "single-spa-react": "^2.14.0"
"knowdesign": "1.3.7"
}, },
"devDependencies": { "devDependencies": {
"@ant-design/icons": "^4.6.2", "@ant-design/icons": "^4.6.2",
@@ -58,6 +59,7 @@
"@pmmmwh/react-refresh-webpack-plugin": "^0.5.1", "@pmmmwh/react-refresh-webpack-plugin": "^0.5.1",
"@types/lodash": "^4.14.138", "@types/lodash": "^4.14.138",
"@types/react-dom": "^17.0.5", "@types/react-dom": "^17.0.5",
"@types/react-router": "5.1.18",
"@types/react-router-dom": "^5.3.3", "@types/react-router-dom": "^5.3.3",
"@types/single-spa-react": "^2.12.0", "@types/single-spa-react": "^2.12.0",
"@typescript-eslint/eslint-plugin": "4.13.0", "@typescript-eslint/eslint-plugin": "4.13.0",

View File

@@ -22,6 +22,20 @@
display: flex; display: flex;
justify-content: space-between; justify-content: space-between;
margin-bottom: 12px; margin-bottom: 12px;
.left,
.right {
display: flex;
align-items: center;
}
.left .refresh-icon {
font-size: 20px;
color: #74788d;
cursor: pointer;
}
.right .search-input {
width: 248px;
margin-right: 8px;
}
} }
} }
} }

View File

@@ -35,11 +35,20 @@ serviceInstance.interceptors.request.use(
// 响应拦截 // 响应拦截
serviceInstance.interceptors.response.use( serviceInstance.interceptors.response.use(
(config: any) => { (config: any) => {
return config.data; const res: { code: number; message: string; data: any } = config.data;
if (res.code !== 0 && res.code !== 200) {
const desc = res.message;
notification.error({
message: desc,
duration: 3,
});
throw res;
}
return res;
}, },
(err: any) => { (err: any) => {
const config = err.config; const config = err?.config;
if (!config || !config.retryTimes) return dealResponse(err, config.customNotification); if (!config || !config.retryTimes) return dealResponse(err);
const { __retryCount = 0, retryDelay = 300, retryTimes } = config; const { __retryCount = 0, retryDelay = 300, retryTimes } = config;
config.__retryCount = __retryCount; config.__retryCount = __retryCount;
if (__retryCount >= retryTimes) { if (__retryCount >= retryTimes) {

View File

@@ -1,6 +1,6 @@
import React, { useLayoutEffect } from 'react'; import React, { useLayoutEffect } from 'react';
import { Utils, AppContainer } from 'knowdesign'; import { Utils, AppContainer } from 'knowdesign';
import { goLogin } from 'constants/axiosConfig'; import { goLogin } from '@src/constants/axiosConfig';
// 权限对应表 // 权限对应表
export enum ConfigPermissionMap { export enum ConfigPermissionMap {

View File

@@ -15,6 +15,7 @@ import {
AppContainer, AppContainer,
Utils, Utils,
} from 'knowdesign'; } from 'knowdesign';
import { IconFont } from '@knowdesign/icons';
import { PlusOutlined } from '@ant-design/icons'; import { PlusOutlined } from '@ant-design/icons';
import moment from 'moment'; import moment from 'moment';
// 引入代码编辑器 // 引入代码编辑器
@@ -26,8 +27,8 @@ import 'codemirror/addon/selection/active-line';
import 'codemirror/addon/edit/closebrackets'; import 'codemirror/addon/edit/closebrackets';
require('codemirror/mode/xml/xml'); require('codemirror/mode/xml/xml');
require('codemirror/mode/javascript/javascript'); require('codemirror/mode/javascript/javascript');
import api from 'api'; import api from '@src/api';
import { defaultPagination } from 'constants/common'; import { defaultPagination } from '@src/constants/common';
import TypicalListCard from '../../components/TypicalListCard'; import TypicalListCard from '../../components/TypicalListCard';
import { ConfigPermissionMap } from '../CommonConfig'; import { ConfigPermissionMap } from '../CommonConfig';
import { ConfigOperate, ConfigProps } from './config'; import { ConfigOperate, ConfigProps } from './config';
@@ -384,7 +385,7 @@ export default () => {
const onDelete = (record: ConfigProps) => { const onDelete = (record: ConfigProps) => {
confirm({ confirm({
title: '确定删除配置吗?', title: '确定删除配置吗?',
content: `配置${record.valueName}${record.status === 1 ? '为启用状态,无法删除' : ''}`, content: `配置 [${record.valueName}] ${record.status === 1 ? '为启用状态,无法删除' : ''}`,
centered: true, centered: true,
okText: '删除', okText: '删除',
okType: 'primary', okType: 'primary',
@@ -398,9 +399,11 @@ export default () => {
}, },
maskClosable: true, maskClosable: true,
onOk() { onOk() {
return request(api.editConfig, { return request(api.delConfig, {
method: 'POST', method: 'DELETE',
data: record.id, params: {
id: record.id,
},
}).then((_) => { }).then((_) => {
message.success('删除成功'); message.success('删除成功');
getConfigList(); getConfigList();
@@ -431,22 +434,28 @@ export default () => {
<TypicalListCard title="配置管理"> <TypicalListCard title="配置管理">
<div className="config-manage-page"> <div className="config-manage-page">
<div className="operate-bar"> <div className="operate-bar">
<Form form={form} layout="inline" onFinish={() => getConfigList({ page: 1 })}> <div className="left">
<Form.Item name="valueGroup"> <div className="refresh-icon" onClick={() => getConfigList()}>
<Select style={{ width: 180 }} placeholder="请选择模块" options={configGroupList} /> <IconFont className="icon" type="icon-shuaxin1" />
</Form.Item> </div>
<Form.Item name="valueName"> <Divider type="vertical" style={{ height: 20, top: 0 }} />
<Input style={{ width: 180 }} placeholder="请输入配置键" /> <Form form={form} layout="inline" onFinish={() => getConfigList({ page: 1 })}>
</Form.Item> <Form.Item name="valueGroup">
<Form.Item name="memo"> <Select style={{ width: 180 }} placeholder="请选择模块" options={configGroupList} />
<Input style={{ width: 180 }} placeholder="请输入描述" /> </Form.Item>
</Form.Item> <Form.Item name="valueName">
<Form.Item> <Input style={{ width: 180 }} placeholder="请输入配置键" />
<Button type="primary" ghost htmlType="submit"> </Form.Item>
<Form.Item name="memo">
</Button> <Input style={{ width: 180 }} placeholder="请输入描述" />
</Form.Item> </Form.Item>
</Form> <Form.Item>
<Button type="primary" ghost htmlType="submit">
</Button>
</Form.Item>
</Form>
</div>
{global.hasPermission && global.hasPermission(ConfigPermissionMap.CONFIG_ADD) ? ( {global.hasPermission && global.hasPermission(ConfigPermissionMap.CONFIG_ADD) ? (
<Button <Button
type="primary" type="primary"

View File

@@ -1,7 +1,8 @@
import React, { useEffect, useState } from 'react'; import React, { useEffect, useState } from 'react';
import { Button, Form, Input, Select, ProTable, DatePicker, Utils, Tooltip } from 'knowdesign'; import { Button, Form, Input, Select, ProTable, DatePicker, Utils, Tooltip, Divider } from 'knowdesign';
import api from 'api'; import { IconFont } from '@knowdesign/icons';
import { defaultPagination } from 'constants/common'; import api from '@src/api';
import { defaultPagination } from '@src/constants/common';
import TypicalListCard from '../../components/TypicalListCard'; import TypicalListCard from '../../components/TypicalListCard';
import './index.less'; import './index.less';
import moment from 'moment'; import moment from 'moment';
@@ -119,25 +120,32 @@ export default () => {
<> <>
<TypicalListCard title="操作记录"> <TypicalListCard title="操作记录">
<div className="operate-bar"> <div className="operate-bar">
<Form form={form} layout="inline" onFinish={() => getData({ page: 1 })}> <div className="left">
<Form.Item name="targetType"> <div className="refresh-icon" onClick={() => getData()}>
<Select placeholder="请选择模块" options={configGroupList} style={{ width: 160 }} /> <IconFont className="icon" type="icon-shuaxin1" />
</Form.Item> </div>
<Form.Item name="target"> <Divider type="vertical" style={{ height: 20, top: 0 }} />
<Input placeholder="请输入操作对象" />
</Form.Item> <Form form={form} layout="inline" onFinish={() => getData({ page: 1 })}>
<Form.Item name="detail"> <Form.Item name="targetType">
<Input placeholder="请输入操作内容" /> <Select placeholder="请选择模块" options={configGroupList} style={{ width: 160 }} />
</Form.Item> </Form.Item>
<Form.Item name="time"> <Form.Item name="target">
<RangePicker showTime /> <Input placeholder="请输入操作对象" />
</Form.Item> </Form.Item>
<Form.Item> <Form.Item name="detail">
<Button type="primary" ghost htmlType="submit"> <Input placeholder="请输入操作内容" />
</Form.Item>
</Button> <Form.Item name="time">
</Form.Item> <RangePicker showTime />
</Form> </Form.Item>
<Form.Item>
<Button type="primary" ghost htmlType="submit">
</Button>
</Form.Item>
</Form>
</div>
</div> </div>
<ProTable <ProTable

View File

@@ -73,12 +73,12 @@ const CheckboxGroupContainer = (props: CheckboxGroupType) => {
</Checkbox> </Checkbox>
</div> </div>
<Checkbox.Group disabled={disabled} style={{ width: '100%' }} value={checkedList} onChange={onCheckedChange}> <Checkbox.Group disabled={disabled} style={{ width: '100%' }} value={checkedList} onChange={onCheckedChange}>
<Row gutter={[34, 10]}> <Row gutter={[10, 10]}>
{options.map((option) => { {options.map((option) => {
return ( return (
<Col span={8} key={option.value}> <Col span={8} key={option.value}>
<Checkbox value={option.value} className="checkbox-content-ellipsis"> <Checkbox value={option.value} className="checkbox-content-ellipsis">
{option.label} {option.label.replace('Cluster-Load', '')}
</Checkbox> </Checkbox>
</Col> </Col>
); );

View File

@@ -20,10 +20,10 @@ import {
IconFont, IconFont,
} from 'knowdesign'; } from 'knowdesign';
import moment from 'moment'; import moment from 'moment';
import { CloseOutlined, LoadingOutlined, PlusOutlined } from '@ant-design/icons'; import { LoadingOutlined, PlusOutlined } from '@ant-design/icons';
import { defaultPagination } from 'constants/common'; import { defaultPagination } from '@src/constants/common';
import { RoleProps, PermissionNode, AssignUser, RoleOperate, FormItemPermission } from './config'; import { RoleProps, PermissionNode, AssignUser, RoleOperate, FormItemPermission } from './config';
import api from 'api'; import api from '@src/api';
import CheckboxGroupContainer from './CheckboxGroupContainer'; import CheckboxGroupContainer from './CheckboxGroupContainer';
import { ConfigPermissionMap } from '../CommonConfig'; import { ConfigPermissionMap } from '../CommonConfig';
@@ -50,11 +50,21 @@ const RoleDetailAndUpdate = forwardRef((props, ref): JSX.Element => {
useEffect(() => { useEffect(() => {
const globalPermissions = global.permissions; const globalPermissions = global.permissions;
if (globalPermissions && globalPermissions.length) { if (globalPermissions && globalPermissions.length) {
const sysPermissions = globalPermissions.map((sys: PermissionNode) => ({ const sysPermissions = globalPermissions.map((sys: PermissionNode) => {
id: sys.id, const result = {
name: sys.permissionName, id: sys.id,
options: sys.childList.map((node) => ({ label: node.permissionName, value: node.id })), name: sys.permissionName,
})); essentialPermission: undefined,
options: [],
};
result.options = sys.childList.map((node) => {
if (node.permissionName === '多集群管理查看' || node.permissionName === '系统管理查看') {
result.essentialPermission = { label: node.permissionName, value: node.id };
}
return { label: node.permissionName, value: node.id };
});
return result;
});
setPermissions(sysPermissions); setPermissions(sysPermissions);
} }
}, [global]); }, [global]);
@@ -77,10 +87,12 @@ const RoleDetailAndUpdate = forwardRef((props, ref): JSX.Element => {
const onSubmit = () => { const onSubmit = () => {
form.validateFields().then((formData) => { form.validateFields().then((formData) => {
formData.permissionIdList = formData.permissionIdList.filter((l) => l);
formData.permissionIdList.forEach((arr, i) => { formData.permissionIdList.forEach((arr, i) => {
// 如果分配的系统下的子权限,自动赋予该系统的权限 // 如果分配的系统下的子权限,自动赋予该系统的权限
if (arr !== null && arr.length) { if (!Array.isArray(arr)) {
arr = [];
}
if (arr?.length) {
arr.push(permissions[i].id); arr.push(permissions[i].id);
} }
}); });
@@ -210,10 +222,20 @@ const RoleDetailAndUpdate = forwardRef((props, ref): JSX.Element => {
<Form.Item <Form.Item
label="分配权限" label="分配权限"
name="permissionIdList" name="permissionIdList"
required
rules={[ rules={[
() => ({ () => ({
validator(_, value) { validator(_, value) {
if (Array.isArray(value) && value.some((item) => !!item?.length)) { if (Array.isArray(value) && value.some((item) => !!item?.length)) {
const errs = [];
value.forEach((arr, i) => {
if (arr?.length && !arr.includes(permissions[i].essentialPermission.value)) {
errs.push(`[${permissions[i].essentialPermission.label}]`);
}
});
if (errs.length) {
return Promise.reject(`您必须分配 ${errs.join(' 和 ')} 权限`);
}
return Promise.resolve(); return Promise.resolve();
} }
return Promise.reject(new Error('请为角色至少分配一项权限')); return Promise.reject(new Error('请为角色至少分配一项权限'));
@@ -589,38 +611,45 @@ export default (props: { curTabKey: string }): JSX.Element => {
return ( return (
<> <>
<div className="operate-bar-right"> <div className="operate-bar">
<Input <div className="left">
className="search-input" <div className="refresh-icon" onClick={() => getRoleList()}>
suffix={ <IconFont className="icon" type="icon-shuaxin1" />
<IconFont </div>
type="icon-fangdajing" </div>
onClick={(_) => { <div className="right">
setSearchKeywords(searchKeywordsInput); <Input
}} className="search-input"
style={{ fontSize: '16px' }} suffix={
/> <IconFont
} type="icon-fangdajing"
placeholder="请输入角色名称" onClick={(_) => {
value={searchKeywordsInput} setSearchKeywords(searchKeywordsInput);
onPressEnter={(_) => { }}
setSearchKeywords(searchKeywordsInput); style={{ fontSize: '16px' }}
}} />
onChange={(e) => { }
setSearchKeywordsInput(e.target.value); placeholder="请输入角色名称"
}} value={searchKeywordsInput}
/> onPressEnter={(_) => {
{global.hasPermission && global.hasPermission(ConfigPermissionMap.ROLE_ADD) ? ( setSearchKeywords(searchKeywordsInput);
<Button }}
type="primary" onChange={(e) => {
icon={<PlusOutlined />} setSearchKeywordsInput(e.target.value);
onClick={() => detailRef.current.onOpen(true, RoleOperate.Add, getRoleList, undefined)} }}
> />
{global.hasPermission && global.hasPermission(ConfigPermissionMap.ROLE_ADD) ? (
</Button> <Button
) : ( type="primary"
<></> icon={<PlusOutlined />}
)} onClick={() => detailRef.current.onOpen(true, RoleOperate.Add, getRoleList, undefined)}
>
</Button>
) : (
<></>
)}
</div>
</div> </div>
<ProTable <ProTable

View File

@@ -1,12 +1,13 @@
import React, { forwardRef, useCallback, useEffect, useImperativeHandle, useRef, useState } from 'react'; import React, { forwardRef, useCallback, useEffect, useImperativeHandle, useRef, useState } from 'react';
import { Form, ProTable, Select, Button, Input, Modal, message, Drawer, Space, Divider, AppContainer, Utils } from 'knowdesign'; import { Form, ProTable, Select, Button, Input, Modal, message, Drawer, Space, Divider, AppContainer, Utils } from 'knowdesign';
import { IconFont } from '@knowdesign/icons';
import { PlusOutlined, QuestionCircleOutlined } from '@ant-design/icons'; import { PlusOutlined, QuestionCircleOutlined } from '@ant-design/icons';
import moment from 'moment'; import moment from 'moment';
import { defaultPagination } from 'constants/common'; import { defaultPagination } from '@src/constants/common';
import { UserProps, UserOperate } from './config'; import { UserProps, UserOperate } from './config';
import CheckboxGroupContainer from './CheckboxGroupContainer'; import CheckboxGroupContainer from './CheckboxGroupContainer';
import TagsWithHide from '../../components/TagsWithHide/index'; import TagsWithHide from '../../components/TagsWithHide/index';
import api from 'api'; import api from '@src/api';
import { ConfigPermissionMap } from '../CommonConfig'; import { ConfigPermissionMap } from '../CommonConfig';
const { confirm } = Modal; const { confirm } = Modal;
@@ -341,22 +342,29 @@ export default (props: { curTabKey: string }) => {
return ( return (
<> <>
<div className="operate-bar"> <div className="operate-bar">
<Form form={form} layout="inline" onFinish={() => getUserList({ page: 1 })}> <div className="left">
<Form.Item name="userName"> <div className="refresh-icon" onClick={() => getUserList()}>
<Input placeholder="请输入用户账号" /> <IconFont className="icon" type="icon-shuaxin1" />
</Form.Item> </div>
<Form.Item name="realName"> <Divider type="vertical" style={{ height: 20, top: 0 }} />
<Input placeholder="请输入用户实名" />
</Form.Item> <Form form={form} layout="inline" onFinish={() => getUserList({ page: 1 })}>
<Form.Item name="roleId"> <Form.Item name="userName">
<Select style={{ width: 190 }} placeholder="选择平台已创建的角色名" options={simpleRoleList} /> <Input placeholder="请输入用户账号" />
</Form.Item> </Form.Item>
<Form.Item> <Form.Item name="realName">
<Button type="primary" ghost htmlType="submit"> <Input placeholder="请输入用户实名" />
</Form.Item>
</Button> <Form.Item name="roleId">
</Form.Item> <Select style={{ width: 190 }} placeholder="选择平台已创建的角色名" options={simpleRoleList} />
</Form> </Form.Item>
<Form.Item>
<Button type="primary" ghost htmlType="submit">
</Button>
</Form.Item>
</Form>
</div>
{global.hasPermission && global.hasPermission(ConfigPermissionMap.USER_ADD) ? ( {global.hasPermission && global.hasPermission(ConfigPermissionMap.USER_ADD) ? (
<Button <Button
type="primary" type="primary"

View File

@@ -59,5 +59,6 @@ export enum RoleOperate {
export interface FormItemPermission { export interface FormItemPermission {
id: number; id: number;
name: string; name: string;
essentialPermission: { label: string; value: number };
options: { label: string; value: number }[]; options: { label: string; value: number }[];
} }

View File

@@ -44,13 +44,3 @@
.role-tab-assign-user .desc-row { .role-tab-assign-user .desc-row {
margin-bottom: 24px; margin-bottom: 24px;
} }
.operate-bar-right {
display: flex;
justify-content: right;
margin-bottom: 12px;
.search-input {
width: 248px;
margin-right: 8px;
}
}

View File

@@ -1,56 +1,9 @@
/* eslint-disable */
const path = require('path'); const path = require('path');
require('dotenv').config({ path: path.resolve(process.cwd(), '../../.env') }); require('dotenv').config({ path: path.resolve(process.cwd(), '../../.env') });
const isProd = process.env.NODE_ENV === 'production';
const HtmlWebpackPlugin = require('html-webpack-plugin');
const webpack = require('webpack');
const merge = require('webpack-merge'); const merge = require('webpack-merge');
const pkgJson = require('./package'); const devMode = process.env.NODE_ENV === 'development';
const getWebpackCommonConfig = require('./config/d1-webpack.base'); const commonConfig = require('./config/webpack.common');
const outPath = path.resolve(__dirname, `../../../km-rest/src/main/resources/templates/${pkgJson.ident}`); const devConfig = require('./config/webpack.dev');
const jsFileName = isProd ? '[name]-[chunkhash].js' : '[name].js'; const prodConfig = require('./config/webpack.prod');
module.exports = merge(getWebpackCommonConfig(), { module.exports = merge(commonConfig, devMode ? devConfig : prodConfig);
mode: isProd ? 'production' : 'development',
entry: {
[pkgJson.ident]: ['./src/index.tsx'],
},
plugins: [
new webpack.DefinePlugin({
'process.env': {
NODE_ENV: JSON.stringify(process.env.NODE_ENV),
RUN_ENV: JSON.stringify(process.env.RUN_ENV),
},
}),
new HtmlWebpackPlugin({
meta: {
manifest: 'manifest.json',
},
template: './src/index.html',
inject: 'body',
}),
],
output: {
path: outPath,
publicPath: isProd ? `${process.env.PUBLIC_PATH}/${pkgJson.ident}/` : `http://localhost:${pkgJson.port}/${pkgJson.ident}/`,
library: pkgJson.ident,
libraryTarget: 'amd',
filename: jsFileName,
chunkFilename: jsFileName,
},
devtool: isProd ? 'none' : 'cheap-module-eval-source-map',
devServer: {
host: '127.0.0.1',
port: pkgJson.port,
hot: true,
open: false,
publicPath: `http://localhost:${pkgJson.port}/${pkgJson.ident}/`,
inline: true,
disableHostCheck: true,
historyApiFallback: true,
headers: {
'Access-Control-Allow-Origin': '*',
},
proxy: {},
},
});

View File

@@ -9,6 +9,5 @@ build/
coverage coverage
versions/ versions/
debug.log debug.log
package-lock.json
yarn.lock yarn.lock
.d1-workspace.json .d1-workspace.json

View File

@@ -1,17 +1,21 @@
## 使用说明 ## 使用说明
### 依赖安装: ### 依赖安装(如在 km-console 目录下执行 npm run i 安装过依赖,这步可以省略)
``` ```
npm install npm install
``` ```
注意,这种方式只会安装当前应用的依赖。如果您不了解,推荐在 km-console 目录下执行 npm run i 安装依赖。
### 启动: ### 启动:
``` ```
npm run start npm run start
``` ```
启动后访问地址为 http://localhost:8000
### 构建: ### 构建:
``` ```

View File

@@ -1,183 +0,0 @@
/* eslint-disable */
const MiniCssExtractPlugin = require('mini-css-extract-plugin');
const ProgressBarPlugin = require('progress-bar-webpack-plugin');
const CaseSensitivePathsPlugin = require('case-sensitive-paths-webpack-plugin');
const { CleanWebpackPlugin } = require('clean-webpack-plugin');
const CoverHtmlWebpackPlugin = require('./CoverHtmlWebpackPlugin.js');
var webpackConfigResolveAlias = require('./webpackConfigResolveAlias');
const TerserJSPlugin = require('terser-webpack-plugin');
const OptimizeCSSAssetsPlugin = require('optimize-css-assets-webpack-plugin');
const theme = require('./theme');
const ReactRefreshWebpackPlugin = require('@pmmmwh/react-refresh-webpack-plugin');
const isProd = process.env.NODE_ENV === 'production';
const babelOptions = {
cacheDirectory: true,
babelrc: false,
presets: [require.resolve('@babel/preset-env'), require.resolve('@babel/preset-typescript'), require.resolve('@babel/preset-react')],
plugins: [
[require.resolve('@babel/plugin-proposal-decorators'), { legacy: true }],
[require.resolve('@babel/plugin-proposal-class-properties'), { loose: true }],
[require.resolve('@babel/plugin-proposal-private-property-in-object'), { loose: true }],
[require.resolve('@babel/plugin-proposal-private-methods'), { loose: true }],
require.resolve('@babel/plugin-proposal-export-default-from'),
require.resolve('@babel/plugin-proposal-export-namespace-from'),
require.resolve('@babel/plugin-proposal-object-rest-spread'),
require.resolve('@babel/plugin-transform-runtime'),
!isProd && require.resolve('react-refresh/babel'),
]
.filter(Boolean)
.concat([
[
'babel-plugin-import',
{
libraryName: 'antd',
style: true,
},
],
'@babel/plugin-transform-object-assign',
]),
};
module.exports = () => {
const cssFileName = isProd ? '[name]-[chunkhash].css' : '[name].css';
const plugins = [
new CoverHtmlWebpackPlugin(),
new ProgressBarPlugin(),
new CaseSensitivePathsPlugin(),
new MiniCssExtractPlugin({
filename: cssFileName,
}),
!isProd &&
new ReactRefreshWebpackPlugin({
overlay: false,
}),
].filter(Boolean);
const resolve = {
symlinks: false,
extensions: ['.web.jsx', '.web.js', '.ts', '.tsx', '.js', '.jsx', '.json'],
alias: webpackConfigResolveAlias,
};
if (isProd) {
plugins.push(new CleanWebpackPlugin());
}
if (!isProd) {
resolve.mainFields = ['module', 'browser', 'main'];
}
return {
externals: isProd
? [
/^react$/,
/^react\/lib.*/,
/^react-dom$/,
/.*react-dom.*/,
/^single-spa$/,
/^single-spa-react$/,
/^moment$/,
/^antd$/,
/^lodash$/,
/^echarts$/,
/^react-router$/,
/^react-router-dom$/,
]
: [],
resolve,
plugins,
module: {
rules: [
{
parser: { system: false },
},
{
test: /\.(js|jsx)$/,
exclude: /node_modules/,
use: [
{
loader: 'babel-loader',
options: babelOptions,
},
],
},
{
test: /\.(ts|tsx)$/,
use: [
{
loader: 'babel-loader',
options: babelOptions,
},
{
loader: 'ts-loader',
options: {
allowTsInNodeModules: true,
},
},
],
},
{
test: /\.(png|svg|jpeg|jpg|gif|ttf|woff|woff2|eot|pdf|otf)$/,
use: [
{
loader: 'file-loader',
options: {
name: '[name].[ext]',
outputPath: './assets/image/',
esModule: false,
},
},
],
},
{
test: /\.(css|less)$/,
use: [
{
loader: MiniCssExtractPlugin.loader,
},
'css-loader',
{
loader: 'less-loader',
options: {
javascriptEnabled: true,
modifyVars: theme,
},
},
],
},
],
},
optimization: Object.assign(
isProd
? {
splitChunks: {
cacheGroups: {
vendor: {
test: /[\\/]node_modules[\\/]/,
chunks: 'all',
name: 'vendor',
priority: 10,
enforce: true,
minChunks: 1,
maxSize: 3000000,
},
},
},
minimizer: [
new TerserJSPlugin({
cache: true,
sourceMap: true,
}),
new OptimizeCSSAssetsPlugin({}),
],
}
: {}
),
devtool: isProd ? 'cheap-module-source-map' : '',
node: {
fs: 'empty',
net: 'empty',
tls: 'empty',
},
};
};

View File

@@ -0,0 +1,123 @@
const path = require('path');
const theme = require('./theme');
const webpack = require('webpack');
const HtmlWebpackPlugin = require('html-webpack-plugin');
const MiniCssExtractPlugin = require('mini-css-extract-plugin');
const ProgressBarPlugin = require('progress-bar-webpack-plugin');
const CoverHtmlWebpackPlugin = require('./CoverHtmlWebpackPlugin.js');
const CaseSensitivePathsPlugin = require('case-sensitive-paths-webpack-plugin');
const devMode = process.env.NODE_ENV === 'development';
const babelOptions = {
cacheDirectory: true,
babelrc: false,
presets: [require.resolve('@babel/preset-env'), require.resolve('@babel/preset-typescript'), require.resolve('@babel/preset-react')],
plugins: [
[require.resolve('@babel/plugin-proposal-decorators'), { legacy: true }],
[require.resolve('@babel/plugin-proposal-class-properties'), { loose: true }],
[require.resolve('@babel/plugin-proposal-private-property-in-object'), { loose: true }],
[require.resolve('@babel/plugin-proposal-private-methods'), { loose: true }],
require.resolve('@babel/plugin-proposal-export-default-from'),
require.resolve('@babel/plugin-proposal-export-namespace-from'),
require.resolve('@babel/plugin-proposal-object-rest-spread'),
require.resolve('@babel/plugin-transform-runtime'),
devMode && require.resolve('react-refresh/babel'),
devMode && [
'babel-plugin-import',
{
libraryName: 'antd',
style: true,
},
],
].filter(Boolean),
};
module.exports = {
entry: {
layout: ['./src/index.tsx'],
},
resolve: {
symlinks: false,
extensions: ['.web.jsx', '.web.js', '.ts', '.tsx', '.js', '.jsx', '.json'],
alias: {
'@src': path.resolve('src'),
},
},
plugins: [
new CoverHtmlWebpackPlugin(),
new ProgressBarPlugin(),
new CaseSensitivePathsPlugin(),
new webpack.DefinePlugin({
'process.env': {
NODE_ENV: JSON.stringify(process.env.NODE_ENV),
RUN_ENV: JSON.stringify(process.env.RUN_ENV),
BUSINESS_VERSION: process.env.BUSINESS_VERSION === 'true',
PUBLIC_PATH: JSON.stringify(process.env.PUBLIC_PATH),
},
}),
new HtmlWebpackPlugin({
meta: {
manifest: 'manifest.json',
},
template: './src/index.html',
favicon: path.resolve('favicon.ico'),
inject: 'body',
}),
],
module: {
rules: [
{
parser: { system: false },
},
{
test: /\.(js|jsx|ts|tsx)$/,
exclude: /node_modules/,
use: [
{
loader: 'babel-loader',
options: babelOptions,
},
{
loader: 'ts-loader',
options: {
allowTsInNodeModules: true,
},
},
],
},
{
test: /\.(png|svg|jpeg|jpg|gif|ttf|woff|woff2|eot|pdf|otf)$/,
use: [
{
loader: 'file-loader',
options: {
name: '[name].[ext]',
outputPath: './assets/image/',
esModule: false,
},
},
],
},
{
test: /\.(css|less)$/,
use: [
MiniCssExtractPlugin.loader,
'css-loader',
{
loader: 'less-loader',
options: {
javascriptEnabled: true,
modifyVars: theme,
},
},
],
},
],
},
node: {
fs: 'empty',
net: 'empty',
tls: 'empty',
},
stats: 'errors-warnings',
};

View File

@@ -0,0 +1,45 @@
const MiniCssExtractPlugin = require('mini-css-extract-plugin');
const ReactRefreshWebpackPlugin = require('@pmmmwh/react-refresh-webpack-plugin');
module.exports = {
mode: 'development',
plugins: [
new MiniCssExtractPlugin(),
new ReactRefreshWebpackPlugin({
overlay: false,
}),
],
output: {
path: '/',
publicPath: '/',
filename: '[name].js',
chunkFilename: '[name].js',
library: 'layout',
libraryTarget: 'amd',
},
devServer: {
host: 'localhost',
port: 8000,
hot: true,
open: true,
openPage: 'http://localhost:8000/',
inline: true,
historyApiFallback: true,
publicPath: `http://localhost:8000/`,
headers: {
'cache-control': 'no-cache',
pragma: 'no-cache',
'Access-Control-Allow-Origin': '*',
},
proxy: {
'/ks-km/api/v3': {
changeOrigin: true,
target: 'http://localhost:8080/',
},
'/logi-security/api/v1': {
changeOrigin: true,
target: 'http://localhost:8080/',
},
},
},
};

View File

@@ -0,0 +1,79 @@
const path = require('path');
const { CleanWebpackPlugin } = require('clean-webpack-plugin');
const CountPlugin = require('./CountComponentWebpackPlugin');
const MiniCssExtractPlugin = require('mini-css-extract-plugin');
const CopyWebpackPlugin = require('copy-webpack-plugin');
const TerserJSPlugin = require('terser-webpack-plugin');
const OptimizeCSSAssetsPlugin = require('optimize-css-assets-webpack-plugin');
const outputPath = path.resolve(process.cwd(), `../../../km-rest/src/main/resources/templates/layout`);
module.exports = {
mode: 'production',
plugins: [
new CleanWebpackPlugin(),
new CountPlugin({
pathname: 'knowdesign',
startCount: true,
isExportExcel: false,
}),
new MiniCssExtractPlugin({
filename: '[name]-[chunkhash].css',
}),
new CopyWebpackPlugin([
{
from: path.resolve(process.cwd(), 'static'),
to: path.resolve(outputPath, '../static'),
},
{
from: path.resolve(process.cwd(), 'favicon.ico'),
to: path.resolve(outputPath, '../favicon.ico'),
},
]),
],
externals: [
/^react$/,
/^react\/lib.*/,
/^react-dom$/,
/.*react-dom.*/,
/^single-spa$/,
/^single-spa-react$/,
/^moment$/,
/^antd$/,
/^lodash$/,
/^echarts$/,
/^react-router$/,
/^react-router-dom$/,
],
output: {
path: outputPath,
publicPath: process.env.PUBLIC_PATH + '/layout/',
filename: '[name]-[chunkhash].js',
chunkFilename: '[name]-[chunkhash].js',
library: 'layout',
libraryTarget: 'amd',
},
optimization: {
splitChunks: {
cacheGroups: {
vendor: {
test: /[\\/]node_modules[\\/]/,
chunks: 'all',
name: 'vendor',
priority: 10,
enforce: true,
minChunks: 1,
maxSize: 3000000,
},
},
},
minimizer: [
new TerserJSPlugin({
cache: true,
sourceMap: true,
}),
new OptimizeCSSAssetsPlugin({}),
],
},
devtool: 'none',
};

View File

@@ -1,5 +0,0 @@
var path = require('path');
module.exports = {
react: path.resolve('./node_modules/react'),
};

File diff suppressed because it is too large Load Diff

View File

@@ -17,7 +17,7 @@
"scripts": { "scripts": {
"test": "echo \"Error: run tests from root\" && exit 1", "test": "echo \"Error: run tests from root\" && exit 1",
"start": "cross-env NODE_ENV=development webpack-dev-server", "start": "cross-env NODE_ENV=development webpack-dev-server",
"build": "rm -rf ../../pub/layout & cross-env NODE_ENV=production webpack --max_old_space_size=8000" "build": "cross-env NODE_ENV=production webpack --max_old_space_size=8000"
}, },
"browserslist": { "browserslist": {
"production": [ "production": [
@@ -35,10 +35,12 @@
"dependencies": { "dependencies": {
"@ant-design/compatible": "^1.0.8", "@ant-design/compatible": "^1.0.8",
"@ant-design/icons": "^4.6.2", "@ant-design/icons": "^4.6.2",
"@knowdesign/icons": "^1.0.1",
"@types/react": "^17.0.39", "@types/react": "^17.0.39",
"@types/react-copy-to-clipboard": "^5.0.2", "@types/react-copy-to-clipboard": "^5.0.2",
"@types/react-dom": "^17.0.11", "@types/react-dom": "^17.0.11",
"@types/react-highlight-words": "^0.16.0", "@types/react-highlight-words": "^0.16.0",
"@types/react-router": "5.1.18",
"@types/react-router-dom": "^5.3.3", "@types/react-router-dom": "^5.3.3",
"@types/react-transition-group": "^4.2.2", "@types/react-transition-group": "^4.2.2",
"@types/react-virtualized": "^9.21.13", "@types/react-virtualized": "^9.21.13",
@@ -48,6 +50,7 @@
"crypto-js": "^4.1.1", "crypto-js": "^4.1.1",
"dotenv": "^16.0.1", "dotenv": "^16.0.1",
"html-webpack-plugin": "^4.0.0", "html-webpack-plugin": "^4.0.0",
"knowdesign": "^1.3.7",
"lodash": "^4.17.21", "lodash": "^4.17.21",
"moment": "^2.24.0", "moment": "^2.24.0",
"react": "16.12.0", "react": "16.12.0",
@@ -55,12 +58,10 @@
"react-cron-antd": "^1.1.2", "react-cron-antd": "^1.1.2",
"react-dom": "16.12.0", "react-dom": "16.12.0",
"react-intl": "^3.2.1", "react-intl": "^3.2.1",
"react-joyride": "^2.5.0", "react-joyride": "^2.5.3",
"single-spa": "5.9.3", "single-spa": "5.9.3",
"single-spa-react": "2.14.0", "single-spa-react": "2.14.0",
"webpack-bundle-analyzer": "^4.5.0", "webpack-bundle-analyzer": "^4.5.0"
"knowdesign": "1.3.7",
"tree-changes": "0.9.1"
}, },
"devDependencies": { "devDependencies": {
"@babel/core": "^7.5.5", "@babel/core": "^7.5.5",

View File

@@ -3,7 +3,8 @@ import '@babel/polyfill';
import React, { useState, useEffect, useLayoutEffect } from 'react'; import React, { useState, useEffect, useLayoutEffect } from 'react';
import { BrowserRouter, Switch, Route, useLocation, useHistory } from 'react-router-dom'; import { BrowserRouter, Switch, Route, useLocation, useHistory } from 'react-router-dom';
import { get as lodashGet } from 'lodash'; import { get as lodashGet } from 'lodash';
import { DProLayout, AppContainer, IconFont, Menu, Utils, Page403, Page404, Page500, Modal } from 'knowdesign'; import { DProLayout, AppContainer, Menu, Utils, Page403, Page404, Page500, Modal } from 'knowdesign';
import { IconFont } from '@knowdesign/icons';
import dantdZhCN from 'knowdesign/es/locale/zh_CN'; import dantdZhCN from 'knowdesign/es/locale/zh_CN';
import dantdEnUS from 'knowdesign/es/locale/en_US'; import dantdEnUS from 'knowdesign/es/locale/en_US';
import { DotChartOutlined } from '@ant-design/icons'; import { DotChartOutlined } from '@ant-design/icons';
@@ -59,6 +60,7 @@ const logout = () => {
}).then((res) => { }).then((res) => {
window.location.href = '/login'; window.location.href = '/login';
}); });
localStorage.removeItem('userInfo');
}; };
const LicenseLimitModal = () => { const LicenseLimitModal = () => {
@@ -117,7 +119,7 @@ const AppContent = (props: { setlanguage: (language: string) => void }) => {
<DProLayout.Container <DProLayout.Container
headerProps={{ headerProps={{
title: ( title: (
<div> <div style={{ cursor: 'pointer' }}>
<img className="header-logo" src={ksLogo} /> <img className="header-logo" src={ksLogo} />
</div> </div>
), ),

View File

@@ -90,7 +90,7 @@ export default () => {
return ( return (
<div> <div>
<span style={{ display: 'inline-block', marginRight: '8px' }}>Similar Config</span> <span style={{ display: 'inline-block', marginRight: '8px' }}>Similar Config</span>
<Tooltip overlayClassName="rebalance-tooltip" title="所有broker配置是否一致"> <Tooltip overlayClassName="rebalance-tooltip" title="所有Broker配置是否一致">
<QuestionCircleOutlined /> <QuestionCircleOutlined />
</Tooltip> </Tooltip>
</div> </div>
@@ -111,7 +111,7 @@ export default () => {
]; ];
setCardData(cordRightMap); setCardData(cordRightMap);
}); });
Promise.all([brokerMetric, brokersState]).then((res) => { Promise.all([brokerMetric, brokersState]).finally(() => {
setLoading(false); setLoading(false);
}); });
}, [routeParams.clusterId]); }, [routeParams.clusterId]);

View File

@@ -1,12 +1,14 @@
import React, { useState, useEffect } from 'react'; import React, { useState, useEffect } from 'react';
import { useParams } from 'react-router-dom'; import { useParams } from 'react-router-dom';
import CardBar from './index'; import CardBar from './index';
import { IconFont, Tag, Utils, Tooltip, Popover } from 'knowdesign'; import { Tag, Utils, Tooltip, Popover, AppContainer } from 'knowdesign';
import { IconFont } from '@knowdesign/icons';
import api from '@src/api'; import api from '@src/api';
import StateChart from './StateChart'; import StateChart from './StateChart';
import ClusterNorms from '@src/pages/LoadRebalance/ClusterNorms'; import ClusterNorms from '@src/pages/LoadRebalance/ClusterNorms';
import { QuestionCircleOutlined } from '@ant-design/icons'; import { QuestionCircleOutlined } from '@ant-design/icons';
import moment from 'moment'; import moment from 'moment';
import { ClustersPermissionMap } from '@src/pages/CommonConfig';
const transUnitTimePro = (ms: number, num = 0) => { const transUnitTimePro = (ms: number, num = 0) => {
if (!ms) return ''; if (!ms) return '';
@@ -23,6 +25,7 @@ const transUnitTimePro = (ms: number, num = 0) => {
}; };
const LoadRebalanceCardBar = (props: any) => { const LoadRebalanceCardBar = (props: any) => {
const [global] = AppContainer.useGlobalValue();
const { clusterId } = useParams<{ const { clusterId } = useParams<{
clusterId: string; clusterId: string;
}>(); }>();
@@ -53,12 +56,14 @@ const LoadRebalanceCardBar = (props: any) => {
return ( return (
<div style={{ height: '20px' }}> <div style={{ height: '20px' }}>
<span style={{ display: 'inline-block', marginRight: '8px' }}>State</span> <span style={{ display: 'inline-block', marginRight: '8px' }}>State</span>
<IconFont {global.hasPermission(ClustersPermissionMap.REBALANCE_SETTING) && (
className="cutomIcon-config" <IconFont
style={{ fontSize: '15px' }} className="cutomIcon-config"
onClick={() => setNormsVisible(true)} style={{ fontSize: '15px' }}
type="icon-shezhi" onClick={() => setNormsVisible(true)}
></IconFont> type="icon-shezhi"
></IconFont>
)}
</div> </div>
); );
}, },
@@ -134,15 +139,15 @@ const LoadRebalanceCardBar = (props: any) => {
// content={ // content={
// <div style={{ color: '#495057' }}> // <div style={{ color: '#495057' }}>
// <div> // <div>
// <IconFont className="cutomIcon" type="icon-chaoguo" /> // <IconFont className="cutomIcon cutomIcon-red" type="icon-chaoguo" />
// 超过均衡区间的有: {cpu?.bigNu || 0} // 超过均衡区间的有: {cpu?.bigNu || 0}
// </div> // </div>
// <div style={{ margin: '6px 0' }}> // <div style={{ margin: '6px 0' }}>
// <IconFont className="cutomIcon" type="icon-qujian" /> // <IconFont className="cutomIcon cutomIcon-green" type="icon-qujian" />
// 在均衡区间内的有: {cpu?.betweenNu || 0} // 在均衡区间内的有: {cpu?.betweenNu || 0}
// </div> // </div>
// <div> // <div>
// <IconFont className="cutomIcon" type="icon-diyu" /> // <IconFont className="cutomIcon cutomIcon-red" type="icon-diyu" />
// 低于均衡区间的有: {cpu?.smallNu || 0} // 低于均衡区间的有: {cpu?.smallNu || 0}
// </div> // </div>
// </div> // </div>
@@ -198,15 +203,15 @@ const LoadRebalanceCardBar = (props: any) => {
content={ content={
<div style={{ color: '#495057' }}> <div style={{ color: '#495057' }}>
<div> <div>
<IconFont className="cutomIcon" type="icon-chaoguo" /> <IconFont className="cutomIcon cutomIcon-red" type="icon-chaoguo" />
: {disk?.bigNu || 0} : {disk?.bigNu || 0}
</div> </div>
<div style={{ margin: '6px 0' }}> <div style={{ margin: '6px 0' }}>
<IconFont className="cutomIcon" type="icon-qujian" /> <IconFont className="cutomIcon cutomIcon-green" type="icon-qujian" />
: {disk?.betweenNu || 0} : {disk?.betweenNu || 0}
</div> </div>
<div> <div>
<IconFont className="cutomIcon" type="icon-diyu" /> <IconFont className="cutomIcon cutomIcon-red" type="icon-diyu" />
: {disk?.smallNu || 0} : {disk?.smallNu || 0}
</div> </div>
</div> </div>
@@ -263,15 +268,15 @@ const LoadRebalanceCardBar = (props: any) => {
content={ content={
<div style={{ color: '#495057' }}> <div style={{ color: '#495057' }}>
<div> <div>
<IconFont className="cutomIcon" type="icon-chaoguo" /> <IconFont className="cutomIcon cutomIcon-red" type="icon-chaoguo" />
: {bytesIn?.bigNu || 0} : {bytesIn?.bigNu || 0}
</div> </div>
<div style={{ margin: '6px 0' }}> <div style={{ margin: '6px 0' }}>
<IconFont className="cutomIcon" type="icon-qujian" /> <IconFont className="cutomIcon cutomIcon-green" type="icon-qujian" />
: {bytesIn?.betweenNu || 0} : {bytesIn?.betweenNu || 0}
</div> </div>
<div> <div>
<IconFont className="cutomIcon" type="icon-diyu" /> <IconFont className="cutomIcon cutomIcon-red" type="icon-diyu" />
: {bytesIn?.smallNu || 0} : {bytesIn?.smallNu || 0}
</div> </div>
</div> </div>
@@ -328,15 +333,15 @@ const LoadRebalanceCardBar = (props: any) => {
content={ content={
<div style={{ color: '#495057' }}> <div style={{ color: '#495057' }}>
<div> <div>
<IconFont className="cutomIcon" type="icon-chaoguo" /> <IconFont className="cutomIcon cutomIcon-red" type="icon-chaoguo" />
: {bytesOut?.bigNu || 0} : {bytesOut?.bigNu || 0}
</div> </div>
<div style={{ margin: '6px 0' }}> <div style={{ margin: '6px 0' }}>
<IconFont className="cutomIcon" type="icon-qujian" /> <IconFont className="cutomIcon cutomIcon-green" type="icon-qujian" />
: {bytesOut?.betweenNu || 0} : {bytesOut?.betweenNu || 0}
</div> </div>
<div> <div>
<IconFont className="cutomIcon" type="icon-diyu" /> <IconFont className="cutomIcon cutomIcon-red" type="icon-diyu" />
: {bytesOut?.smallNu || 0} : {bytesOut?.smallNu || 0}
</div> </div>
</div> </div>

View File

@@ -59,7 +59,7 @@ const EchartsExample = (props: any) => {
normal: { normal: {
color: (params: any) => { color: (params: any) => {
// 定义一个颜色数组colorList // 定义一个颜色数组colorList
const colorList = ['#00C0A2', '#CED4DA', '#FF7066']; const colorList = ['#FF7066', '#00C0A2', '#FF7066'];
return colorList[params.dataIndex]; return colorList[params.dataIndex];
}, },
}, },

View File

@@ -2,7 +2,8 @@ import React, { useState, useEffect } from 'react';
import { useParams } from 'react-router-dom'; import { useParams } from 'react-router-dom';
import CardBar from '@src/components/CardBar'; import CardBar from '@src/components/CardBar';
import { healthDataProps } from '.'; import { healthDataProps } from '.';
import { IconFont, Utils } from 'knowdesign'; import { Utils } from 'knowdesign';
import { IconFont } from '@knowdesign/icons';
import api from '@src/api'; import api from '@src/api';
import { healthScoreCondition } from './const'; import { healthScoreCondition } from './const';
import { hashDataParse } from '@src/constants/common'; import { hashDataParse } from '@src/constants/common';

View File

@@ -203,11 +203,19 @@
background: rgba(33, 37, 41, 0.04); background: rgba(33, 37, 41, 0.04);
} }
.cutomIcon { .anticon.cutomIcon {
display: inline-block; display: inline-block;
margin-right: 2px; margin-right: 2px;
} }
.anticon.cutomIcon-red {
color: #ff7066;
}
.anticon.cutomIcon-green {
color: #00c0a2;
}
.rebalance-tooltip { .rebalance-tooltip {
.dcloud-tooltip-inner { .dcloud-tooltip-inner {
min-height: 20px; min-height: 20px;

View File

@@ -1,6 +1,7 @@
import React, { useEffect, useState } from 'react'; import React, { useEffect, useState } from 'react';
import { useParams } from 'react-router-dom'; import { useParams } from 'react-router-dom';
import { Drawer, IconFont, Select, Spin, Table } from 'knowdesign'; import { Drawer, Select, Spin, Table } from 'knowdesign';
import { IconFont } from '@knowdesign/icons';
import { Utils, Progress } from 'knowdesign'; import { Utils, Progress } from 'knowdesign';
import './index.less'; import './index.less';
import api from '@src/api'; import api from '@src/api';
@@ -110,8 +111,8 @@ const CardBar = (props: CardBarProps) => {
const promise = record const promise = record
? Utils.request(path) ? Utils.request(path)
: Utils.request(path, { : Utils.request(path, {
params: { dimensionCode: sceneObj.code }, params: { dimensionCode: sceneObj.code },
}); });
promise.then((data: any[]) => { promise.then((data: any[]) => {
setHealthCheckDetailList(data); setHealthCheckDetailList(data);
}); });

View File

@@ -0,0 +1,32 @@
.content-with-copy {
display: flex;
align-items: center;
.content {
flex: 1;
display: inline-block;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
word-break: break-all;
}
.copy-icon {
width: 20px;
height: 20px;
padding-top: 2px;
border-radius: 50%;
margin-left: 4px;
font-size: 16px;
color: #adb5bc;
opacity: 0;
&:hover {
background: rgba(33, 37, 41, 0.04);
color: #74788d;
}
}
}
.dcloud-table-cell-row-hover {
.copy-icon {
opacity: 1;
}
}

View File

@@ -0,0 +1,35 @@
import { CheckCircleFilled } from '@ant-design/icons';
import { Tooltip } from 'knowdesign';
import React, { useState } from 'react';
import CopyToClipboard from 'react-copy-to-clipboard';
import { IconFont } from '@knowdesign/icons';
import './index.less';
const ContentWithCopy = (props: { content: string }) => {
const { content } = props;
const [visible, setVisible] = useState(false);
return (
<CopyToClipboard text={content}>
<div className="content-with-copy">
<Tooltip title={content}>
<span className="content">{content}</span>
</Tooltip>
{content && (
<Tooltip
title={
<span>
<CheckCircleFilled style={{ color: '#00b365' }} />
</span>
}
visible={visible}
onVisibleChange={() => setVisible(false)}
>
<IconFont className="copy-icon" type="icon-fuzhi" onClick={() => setVisible(true)} />
</Tooltip>
)}
</div>
</CopyToClipboard>
);
};
export default ContentWithCopy;

View File

@@ -1,5 +1,6 @@
import React, { useState } from 'react'; import React, { useState } from 'react';
import { Input, IconFont } from 'knowdesign'; import { Input } from 'knowdesign';
import { IconFont } from '@knowdesign/icons';
import './style/index.less'; import './style/index.less';
interface IObjectProps { interface IObjectProps {

View File

@@ -1,18 +1,21 @@
import React, { forwardRef, useEffect, useImperativeHandle, useRef, useState } from 'react'; import React, { forwardRef, useEffect, useImperativeHandle, useMemo, useRef, useState } from 'react';
import { AppContainer, Button, Drawer, IconFont, message, Spin, Table, SingleChart, Utils, Tooltip } from 'knowdesign'; import { AppContainer, Drawer, Spin, Table, SingleChart, Utils, Tooltip } from 'knowdesign';
import moment from 'moment'; import moment from 'moment';
import api, { MetricType } from '@src/api'; import api, { MetricType } from '@src/api';
import { MetricDefaultChartDataType, MetricChartDataType, formatChartData } from '@src/constants/chartConfig';
import { useParams } from 'react-router-dom'; import { useParams } from 'react-router-dom';
import { debounce } from 'lodash'; import { debounce } from 'lodash';
import { MetricDefaultChartDataType, MetricChartDataType, formatChartData, getDetailChartConfig } from './config'; import { getDetailChartConfig } from './config';
import { UNIT_MAP } from '@src/constants/chartConfig'; import { UNIT_MAP } from '@src/constants/chartConfig';
import { CloseOutlined } from '@ant-design/icons'; import RenderEmpty from '../RenderEmpty';
interface ChartDetailProps { interface ChartDetailProps {
metricType: MetricType; metricType: MetricType;
metricName: string; metricName: string;
queryLines: string[]; queryLines: string[];
onClose: () => void; setSliderRange: (range: string) => void;
// eslint-disable-next-line @typescript-eslint/ban-types
setDisposeChartInstance: Function;
} }
interface MetricTableInfo { interface MetricTableInfo {
@@ -24,6 +27,18 @@ interface MetricTableInfo {
color: string; color: string;
} }
interface ChartInfo {
chartInstance?: echarts.ECharts;
isLoadingAdditionData?: boolean;
isLoadedFullData?: boolean;
fullTimeRange?: readonly [number, number];
curTimeRange?: readonly [number, number];
sliderPos?: readonly [number, number];
transformUnit?: [string, number];
fullMetricData?: MetricChartDataType;
oldDataZoomOption?: any;
}
interface DataZoomEventProps { interface DataZoomEventProps {
type: 'datazoom'; type: 'datazoom';
// 缩放的开始位置的百分比0 - 100 // 缩放的开始位置的百分比0 - 100
@@ -34,12 +49,8 @@ interface DataZoomEventProps {
// 缩放区默认选中范围比例0.011 // 缩放区默认选中范围比例0.011
const DATA_ZOOM_DEFAULT_SCALE = 0.25; const DATA_ZOOM_DEFAULT_SCALE = 0.25;
// 选中范围最少展示的时间长度(默认 10 分钟),单位: ms
const LEAST_SELECTED_TIME_RANGE = 1 * 60 * 1000;
// 单次向服务器请求数据的范围(默认 6 小时,超过后采集频率间隔会变长),单位: ms // 单次向服务器请求数据的范围(默认 6 小时,超过后采集频率间隔会变长),单位: ms
const DEFAULT_REQUEST_TIME_RANGE = 6 * 60 * 60 * 1000; const DEFAULT_REQUEST_TIME_RANGE = 6 * 60 * 60 * 1000;
// 采样间隔,影响前端补点逻辑,单位: ms
const DEFAULT_POINT_INTERVAL = 60 * 1000;
// 向服务器每轮请求的数量 // 向服务器每轮请求的数量
const DEFAULT_REQUEST_COUNT = 6; const DEFAULT_REQUEST_COUNT = 6;
// 进入详情页默认展示的时间范围 // 进入详情页默认展示的时间范围
@@ -47,70 +58,15 @@ const DEFAULT_ENTER_TIME_RANGE = 2 * 60 * 60 * 1000;
// 预缓存数据阈值,图表展示数据的开始时间处于前端缓存数据的时间范围的前 40% 时,向服务器请求数据 // 预缓存数据阈值,图表展示数据的开始时间处于前端缓存数据的时间范围的前 40% 时,向服务器请求数据
const PRECACHE_THRESHOLD = 0.4; const PRECACHE_THRESHOLD = 0.4;
// 表格列
const colunms = [
{
title: 'Host',
dataIndex: 'name',
width: 200,
render(name: string, record: any) {
return (
<div style={{ display: 'flex', alignItems: 'center' }}>
<div style={{ width: 8, height: 2, marginRight: 4, background: record.color }}></div>
<span>{name}</span>
</div>
);
},
},
{
title: 'Avg',
dataIndex: 'avg',
width: 120,
render(num: number) {
return num.toFixed(2);
},
},
{
title: 'Max',
dataIndex: 'max',
width: 120,
render(num: number, record: any) {
return (
<div>
<span>{num.toFixed(2)}</span>
</div>
);
},
},
{
title: 'Min',
dataIndex: 'min',
width: 120,
render(num: number, record: any) {
return (
<div>
<span>{num.toFixed(2)}</span>
</div>
);
},
},
{
title: 'Latest',
dataIndex: 'latest',
width: 120,
render(latest: number[]) {
return `${latest[1].toFixed(2)}`;
},
},
];
const ChartDetail = (props: ChartDetailProps) => { const ChartDetail = (props: ChartDetailProps) => {
const [global] = AppContainer.useGlobalValue(); const [global] = AppContainer.useGlobalValue();
const { clusterId } = useParams<{ const { clusterId } = useParams<{
clusterId: string; clusterId: string;
}>(); }>();
const { metricType, metricName, queryLines, onClose } = props; const { metricType, metricName, queryLines, setSliderRange, setDisposeChartInstance } = props;
// 初始化拖拽防抖函数
const debouncedZoomDrag = useRef(null);
// 存储图表相关的不需要触发渲染的数据,用于计算图表展示状态并进行操作 // 存储图表相关的不需要触发渲染的数据,用于计算图表展示状态并进行操作
const chartInfo = useRef( const chartInfo = useRef(
(() => { (() => {
@@ -119,16 +75,16 @@ const ChartDetail = (props: ChartDetailProps) => {
const curTimeRange = [curTime - DEFAULT_ENTER_TIME_RANGE, curTime] as const; const curTimeRange = [curTime - DEFAULT_ENTER_TIME_RANGE, curTime] as const;
return { return {
chartInstance: undefined as echarts.ECharts, chartInstance: undefined,
isLoadingAdditionData: false,
isLoadedFullData: false, isLoadedFullData: false,
fullTimeRange: curTimeRange, fullTimeRange: curTimeRange,
fullMetricData: {} as MetricChartDataType, fullMetricData: {} as MetricChartDataType,
curTimeRange, curTimeRange,
oldDataZoomOption: {} as any, oldDataZoomOption: {},
sliderPos: [0, 0] as readonly [number, number], sliderPos: [0, 0],
sliderRange: '', transformUnit: undefined,
transformUnit: undefined as [string, number], } as ChartInfo;
};
})() })()
); );
@@ -137,8 +93,76 @@ const ChartDetail = (props: ChartDetailProps) => {
const [curMetricData, setCurMetricData] = useState<MetricChartDataType>(); const [curMetricData, setCurMetricData] = useState<MetricChartDataType>();
// 图表数据的各项计算指标 // 图表数据的各项计算指标
const [tableInfo, setTableInfo] = useState<MetricTableInfo[]>([]); const [tableInfo, setTableInfo] = useState<MetricTableInfo[]>([]);
// 选中展示的图表 const [linesStatus, setLinesStatus] = useState<{
const [selectedLines, setSelectedLines] = useState<string[]>([]); [lineName: string]: boolean;
}>({});
// 表格列
const colunms = useMemo(
() => [
{
title: metricType === MetricType.Broker ? 'Host' : 'Topic',
dataIndex: 'name',
width: 200,
render(name: string, record: any) {
return (
<div style={{ display: 'flex', alignItems: 'center' }}>
<div style={{ width: 8, height: 2, marginRight: 4, background: record.color }}></div>
<span>{name}</span>
</div>
);
},
},
{
title: 'Avg',
dataIndex: 'avg',
width: 120,
render(num: number) {
return num.toFixed(2);
},
},
{
title: 'Max',
dataIndex: 'max',
width: 120,
render(num: number, record: any) {
return (
<div>
<span>{num.toFixed(2)}</span>
</div>
);
},
},
{
title: 'Min',
dataIndex: 'min',
width: 120,
render(num: number, record: any) {
return (
<div>
<span>{num.toFixed(2)}</span>
</div>
);
},
},
{
title: 'Latest',
dataIndex: 'latest',
width: 120,
render(latest: number[]) {
return `${latest[1].toFixed(2)}`;
},
},
],
[metricType]
);
const updateChartInfo = (changedInfo: ChartInfo) => {
chartInfo.current = {
...chartInfo.current,
...changedInfo,
};
};
// 请求图表数据 // 请求图表数据
const getMetricChartData = ([startTime, endTime]: readonly [number, number]) => { const getMetricChartData = ([startTime, endTime]: readonly [number, number]) => {
@@ -175,11 +199,10 @@ const ChartDetail = (props: ChartDetailProps) => {
// 如果滑块整体拖动,则只更新拖动后滑块的位(保留小数点后三位是防止低位值的干扰) // 如果滑块整体拖动,则只更新拖动后滑块的位(保留小数点后三位是防止低位值的干扰)
if (oldScale.toFixed(3) === newScale.toFixed(3)) { if (oldScale.toFixed(3) === newScale.toFixed(3)) {
chartInfo.current = { updateChartInfo({
...chartInfo.current,
sliderPos: [newStartSliderPos, newEndSliderPos], sliderPos: [newStartSliderPos, newEndSliderPos],
oldDataZoomOption: newDataZoomOption, oldDataZoomOption: newDataZoomOption,
}; });
renderTableInfo(); renderTableInfo();
return false; return false;
@@ -217,23 +240,14 @@ const ChartDetail = (props: ChartDetailProps) => {
} }
} else { } else {
// 3. 滑块拖动后缩放比例变小 // 3. 滑块拖动后缩放比例变小
// 判断拖动后选择的时间范围并提示
if (newEndSliderPos - newStartSliderPos < LEAST_SELECTED_TIME_RANGE) {
// TODO: 补充逻辑
updateChartData([oldStartTimestamp, oldEndTimestamp], [oldStartSliderPos, oldEndSliderPos]);
message.warning(`当前选择范围小于 ${LEAST_SELECTED_TIME_RANGE / 60 / 1000} 分钟,图表可能无数据`);
return true;
}
const isOldLarger = oldScale - DATA_ZOOM_DEFAULT_SCALE > 0.01; const isOldLarger = oldScale - DATA_ZOOM_DEFAULT_SCALE > 0.01;
const isNewLarger = newScale - DATA_ZOOM_DEFAULT_SCALE > 0.01; const isNewLarger = newScale - DATA_ZOOM_DEFAULT_SCALE > 0.01;
if (isOldLarger && isNewLarger) { if (isOldLarger && isNewLarger) {
// 如果拖拽前后比例均高于默认比例,则不对图表展示范围进行操作 // 如果拖拽前后比例均高于默认比例,则不对图表展示范围进行操作
chartInfo.current = { updateChartInfo({
...chartInfo.current,
sliderPos: [newStartSliderPos, newEndSliderPos], sliderPos: [newStartSliderPos, newEndSliderPos],
oldDataZoomOption: newDataZoomOption, oldDataZoomOption: newDataZoomOption,
}; });
renderTableInfo(); renderTableInfo();
return true; return true;
} else { } else {
@@ -259,79 +273,98 @@ const ChartDetail = (props: ChartDetailProps) => {
const updateChartData = (timeRange: [number, number], sliderPos: [number, number]) => { const updateChartData = (timeRange: [number, number], sliderPos: [number, number]) => {
const { const {
fullTimeRange: [fullStartTimestamp, fullEndTimestamp], fullTimeRange: [fullStartTimestamp, fullEndTimestamp],
fullMetricData,
isLoadedFullData, isLoadedFullData,
} = chartInfo.current; } = chartInfo.current;
let leftBoundaryTimestamp = Math.floor(timeRange[0]); const leftBoundaryTimestamp = Math.floor(timeRange[0]);
const isNeedCacheExtraData = leftBoundaryTimestamp < fullStartTimestamp + (fullEndTimestamp - fullStartTimestamp) * PRECACHE_THRESHOLD; const isNeedCacheExtraData = leftBoundaryTimestamp < fullStartTimestamp + (fullEndTimestamp - fullStartTimestamp) * PRECACHE_THRESHOLD;
let isRendered = false; let isRendered = false;
// 如果本地存储的数据足够展示或者已经获取到所有数据,则展示数据 // 如果本地存储的数据足够展示或者已经获取到所有数据,则展示数据
if (leftBoundaryTimestamp > fullStartTimestamp || isLoadedFullData) { if (leftBoundaryTimestamp > fullStartTimestamp || isLoadedFullData) {
chartInfo.current = { updateChartInfo({
...chartInfo.current,
curTimeRange: [leftBoundaryTimestamp > fullStartTimestamp ? leftBoundaryTimestamp : fullStartTimestamp, timeRange[1]], curTimeRange: [leftBoundaryTimestamp > fullStartTimestamp ? leftBoundaryTimestamp : fullStartTimestamp, timeRange[1]],
sliderPos, sliderPos,
}; });
renderNewMetricData(); renderNewMetricData();
isRendered = true; isRendered = true;
} }
if (!isLoadedFullData && isNeedCacheExtraData) { if (!isLoadedFullData && isNeedCacheExtraData) {
// 向服务器请求新的数据缓存 getAdditionChartData(!isRendered, leftBoundaryTimestamp, timeRange[1], sliderPos);
let reqEndTime = fullStartTimestamp; }
const requestArr: any[] = []; };
const requestTimeRanges: [number, number][] = [];
for (let i = 0; i < DEFAULT_REQUEST_COUNT; i++) {
setTimeout(() => {
const nextReqEndTime = reqEndTime - DEFAULT_REQUEST_TIME_RANGE;
requestArr.unshift(getMetricChartData([nextReqEndTime, reqEndTime]));
requestTimeRanges.unshift([nextReqEndTime, reqEndTime]);
reqEndTime = nextReqEndTime;
// 当最后一次请求发送后,处理返回 // 缓存增量的图表数据
if (i === DEFAULT_REQUEST_COUNT - 1) { const getAdditionChartData = (
Promise.all(requestArr).then((resList) => { needRender: boolean,
let isSettle = -1; leftBoundaryTimestamp: number,
// 填充增量的图表数据 rightBoundaryTimestamp: number,
resList.forEach((res: MetricDefaultChartDataType[], i) => { sliderPos?: [number, number]
// 图表没有返回数据的情况 ) => {
if (!res?.length) { const {
if (isSettle === -1) { fullTimeRange: [fullStartTimestamp, fullEndTimestamp],
chartInfo.current = { fullMetricData,
...chartInfo.current, isLoadingAdditionData,
// 标记数据已经全部加载完毕 } = chartInfo.current;
isLoadedFullData: true,
};
isSettle = i;
}
} else {
resolveAdditionChartData(res, requestTimeRanges[i]);
}
});
// 更新左侧边界为当前已获取到数据的最小边界
const curLocalStartTimestamp = Number(fullMetricData.metricLines.map((line) => line.data[0][0]).sort()[0]);
if (leftBoundaryTimestamp < curLocalStartTimestamp) {
leftBoundaryTimestamp = curLocalStartTimestamp;
}
chartInfo.current = { // 当前有缓存数据的任务时,直接退出
...chartInfo.current, if (isLoadingAdditionData) {
fullTimeRange: [reqEndTime - DEFAULT_REQUEST_TIME_RANGE, fullEndTimestamp], return false;
sliderPos, }
}; updateChartInfo({
if (!isRendered) { isLoadingAdditionData: true,
chartInfo.current = { });
...chartInfo.current,
curTimeRange: [leftBoundaryTimestamp, timeRange[1]], let reqEndTime = fullStartTimestamp;
}; const requestArr: any[] = [];
renderNewMetricData(); const requestTimeRanges: [number, number][] = [];
for (let i = 0; i < DEFAULT_REQUEST_COUNT; i++) {
setTimeout(() => {
const nextReqEndTime = reqEndTime - DEFAULT_REQUEST_TIME_RANGE;
requestArr.push(getMetricChartData([nextReqEndTime, reqEndTime]));
requestTimeRanges.push([nextReqEndTime, reqEndTime]);
reqEndTime = nextReqEndTime;
// 当最后一次请求发送后,处理返回
if (i === DEFAULT_REQUEST_COUNT - 1) {
Promise.all(requestArr).then((resList) => {
// 填充增量的图表数据
resList.forEach((res: MetricDefaultChartDataType[], i) => {
// 最后一个请求返回数据为空时,认为已获取到全部图表数据
if (!res?.length) {
// 标记数据已经全部加载完毕
i === resList.length - 1 &&
updateChartInfo({
isLoadedFullData: true,
});
} else {
// TODO: res 可能为 [],需要处理兼容
resolveAdditionChartData(res, requestTimeRanges[i]);
} }
}); });
}
}, i * 10); // 更新左侧边界为当前已获取到数据的最小边界
} const curLocalStartTimestamp = Number(fullMetricData.metricLines.map((line) => line?.data?.[0]?.[0]).sort()[0]);
if (leftBoundaryTimestamp < curLocalStartTimestamp) {
leftBoundaryTimestamp = curLocalStartTimestamp;
}
updateChartInfo({
fullTimeRange: [reqEndTime - DEFAULT_REQUEST_TIME_RANGE, fullEndTimestamp],
...(sliderPos ? { sliderPos } : {}),
isLoadingAdditionData: false,
});
if (needRender) {
updateChartInfo({
curTimeRange: [leftBoundaryTimestamp, rightBoundaryTimestamp],
});
renderNewMetricData();
}
});
}
}, i * 10);
} }
return true;
}; };
// 处理增量图表数据 // 处理增量图表数据
@@ -342,8 +375,6 @@ const ChartDetail = (props: ChartDetailProps) => {
global.getMetricDefine || {}, global.getMetricDefine || {},
metricType, metricType,
timeRange, timeRange,
DEFAULT_POINT_INTERVAL,
false,
chartInfo.current.transformUnit chartInfo.current.transformUnit
) as MetricChartDataType[]; ) as MetricChartDataType[];
// 增量填充图表数据 // 增量填充图表数据
@@ -362,7 +393,7 @@ const ChartDetail = (props: ChartDetailProps) => {
}); });
}; };
// 根据需要展示的时间范围过滤出对应的数据展示 // 根据需要展示的时间范围过滤出对应的数据
const renderNewMetricData = () => { const renderNewMetricData = () => {
const { fullMetricData, curTimeRange } = chartInfo.current; const { fullMetricData, curTimeRange } = chartInfo.current;
const newMetricData = { ...fullMetricData }; const newMetricData = { ...fullMetricData };
@@ -378,12 +409,25 @@ const ChartDetail = (props: ChartDetailProps) => {
}); });
newMetricData.metricLines[i] = line; newMetricData.metricLines[i] = line;
}); });
// 只过滤出当前时间段有数据点的线条,确保 Table 统一展示 // 只过滤出当前时间段有数据点的线条,确保 Table 统一展示
newMetricData.metricLines = newMetricData.metricLines.filter((line) => line.data.length); newMetricData.metricLines = newMetricData.metricLines.filter((line) => line.data.length);
setCurMetricData(newMetricData); setCurMetricData(newMetricData);
setLinesStatus((curStatus) => {
// 过滤维持线条选中状态
const newLinesStatus = { ...curStatus };
const newLineNames = newMetricData.metricLines.map((line) => line.name);
newLineNames.forEach((name) => {
if (newLinesStatus[name] === undefined) {
newLinesStatus[name] = false;
}
});
return newLinesStatus;
});
}; };
// 计算当前选中范围 // 计算展示当前拖拽轴选中的时间范围
const calculateSliderRange = () => { const calculateSliderRange = () => {
const { sliderPos } = chartInfo.current; const { sliderPos } = chartInfo.current;
let minutes = Number(((sliderPos[1] - sliderPos[0]) / 60 / 1000).toFixed(2)); let minutes = Number(((sliderPos[1] - sliderPos[0]) / 60 / 1000).toFixed(2));
@@ -398,13 +442,11 @@ const ChartDetail = (props: ChartDetailProps) => {
hours = Number((hours % 24).toFixed(2)); hours = Number((hours % 24).toFixed(2));
} }
chartInfo.current = { const sliderRange = ` 当前选中范围: ${days > 0 ? `${days}` : ''}${hours > 0 ? `${hours} 小时 ` : ''}${minutes} 分钟`;
...chartInfo.current, setSliderRange(sliderRange);
sliderRange: ` 当前选中范围: ${days > 0 ? `${days}` : ''}${hours > 0 ? `${hours} 小时 ` : ''}${minutes} 分钟`,
};
}; };
// 遍历图表,获取需要的指标数据展示到 Table // 遍历图表,计算得到指标聚合数据展示到表格
const renderTableInfo = () => { const renderTableInfo = () => {
const tableData: MetricTableInfo[] = []; const tableData: MetricTableInfo[] = [];
const { sliderPos, chartInstance } = chartInfo.current; const { sliderPos, chartInstance } = chartInfo.current;
@@ -447,140 +489,124 @@ const ChartDetail = (props: ChartDetailProps) => {
calculateSliderRange(); calculateSliderRange();
setTableInfo(tableData); setTableInfo(tableData);
setSelectedLines(tableData.map((line) => line.name));
}; };
const tableLineChange = (keys: string[]) => { const tableLineChange = (keys: string[]) => {
const updatedLines: { [name: string]: boolean } = {}; const newLinesStatus = { ...linesStatus };
selectedLines.forEach((name) => !keys.includes(name) && (updatedLines[name] = false));
keys.forEach((name) => !selectedLines.includes(name) && (updatedLines[name] = true));
// 更新 Object.entries(newLinesStatus).forEach(([name, status]) => {
Object.keys(updatedLines).forEach((name) => { if (keys.includes(name)) {
chartInfo.current.chartInstance.dispatchAction({ !status && (newLinesStatus[name] = true);
type: 'legendToggleSelect', } else {
// 图例名称 status && (newLinesStatus[name] = false);
name: name, }
});
}); });
setSelectedLines(keys); setLinesStatus(newLinesStatus);
}; };
// 图表数据更新渲染后,更新图表拖拽轴信息并重新计算列表值
useEffect(() => { useEffect(() => {
if (curMetricData) { if (curMetricData) {
setTimeout(() => { setTimeout(() => {
// 新的图表数据渲染后,更新图表拖拽轴信息
chartInfo.current.oldDataZoomOption = (chartInfo.current.chartInstance.getOption() as any).dataZoom[0]; chartInfo.current.oldDataZoomOption = (chartInfo.current.chartInstance.getOption() as any).dataZoom[0];
}); });
renderTableInfo(); renderTableInfo();
} }
}, [curMetricData]); }, [curMetricData]);
// 更新图例选中状态
useEffect(() => {
Object.entries(linesStatus).map(([name, status]) => {
const type = status ? 'legendSelect' : 'legendUnSelect';
chartInfo.current.chartInstance.dispatchAction({
type,
name,
});
});
}, [linesStatus]);
// 进入详情时,首次获取数据 // 进入详情时,首次获取数据
useEffect(() => { useEffect(() => {
if (metricType && metricName) { if (metricType && metricName) {
setLoading(true); setLoading(true);
const { curTimeRange } = chartInfo.current; const { curTimeRange } = chartInfo.current;
getMetricChartData(curTimeRange).then((res: any[] | null) => { getMetricChartData(curTimeRange).then(
// 如果图表返回数据 (res: any[] | null) => {
if (res?.length) { // 如果图表返回数据
// 格式化图表需要的数据 if (res?.length) {
const formattedMetricData = ( // 格式化图表需要的数据
formatChartData( const formattedMetricData = (
res, formatChartData(res, global.getMetricDefine || {}, metricType, curTimeRange) as MetricChartDataType[]
global.getMetricDefine || {}, )[0];
metricType, // 填充图表数据
curTimeRange, let initFullTimeRange = curTimeRange;
DEFAULT_POINT_INTERVAL, const pointsOfFirstLine = formattedMetricData.metricLines.find((line) => line.data.length).data;
false if (pointsOfFirstLine) {
) as MetricChartDataType[] initFullTimeRange = [
)[0]; pointsOfFirstLine[0][0] as number,
// 填充图表数据 pointsOfFirstLine[pointsOfFirstLine.length - 1][0] as number,
let initFullTimeRange = curTimeRange; ] as const;
const pointsOfFirstLine = formattedMetricData.metricLines.find((line) => line.data.length).data;
if (pointsOfFirstLine) {
initFullTimeRange = [pointsOfFirstLine[0][0] as number, pointsOfFirstLine[pointsOfFirstLine.length - 1][0] as number] as const;
}
// 获取单位保存起来
let transformUnit = undefined;
Object.entries(UNIT_MAP).forEach((unit) => {
if (formattedMetricData.metricUnit.includes(unit[0])) {
transformUnit = unit;
} }
});
chartInfo.current = { // 获取单位保存起来
...chartInfo.current, let transformUnit = undefined;
fullMetricData: formattedMetricData, Object.entries(UNIT_MAP).forEach((unit) => {
fullTimeRange: [...initFullTimeRange], if (formattedMetricData.metricUnit.includes(unit[0])) {
curTimeRange: [...initFullTimeRange], transformUnit = unit;
sliderPos: [ }
initFullTimeRange[1] - (initFullTimeRange[1] - initFullTimeRange[0]) * DATA_ZOOM_DEFAULT_SCALE, });
initFullTimeRange[1],
], updateChartInfo({
transformUnit, fullMetricData: formattedMetricData,
}; fullTimeRange: [...initFullTimeRange],
setCurMetricData(formattedMetricData); curTimeRange: [...initFullTimeRange],
setLoading(false); sliderPos: [
} initFullTimeRange[1] - (initFullTimeRange[1] - initFullTimeRange[0]) * DATA_ZOOM_DEFAULT_SCALE,
}); initFullTimeRange[1],
],
transformUnit,
});
setCurMetricData(formattedMetricData);
const newLinesStatus: { [lineName: string]: boolean } = {};
formattedMetricData.metricLines.forEach((line) => {
newLinesStatus[line.name] = true;
});
setLinesStatus(newLinesStatus);
setLoading(false);
getAdditionChartData(false, initFullTimeRange[0], initFullTimeRange[1]);
}
},
() => setLoading(false)
);
} }
}, []); }, []);
const debounced = debounce(onDataZoomDrag, 300); debouncedZoomDrag.current = debounce(onDataZoomDrag, 300);
return ( return (
<Spin spinning={loading}> <Spin spinning={loading}>
<div className="chart-detail-modal-container"> <div className="chart-detail-modal-container">
{curMetricData && ( {curMetricData ? (
<> <>
<div className="detail-title">
<div className="left">
<div className="title">
<Tooltip
placement="bottomLeft"
title={() => {
let content = '';
const metricDefine = global.getMetricDefine(metricType, curMetricData.metricName);
if (metricDefine) {
content = metricDefine.desc;
}
return content;
}}
>
<span style={{ cursor: 'pointer' }}>
<span>{curMetricData.metricName}</span> <span className="unit">({curMetricData.metricUnit}) </span>
</span>
</Tooltip>
</div>
<div className="info">{chartInfo.current.sliderRange}</div>
</div>
<div className="right">
<Button type="text" size="small" onClick={onClose}>
<CloseOutlined />
</Button>
</div>
</div>
<SingleChart <SingleChart
chartTypeProp="line" chartTypeProp="line"
wrapStyle={{ wrapStyle={{
width: 'auto', width: 'auto',
height: 462, height: 462,
}} }}
// events 事件只注册一次,所以这里使用 ref 来执行防抖函数
onEvents={{ onEvents={{
dataZoom: (record: any) => { dataZoom: (record: any) => debouncedZoomDrag?.current(record),
debounced(record);
},
}} }}
showHeader={false}
propChartData={curMetricData.metricLines} propChartData={curMetricData.metricLines}
optionMergeProps={{ notMerge: true }} optionMergeProps={{ notMerge: true }}
getChartInstance={(chartInstance) => { getChartInstance={(chartInstance) => {
chartInfo.current = { setDisposeChartInstance(() => () => chartInstance.dispose());
...chartInfo.current, updateChartInfo({
chartInstance, chartInstance,
}; });
}} }}
{...getDetailChartConfig(`${curMetricData.metricName}{unit|${curMetricData.metricUnit}}`, chartInfo.current.sliderPos)} {...getDetailChartConfig(`${curMetricData.metricName}{unit|${curMetricData.metricUnit}}`, chartInfo.current.sliderPos)}
/> />
@@ -588,16 +614,10 @@ const ChartDetail = (props: ChartDetailProps) => {
className="detail-table" className="detail-table"
rowKey="name" rowKey="name"
rowSelection={{ rowSelection={{
// hideSelectAll: true,
preserveSelectedRowKeys: true, preserveSelectedRowKeys: true,
selectedRowKeys: selectedLines, selectedRowKeys: Object.entries(linesStatus)
// getCheckboxProps: (record) => { .filter(([, status]) => status)
// return selectedLines.length <= 1 && selectedLines.includes(record.name) .map(([name]) => name),
// ? {
// disabled: true,
// }
// : {};
// },
selections: [Table.SELECTION_INVERT, Table.SELECTION_NONE], selections: [Table.SELECTION_INVERT, Table.SELECTION_NONE],
onChange: (keys: string[]) => tableLineChange(keys), onChange: (keys: string[]) => tableLineChange(keys),
}} }}
@@ -610,6 +630,8 @@ const ChartDetail = (props: ChartDetailProps) => {
pagination={false} pagination={false}
/> />
</> </>
) : (
!loading && <RenderEmpty message="详情加载失败,请重试" height={400} />
)} )}
</div> </div>
</Spin> </Spin>
@@ -618,22 +640,46 @@ const ChartDetail = (props: ChartDetailProps) => {
// eslint-disable-next-line react/display-name // eslint-disable-next-line react/display-name
const ChartDrawer = forwardRef((_, ref) => { const ChartDrawer = forwardRef((_, ref) => {
const [global] = AppContainer.useGlobalValue();
const [visible, setVisible] = useState(false); const [visible, setVisible] = useState(false);
const [dashboardType, setDashboardType] = useState<MetricType>();
const [metricName, setMetricName] = useState<string>();
const [queryLines, setQueryLines] = useState<string[]>([]); const [queryLines, setQueryLines] = useState<string[]>([]);
const [sliderRange, setSliderRange] = useState<string>('');
const [disposeChartInstance, setDisposeChartInstance] = useState<() => void>(() => 0);
const [metricInfo, setMetricInfo] = useState<{
type: MetricType | undefined;
name: string;
unit: string;
desc: string;
}>({
type: undefined,
name: '',
unit: '',
desc: '',
});
const onOpen = (dashboardType: MetricType, metricName: string, queryLines: string[]) => { const onOpen = (dashboardType: MetricType, metricName: string, queryLines: string[]) => {
setDashboardType(dashboardType); const metricDefine = global.getMetricDefine(dashboardType, metricName);
setMetricName(metricName); setMetricInfo({
type: dashboardType,
name: metricName,
unit: metricDefine?.unit || '',
desc: metricDefine?.desc || '',
});
setQueryLines(queryLines); setQueryLines(queryLines);
setVisible(true); setVisible(true);
}; };
const onClose = () => { const onClose = () => {
setVisible(false); setVisible(false);
setDashboardType(undefined); setSliderRange('');
setMetricName(undefined); disposeChartInstance();
setDisposeChartInstance(() => () => 0);
setMetricInfo({
type: undefined,
name: '',
unit: '',
desc: '',
});
}; };
useImperativeHandle(ref, () => ({ useImperativeHandle(ref, () => ({
@@ -641,9 +687,36 @@ const ChartDrawer = forwardRef((_, ref) => {
})); }));
return ( return (
<Drawer width={1080} visible={visible} footer={null} closable={false} maskClosable={false} destroyOnClose={true} onClose={onClose}> <Drawer
{dashboardType && metricName && ( className="overview-chart-detail-drawer"
<ChartDetail metricType={dashboardType} metricName={metricName} queryLines={queryLines} onClose={onClose} /> width={1080}
visible={visible}
title={
<div className="detail-header">
<div className="title">
<Tooltip placement="bottomLeft" title={metricInfo.desc}>
<span style={{ cursor: 'pointer' }}>
<span>{metricInfo.name}</span> <span className="unit">({metricInfo.unit}) </span>
</span>
</Tooltip>
</div>
<div className="slider-info">{sliderRange}</div>
</div>
}
footer={null}
closable={true}
maskClosable={false}
destroyOnClose={true}
onClose={onClose}
>
{metricInfo.type && metricInfo.name && (
<ChartDetail
metricType={metricInfo.type}
metricName={metricInfo.name}
queryLines={queryLines}
setSliderRange={setSliderRange}
setDisposeChartInstance={setDisposeChartInstance}
/>
)} )}
</Drawer> </Drawer>
); );

View File

@@ -1,152 +1,37 @@
import { getUnit, getDataNumberUnit, getBasicChartConfig, CHART_COLOR_LIST } from '@src/constants/chartConfig'; import { getBasicChartConfig, CHART_COLOR_LIST } from '@src/constants/chartConfig';
import { MetricType } from '@src/api';
import { MetricsDefine } from '@src/pages/CommonConfig';
export interface MetricInfo {
name: string;
desc: string;
type: number;
set: boolean;
support: boolean;
}
// 接口返回图表原始数据类型
export interface MetricDefaultChartDataType {
metricName: string;
metricLines: {
name: string;
createTime: number;
updateTime: number;
metricPoints: {
aggType: string;
timeStamp: number;
value: number;
createTime: number;
updateTime: number;
}[];
}[];
}
// 格式化后图表数据类型
export interface MetricChartDataType {
metricName: string;
metricUnit: string;
metricLines: {
name: string;
data: (string | number)[][];
}[];
dragKey?: number;
}
// 补点
export const supplementaryPoints = (
lines: MetricChartDataType['metricLines'],
timeRange: readonly [number, number],
interval: number,
extraCallback?: (point: [number, 0]) => any[]
) => {
lines.forEach(({ data }) => {
let len = data.length;
for (let i = 0; i < len; i++) {
const timestamp = data[i][0] as number;
// 数组第一个点和最后一个点单独处理
if (i === 0) {
let firstPointTimestamp = data[0][0] as number;
while (firstPointTimestamp - interval > timeRange[0]) {
const prePointTimestamp = firstPointTimestamp - interval;
data.unshift(extraCallback ? extraCallback([prePointTimestamp, 0]) : [prePointTimestamp, 0]);
len++;
i++;
firstPointTimestamp = prePointTimestamp;
}
}
if (i === len - 1) {
let lastPointTimestamp = data[len - 1][0] as number;
while (lastPointTimestamp + interval < timeRange[1]) {
const next = lastPointTimestamp + interval;
data.push(extraCallback ? extraCallback([next, 0]) : [next, 0]);
lastPointTimestamp = next;
}
} else if (timestamp + interval < data[i + 1][0]) {
data.splice(i + 1, 0, extraCallback ? extraCallback([timestamp + interval, 0]) : [timestamp + interval, 0]);
len++;
}
}
});
};
// 格式化图表数据
export const formatChartData = (
metricData: MetricDefaultChartDataType[],
getMetricDefine: (type: MetricType, metric: string) => MetricsDefine[keyof MetricsDefine],
metricType: MetricType,
timeRange: readonly [number, number],
supplementaryInterval: number,
needDrag = false,
transformUnit: [string, number] = undefined
): MetricChartDataType[] => {
return metricData.map(({ metricName, metricLines }) => {
const curMetricInfo = (getMetricDefine && getMetricDefine(metricType, metricName)) || null;
const isByteUnit = curMetricInfo?.unit?.toLowerCase().includes('byte');
let maxValue = -1;
const PointsMapMethod = ({ timeStamp, value }: { timeStamp: number; value: string | number }) => {
let parsedValue: string | number = Number(value);
if (Number.isNaN(parsedValue)) {
parsedValue = value;
} else {
// 为避免出现过小的数字影响图表展示效果,图表值统一只保留到小数点后三位
parsedValue = parseFloat(parsedValue.toFixed(3));
if (maxValue < parsedValue) maxValue = parsedValue;
}
return [timeStamp, parsedValue];
};
const chartData = Object.assign(
{
metricName,
metricUnit: curMetricInfo?.unit || '',
metricLines: metricLines
.sort((a, b) => Number(a.name < b.name) - 0.5)
.map(({ name, metricPoints }) => ({
name,
data: metricPoints.map(PointsMapMethod),
})),
},
needDrag ? { dragKey: 999 } : {}
);
chartData.metricLines.forEach(({ data }) => data.sort((a, b) => (a[0] as number) - (b[0] as number)));
supplementaryPoints(chartData.metricLines, timeRange, supplementaryInterval);
// 将所有图表点的值按单位进行转换
if (maxValue > 0) {
const [unitName, unitSize]: [string, number] = transformUnit || isByteUnit ? getUnit(maxValue) : getDataNumberUnit(maxValue);
chartData.metricUnit = isByteUnit
? chartData.metricUnit.toLowerCase().replace('byte', unitName)
: `${unitName}${chartData.metricUnit}`;
chartData.metricLines.forEach(({ data }) => data.forEach((point: any) => (point[1] /= unitSize)));
}
return chartData;
});
};
const seriesCallback = (lines: { name: string; data: [number, string | number][] }[]) => { const seriesCallback = (lines: { name: string; data: [number, string | number][] }[]) => {
const len = CHART_COLOR_LIST.length;
// series 配置 // series 配置
return lines.map((line) => { return lines.map((line, i) => {
return { return {
...line, ...line,
lineStyle: { lineStyle: {
width: 1.5, width: 1.5,
}, },
connectNulls: false,
symbol: 'emptyCircle', symbol: 'emptyCircle',
symbolSize: 4, symbolSize: 4,
smooth: 0.25, smooth: 0.25,
areaStyle: { areaStyle: {
opacity: 0.02, color: {
type: 'linear',
x: 0,
y: 0,
x2: 0,
y2: 1,
colorStops: [
{
offset: 0,
color: CHART_COLOR_LIST[i % len] + '10',
},
{
offset: 1,
color: 'rgba(255,255,255,0)', // 100% 处的颜色
},
],
global: false, // 缺省为 false
},
}, },
}; };
}); });
@@ -189,6 +74,7 @@ export const getDetailChartConfig = (title: string, sliderPos: readonly [number,
startValue: sliderPos[0], startValue: sliderPos[0],
endValue: sliderPos[1], endValue: sliderPos[1],
zoomOnMouseWheel: false, zoomOnMouseWheel: false,
minValueSpan: 10 * 60 * 1000,
}, },
{ {
start: 0, start: 0,

View File

@@ -63,56 +63,63 @@
} }
} }
} }
.overview-chart-detail-drawer {
.chart-detail-modal-container { .dcloud-spin-nested-loading > div > .dcloud-spin.dcloud-spin-spinning {
position: relative; height: 300px;
.expand-icon-box { }
position: absolute; &.dcloud-drawer .dcloud-drawer-body {
z-index: 1000; padding: 0 20px;
top: 14px; }
right: 44px; .detail-header {
width: 24px; display: flex;
height: 24px; align-items: flex-end;
cursor: pointer; font-weight: normal;
font-size: 16px; .title {
text-align: center; font-family: @font-family-bold;
border-radius: 50%; font-size: 18px;
transition: background-color 0.3s ease; color: #495057;
.expand-icon { letter-spacing: 0;
color: #adb5bc; .unit {
line-height: 24px; font-family: @font-family-bold;
} font-size: 14px;
&:hover { letter-spacing: 0.5px;
background: rgba(33, 37, 41, 0.04);
.expand-icon {
color: #74788d;
} }
} }
.slider-info {
margin-left: 10px;
font-size: 12px;
font-family: @font-family;
color: #303a51;
}
} }
.detail-title { .chart-detail-modal-container {
display: flex; position: relative;
justify-content: space-between; overflow: hidden;
align-items: center; .expand-icon-box {
.left { position: absolute;
display: flex; z-index: 1000;
align-items: flex-end; top: 14px;
.title { right: 44px;
font-family: @font-family-bold; width: 24px;
font-size: 18px; height: 24px;
color: #495057; cursor: pointer;
letter-spacing: 0; font-size: 16px;
.unit { text-align: center;
font-family: @font-family-bold; border-radius: 50%;
font-size: 14px; transition: background-color 0.3s ease;
letter-spacing: 0.5px; .expand-icon {
color: #adb5bc;
line-height: 24px;
}
&:hover {
background: rgba(33, 37, 41, 0.04);
.expand-icon {
color: #74788d;
} }
} }
.info { }
margin-left: 10px; .detail-table {
} margin-top: 16px;
} }
} }
.detail-table {
margin-top: 16px;
}
} }

View File

@@ -1,14 +1,21 @@
import React, { useState, useEffect, useRef } from 'react'; import React, { useState, useEffect, useRef } from 'react';
import { arrayMoveImmutable } from 'array-move'; import { arrayMoveImmutable } from 'array-move';
import { Utils, Empty, IconFont, Spin, AppContainer, SingleChart, Tooltip } from 'knowdesign'; import { Utils, Empty, Spin, AppContainer, SingleChart, Tooltip } from 'knowdesign';
import { IconFont } from '@knowdesign/icons';
import { useParams } from 'react-router-dom'; import { useParams } from 'react-router-dom';
import api, { MetricType } from '@src/api'; import api, { MetricType } from '@src/api';
import {
MetricInfo,
MetricDefaultChartDataType,
MetricChartDataType,
formatChartData,
resolveMetricsRank,
} from '@src/constants/chartConfig';
import SingleChartHeader, { KsHeaderOptions } from '../SingleChartHeader'; import SingleChartHeader, { KsHeaderOptions } from '../SingleChartHeader';
import DragGroup from '../DragGroup'; import DragGroup from '../DragGroup';
import ChartDetail from './ChartDetail'; import ChartDetail from './ChartDetail';
import { MetricInfo, MetricDefaultChartDataType, MetricChartDataType, formatChartData, getChartConfig } from './config'; import { getChartConfig } from './config';
import './index.less'; import './index.less';
import { MAX_TIME_RANGE_WITH_SMALL_POINT_INTERVAL } from '@src/constants/common';
interface IcustomScope { interface IcustomScope {
label: string; label: string;
@@ -39,8 +46,8 @@ const DashboardDragChart = (props: PropsType): JSX.Element => {
const [curHeaderOptions, setCurHeaderOptions] = useState<ChartFilterOptions>(); const [curHeaderOptions, setCurHeaderOptions] = useState<ChartFilterOptions>();
const [metricChartData, setMetricChartData] = useState<MetricChartDataType[]>([]); // 指标图表数据列表 const [metricChartData, setMetricChartData] = useState<MetricChartDataType[]>([]); // 指标图表数据列表
const [gridNum, setGridNum] = useState<number>(12); // 图表列布局 const [gridNum, setGridNum] = useState<number>(12); // 图表列布局
const metricRankList = useRef<string[]>([]);
const chartDetailRef = useRef(null); const chartDetailRef = useRef(null);
const chartDragOrder = useRef([]);
const curFetchingTimestamp = useRef(0); const curFetchingTimestamp = useRef(0);
// 获取节点范围列表 // 获取节点范围列表
@@ -60,23 +67,33 @@ const DashboardDragChart = (props: PropsType): JSX.Element => {
setScopeList(list); setScopeList(list);
}; };
// 更新 rank
const updateRank = (metricList: MetricInfo[]) => {
const { list, listInfo, shouldUpdate } = resolveMetricsRank(metricList);
metricRankList.current = list;
if (shouldUpdate) {
setMetricList(listInfo);
}
};
// 获取指标列表 // 获取指标列表
const getMetricList = () => { const getMetricList = () => {
Utils.request(api.getDashboardMetricList(clusterId, dashboardType)).then((res: MetricInfo[] | null) => { Utils.request(api.getDashboardMetricList(clusterId, dashboardType)).then((res: MetricInfo[] | null) => {
if (!res) return; if (!res) return;
const showMetrics = res.filter((metric) => metric.support); const supportMetrics = res.filter((metric) => metric.support);
const selectedMetrics = showMetrics.filter((metric) => metric.set).map((metric) => metric.name); const selectedMetrics = supportMetrics.filter((metric) => metric.set).map((metric) => metric.name);
setMetricsList(showMetrics); updateRank([...supportMetrics]);
setMetricsList(supportMetrics);
setSelectedMetricNames(selectedMetrics); setSelectedMetricNames(selectedMetrics);
}); });
}; };
// 更新指标 // 更新指标
const setMetricList = (metricsSet: { [name: string]: boolean }) => { const setMetricList = (metricDetailDTOList: { metric: string; rank: number; set: boolean }[]) => {
return Utils.request(api.getDashboardMetricList(clusterId, dashboardType), { return Utils.request(api.getDashboardMetricList(clusterId, dashboardType), {
method: 'POST', method: 'POST',
data: { data: {
metricsSet, metricDetailDTOList,
}, },
}); });
}; };
@@ -84,10 +101,11 @@ const DashboardDragChart = (props: PropsType): JSX.Element => {
// 根据筛选项获取图表信息 // 根据筛选项获取图表信息
const getMetricChartData = () => { const getMetricChartData = () => {
!curHeaderOptions.isAutoReload && setLoading(true); !curHeaderOptions.isAutoReload && setLoading(true);
const [startTime, endTime] = curHeaderOptions.rangeTime;
const [startTime, endTime] = curHeaderOptions.rangeTime;
const curTimestamp = Date.now(); const curTimestamp = Date.now();
curFetchingTimestamp.current = curTimestamp; curFetchingTimestamp.current = curTimestamp;
Utils.post(api.getDashboardMetricChartData(clusterId, dashboardType), { Utils.post(api.getDashboardMetricChartData(clusterId, dashboardType), {
startTime, startTime,
endTime, endTime,
@@ -108,36 +126,20 @@ const DashboardDragChart = (props: PropsType): JSX.Element => {
setMetricChartData([]); setMetricChartData([]);
} else { } else {
// 格式化图表需要的数据 // 格式化图表需要的数据
const supplementaryInterval = (endTime - startTime > MAX_TIME_RANGE_WITH_SMALL_POINT_INTERVAL ? 10 : 1) * 60 * 1000;
const formattedMetricData = formatChartData( const formattedMetricData = formatChartData(
res, res,
global.getMetricDefine || {}, global.getMetricDefine || {},
dashboardType, dashboardType,
curHeaderOptions.rangeTime, curHeaderOptions.rangeTime
supplementaryInterval,
true
) as MetricChartDataType[]; ) as MetricChartDataType[];
// 处理图表的拖拽顺 // 指标排
if (chartDragOrder.current && chartDragOrder.current.length) { formattedMetricData.sort((a, b) => metricRankList.current.indexOf(a.metricName) - metricRankList.current.indexOf(b.metricName));
// 根据当前拖拽顺序排列图表数据
formattedMetricData.forEach((metric) => {
const i = chartDragOrder.current.indexOf(metric.metricName);
metric.dragKey = i === -1 ? 999 : i;
});
formattedMetricData.sort((a, b) => a.dragKey - b.dragKey);
}
// 更新当前拖拽顺序(处理新增或减少图表的情况)
chartDragOrder.current = formattedMetricData.map((data) => data.metricName);
setMetricChartData(formattedMetricData); setMetricChartData(formattedMetricData);
} }
setLoading(false); setLoading(false);
}, },
() => { () => curFetchingTimestamp.current === curTimestamp && setLoading(false)
if (curFetchingTimestamp.current === curTimestamp) {
setLoading(false);
}
}
); );
}; };
@@ -163,11 +165,19 @@ const DashboardDragChart = (props: PropsType): JSX.Element => {
// 指标选中项更新回调 // 指标选中项更新回调
const indicatorChangeCallback = (newMetricNames: (string | number)[]) => { const indicatorChangeCallback = (newMetricNames: (string | number)[]) => {
const updateMetrics: { [name: string]: boolean } = {}; const updateMetrics: { metric: string; set: boolean; rank: number }[] = [];
// 需要选中的指标 // 需要选中的指标
newMetricNames.forEach((name) => !selectedMetricNames.includes(name) && (updateMetrics[name] = true)); newMetricNames.forEach(
(name) =>
!selectedMetricNames.includes(name) &&
updateMetrics.push({ metric: name as string, set: true, rank: metricsList.find(({ name: metric }) => metric === name)?.rank })
);
// 取消选中的指标 // 取消选中的指标
selectedMetricNames.forEach((name) => !newMetricNames.includes(name) && (updateMetrics[name] = false)); selectedMetricNames.forEach(
(name) =>
!newMetricNames.includes(name) &&
updateMetrics.push({ metric: name as string, set: false, rank: metricsList.find(({ name: metric }) => metric === name)?.rank })
);
const requestPromise = Object.keys(updateMetrics).length ? setMetricList(updateMetrics) : Promise.resolve(); const requestPromise = Object.keys(updateMetrics).length ? setMetricList(updateMetrics) : Promise.resolve();
requestPromise.then( requestPromise.then(
@@ -186,7 +196,11 @@ const DashboardDragChart = (props: PropsType): JSX.Element => {
// 拖拽结束回调,更新图表顺序,并触发图表的 onDrag 事件( 设置为 false ),允许同步展示图表的 tooltip // 拖拽结束回调,更新图表顺序,并触发图表的 onDrag 事件( 设置为 false ),允许同步展示图表的 tooltip
const dragEnd = ({ oldIndex, newIndex }: { oldIndex: number; newIndex: number }) => { const dragEnd = ({ oldIndex, newIndex }: { oldIndex: number; newIndex: number }) => {
busInstance.emit('onDrag', false); busInstance.emit('onDrag', false);
chartDragOrder.current = arrayMoveImmutable(chartDragOrder.current, oldIndex, newIndex); const originFrom = metricRankList.current.indexOf(metricChartData[oldIndex].metricName);
const originTarget = metricRankList.current.indexOf(metricChartData[newIndex].metricName);
const newList = arrayMoveImmutable(metricRankList.current, originFrom, originTarget);
metricRankList.current = newList;
setMetricList(newList.map((metric, rank) => ({ metric, rank, set: metricsList.find(({ name }) => metric === name)?.set || false })));
setMetricChartData(arrayMoveImmutable(metricChartData, oldIndex, newIndex)); setMetricChartData(arrayMoveImmutable(metricChartData, oldIndex, newIndex));
}; };
@@ -216,8 +230,8 @@ const DashboardDragChart = (props: PropsType): JSX.Element => {
onChange={ksHeaderChange} onChange={ksHeaderChange}
nodeScopeModule={{ nodeScopeModule={{
customScopeList: scopeList, customScopeList: scopeList,
scopeName: `自定义 ${dashboardType === MetricType.Broker ? 'Broker' : 'Topic'} 范围`, scopeName: dashboardType === MetricType.Broker ? 'Broker' : 'Topic',
showSearch: dashboardType === MetricType.Topic, scopeLabel: `自定义 ${dashboardType === MetricType.Broker ? 'Broker' : 'Topic'} 范围`,
}} }}
indicatorSelectModule={{ indicatorSelectModule={{
hide: false, hide: false,

View File

@@ -1,4 +1,5 @@
import { Col, IconFont, Row } from 'knowdesign'; import { Col, Row } from 'knowdesign';
import { IconFont } from '@knowdesign/icons';
import React from 'react'; import React from 'react';
import { SortableContainer, SortableContainerProps, SortableHandle, SortableElement, SortableElementProps } from 'react-sortable-hoc'; import { SortableContainer, SortableContainerProps, SortableHandle, SortableElement, SortableElementProps } from 'react-sortable-hoc';
import './index.less'; import './index.less';

View File

@@ -0,0 +1,15 @@
import React from 'react';
const RenderEmpty = (props: { height?: string | number; message: string }) => {
const { height = 200, message } = props;
return (
<>
<div className="empty-panel" style={{ height }}>
<div className="img" />
<div className="text">{message}</div>
</div>
</>
);
};
export default RenderEmpty;

View File

@@ -1,5 +1,6 @@
import React, { useState, useEffect } from 'react'; import React, { useState, useEffect } from 'react';
import { Drawer, Button, Space, Divider, AppContainer, ProTable, IconFont } from 'knowdesign'; import { Drawer, Button, Space, Divider, AppContainer, ProTable } from 'knowdesign';
import { IconFont } from '@knowdesign/icons';
import { IindicatorSelectModule } from './index'; import { IindicatorSelectModule } from './index';
import './style/indicator-drawer.less'; import './style/indicator-drawer.less';
import { useLocation } from 'react-router-dom'; import { useLocation } from 'react-router-dom';

View File

@@ -1,5 +1,6 @@
import React, { useState, useEffect } from 'react'; import React, { useState, useEffect } from 'react';
import { Radio, Input, Popover, Space, Checkbox, Row, Col, Button, IconFont } from 'knowdesign'; import { Radio, Input, Popover, Space, Checkbox, Row, Col, Button } from 'knowdesign';
import { IconFont } from '@knowdesign/icons';
import { InodeScopeModule } from './index'; import { InodeScopeModule } from './index';
import './style/node-scope.less'; import './style/node-scope.less';
@@ -26,8 +27,8 @@ const OptionsDefault = [
const NodeScope = ({ nodeScopeModule, change }: propsType) => { const NodeScope = ({ nodeScopeModule, change }: propsType) => {
const { const {
customScopeList: customList, customScopeList: customList,
scopeName = '自定义节点范围', scopeName = '',
showSearch = false, scopeLabel = '自定义范围',
searchPlaceholder = '输入内容进行搜索', searchPlaceholder = '输入内容进行搜索',
} = nodeScopeModule; } = nodeScopeModule;
const [topNum, setTopNum] = useState<number>(5); const [topNum, setTopNum] = useState<number>(5);
@@ -70,7 +71,7 @@ const NodeScope = ({ nodeScopeModule, change }: propsType) => {
change(checkedListTemp, false); change(checkedListTemp, false);
setIsTop(false); setIsTop(false);
setTopNum(null); setTopNum(null);
setInputValue(`已选${checkedListTemp?.length}`); setInputValue(`${checkedListTemp?.length}`);
setPopVisible(false); setPopVisible(false);
} }
}; };
@@ -109,7 +110,7 @@ const NodeScope = ({ nodeScopeModule, change }: propsType) => {
{/* <span>时间:</span> */} {/* <span>时间:</span> */}
<div className="flx_con"> <div className="flx_con">
<div className="flx_l"> <div className="flx_l">
<h6 className="time_title">top</h6> <h6 className="time_title"> top </h6>
<Radio.Group <Radio.Group
optionType="button" optionType="button"
buttonStyle="solid" buttonStyle="solid"
@@ -128,7 +129,7 @@ const NodeScope = ({ nodeScopeModule, change }: propsType) => {
</Radio.Group> </Radio.Group>
</div> </div>
<div className="flx_r"> <div className="flx_r">
<h6 className="time_title">{scopeName}</h6> <h6 className="time_title">{scopeLabel}</h6>
<div className="custom-scope"> <div className="custom-scope">
<div className="check-row"> <div className="check-row">
<Checkbox className="check-all" indeterminate={indeterminate} onChange={onCheckAllChange} checked={checkAll}> <Checkbox className="check-all" indeterminate={indeterminate} onChange={onCheckAllChange} checked={checkAll}>
@@ -136,9 +137,7 @@ const NodeScope = ({ nodeScopeModule, change }: propsType) => {
</Checkbox> </Checkbox>
<Input <Input
className="search-input" className="search-input"
suffix={ suffix={<IconFont type="icon-fangdajing" style={{ fontSize: '16px' }} />}
<IconFont type="icon-fangdajing" style={{ fontSize: '16px' }} />
}
size="small" size="small"
placeholder={searchPlaceholder} placeholder={searchPlaceholder}
onChange={(e) => setScopeSearchValue(e.target.value)} onChange={(e) => setScopeSearchValue(e.target.value)}
@@ -148,7 +147,7 @@ const NodeScope = ({ nodeScopeModule, change }: propsType) => {
<Checkbox.Group style={{ width: '100%' }} onChange={checkChange} value={checkedListTemp}> <Checkbox.Group style={{ width: '100%' }} onChange={checkChange} value={checkedListTemp}>
<Row gutter={[10, 12]}> <Row gutter={[10, 12]}>
{customList {customList
.filter((item) => !showSearch || item.label.includes(scopeSearchValue)) .filter((item) => item.label.includes(scopeSearchValue))
.map((item) => ( .map((item) => (
<Col span={12} key={item.value}> <Col span={12} key={item.value}>
<Checkbox value={item.value}>{item.label}</Checkbox> <Checkbox value={item.value}>{item.label}</Checkbox>
@@ -180,6 +179,7 @@ const NodeScope = ({ nodeScopeModule, change }: propsType) => {
return ( return (
<> <>
<div id="d-node-scope"> <div id="d-node-scope">
<div className="scope-title">{scopeName}</div>
<Popover <Popover
trigger={['click']} trigger={['click']}
visible={popVisible} visible={popVisible}

View File

@@ -1,5 +1,6 @@
import React, { useEffect, useState } from 'react'; import React, { useEffect, useState } from 'react';
import { Tooltip, Select, IconFont, Utils, Divider } from 'knowdesign'; import { Tooltip, Select, Utils, Divider, Button } from 'knowdesign';
import { IconFont } from '@knowdesign/icons';
import moment from 'moment'; import moment from 'moment';
import { DRangeTime } from 'knowdesign'; import { DRangeTime } from 'knowdesign';
import IndicatorDrawer from './IndicatorDrawer'; import IndicatorDrawer from './IndicatorDrawer';
@@ -48,7 +49,7 @@ export interface IcustomScope {
export interface InodeScopeModule { export interface InodeScopeModule {
customScopeList: IcustomScope[]; customScopeList: IcustomScope[];
scopeName?: string; scopeName?: string;
showSearch?: boolean; scopeLabel?: string;
searchPlaceholder?: string; searchPlaceholder?: string;
change?: () => void; change?: () => void;
} }
@@ -138,9 +139,13 @@ const SingleChartHeader = ({
}; };
const reloadRangeTime = () => { const reloadRangeTime = () => {
const timeLen = rangeTime[1] - rangeTime[0] || 0; if (isRelativeRangeTime) {
const curTimeStamp = moment().valueOf(); const timeLen = rangeTime[1] - rangeTime[0] || 0;
setRangeTime([curTimeStamp - timeLen, curTimeStamp]); const curTimeStamp = moment().valueOf();
setRangeTime([curTimeStamp - timeLen, curTimeStamp]);
} else {
setRangeTime([...rangeTime]);
}
}; };
const openIndicatorDrawer = () => { const openIndicatorDrawer = () => {
@@ -174,12 +179,10 @@ const SingleChartHeader = ({
{!hideGridSelect && ( {!hideGridSelect && (
<Select className="grid-select" style={{ width: 70 }} value={gridNum} options={GRID_SIZE_OPTIONS} onChange={sizeChange} /> <Select className="grid-select" style={{ width: 70 }} value={gridNum} options={GRID_SIZE_OPTIONS} onChange={sizeChange} />
)} )}
<Divider type="vertical" style={{ height: 20, top: 0 }} /> {(!hideNodeScope || !hideGridSelect) && <Divider type="vertical" style={{ height: 20, top: 0 }} />}
<Tooltip title="点击指标筛选,可选择指标" placement="bottomRight"> <Button type="primary" onClick={openIndicatorDrawer}>
<div className="icon-box" onClick={openIndicatorDrawer}>
<IconFont className="icon" type="icon-shezhi1" /> </Button>
</div>
</Tooltip>
</div> </div>
</div> </div>
</div> </div>

View File

@@ -3,8 +3,13 @@
@import '~knowdesign/es/basic/style/mixins/index'; @import '~knowdesign/es/basic/style/mixins/index';
#d-node-scope { #d-node-scope {
display: flex;
align-items: center;
position: relative; position: relative;
display: inline-block; .scope-title {
font-size: 14px;
color: #74788d;
}
.input-span { .input-span {
cursor: pointer; cursor: pointer;
} }
@@ -29,10 +34,10 @@
box-shadow: none; box-shadow: none;
} }
&.relativeTime { &.relativeTime {
width: 160px; width: 200px;
} }
&.absoluteTime { &.absoluteTime {
width: 300px; width: 200px;
} }
input { input {

Some files were not shown because too many files have changed in this diff Show More