Compare commits

...

224 Commits

Author SHA1 Message Date
EricZeng
89405fe003 Merge pull request #434 from didi/fix_2.5.0
修复console模块关闭问题及前端文件名错误问题
2022-01-13 14:00:01 +08:00
shirenchuang
b9ea3865a5 升级到2.5版本
(cherry picked from commit 5bc6eb6774)
2022-01-13 13:47:21 +08:00
孙超
b5bd643814 修复图片名称大小写问题
(cherry picked from commit ada2718b5e)
2022-01-13 13:46:06 +08:00
shirenchuang
5bc6eb6774 升级到2.5版本 2021-12-16 18:28:51 +08:00
石臻臻的杂货铺
3ba81e9aaa Merge pull request #411 from didi/v2.5.0
升级到2.5版本
2021-12-16 15:28:34 +08:00
shirenchuang
329a9b59c1 升级到2.5版本 2021-12-16 15:08:54 +08:00
EricZeng
22c26e24b1 Merge pull request #410 from lucasun/hotfix/2.5.0_fe
修复顶导V2.5.0版本
2021-12-11 14:30:22 +08:00
孙超
396045177c 修复顶导V2.5.0版本 2021-12-11 14:25:43 +08:00
EricZeng
e311d3767c Merge pull request #407 from didi/dev_v2.5.0
merge dev_v2.5.0 to master
2021-12-01 19:42:03 +08:00
EricZeng
24d7b80244 Merge pull request #406 from kingdomrushing/dev_v2.5.0
我的申请-审批中-审批时间置为空
2021-12-01 13:31:08 +08:00
xuguang
61f99e4d2e 我的申请-审批中-审批时间置为空 2021-12-01 12:54:02 +08:00
EricZeng
d5348bcf49 Merge pull request #405 from lucasun/dev_v2.5.0_fe
我的申请-审批列表列-申请时间、审批时间增加无数据判断
2021-12-01 11:27:58 +08:00
孙超
5d31d66365 我的申请-审批列表列-申请时间、审批时间增加无数据判断 2021-12-01 11:17:12 +08:00
EricZeng
29778a0154 Merge pull request #400 from lucasun/dev_v2.5.0_fe
Dev v2.5.0 fe
2021-11-30 15:01:49 +08:00
Peng
165c0a5866 Update README.md 2021-11-29 19:32:54 +08:00
EricZeng
588323961e Merge pull request #401 from didi/master
合并主分支到2.5开发分支
2021-11-23 18:50:27 +08:00
孙超
fd1c0b71c5 V2.5.0前端 更换二维码&前端bugfix 2021-11-23 17:48:10 +08:00
lucasun
54fbdcadf9 Merge branch 'didi:master' into master 2021-11-23 17:35:27 +08:00
石臻臻的杂货铺
69a30d0cf0 Merge pull request #399 from kingdomrushing/dev_v2.5.0
修复"新添加集群的时候,报watch的空指针异常"问题 & 修复"删除废弃Topic之后,Topic资源治理没有同步删除"问题
2021-11-22 17:35:36 +08:00
xuguang
b8f9b44f38 修复获取topic流量指标未按时间排序问题 2021-11-20 13:30:34 +08:00
xuguang
cbf17d4eb5 修复"新添加集群的时候,报watch的空指针异常"问题 & 修复"删除废弃Topic之后,Topic资源治理没有同步删除"问题 2021-11-19 19:27:19 +08:00
石臻臻的杂货铺
327e025262 Merge pull request #397 from kingdomrushing/dev_v2.5.0
Dev v2.5.0
2021-11-19 14:20:20 +08:00
xuguang
6b1e944bba 修复topic管理中topic编辑备注没有数据回显问题 2021-11-19 13:23:52 +08:00
EricZeng
668ed4d61b Merge pull request #396 from didi/dev_inc_monitor_indicators
补充新增上报监控系统指标说明文档
2021-11-16 22:32:20 +08:00
zengqiao
312c0584ed 补充新增上报监控系统指标说明文档 2021-11-16 22:20:35 +08:00
zengqiao
110d3acb58 补充新增上报监控系统指标说明文档 2021-11-16 22:16:35 +08:00
xuguang
ddbc60283b 将tomcat版本升级为8.5.72 & "我的审批"列表增加"通过时间"列,并支持按该列排序 & JMX连接关闭问题修复 2021-11-16 17:15:58 +08:00
shirenchuang
471bcecfd6 Merge branch 'v2.4.3' into dev_v2.5.0 2021-11-15 12:45:01 +08:00
shirenchuang
0245791b13 Merge remote-tracking branch 'origin/master' into dev_v2.5.0 2021-11-15 11:45:22 +08:00
shirenchuang
4794396ce8 Merge remote-tracking branch 'origin/master' into v2.4.3 2021-11-15 10:49:22 +08:00
EricZeng
c7088779d6 Merge pull request #395 from ZHAOYINRUI/patch-5
Update README.md
2021-11-11 15:21:11 +08:00
ZHAOYINRUI
672905da12 Update README.md 2021-11-11 15:19:29 +08:00
EricZeng
47172b13be Merge pull request #394 from ZHAOYINRUI/patch-4
Update README.md
2021-11-09 14:18:34 +08:00
ZHAOYINRUI
3668a10af6 Update README.md 2021-11-09 12:43:09 +08:00
EricZeng
a4e294c03f Merge pull request #393 from ZHAOYINRUI/patch-3
增加【Kafka中文社区】知识星球二维码
2021-11-08 17:44:15 +08:00
ZHAOYINRUI
3fd6f4003f 增加【Kafka中文社区】知识星球二维码 2021-11-08 17:39:08 +08:00
EricZeng
3eaf5cd530 Merge pull request #378 from didi/dev
白名单接口中仅保留登录接口
2021-09-21 11:09:36 +08:00
zengqiao
c344fd8ca4 白名单接口中仅保留登录接口 2021-09-21 11:00:33 +08:00
EricZeng
09639ca294 Merge pull request #377 from didi/dev
修复Sonar扫描问题
2021-09-21 10:58:36 +08:00
EricZeng
a81b6dca83 Merge pull request #376 from didi/master
merge master
2021-09-21 10:47:55 +08:00
mike.zhangliang
b74aefb08f Update README.md 2021-08-15 15:14:25 +08:00
mike.zhangliang
757f90aa7a Update README.md 2021-08-11 09:08:33 +08:00
EricZeng
7e1b3c552b Merge pull request #360 from ZHAOYINRUI/patch-1
Update 开源版与商业版特性对比.md
2021-08-03 10:02:02 +08:00
ZHAOYINRUI
69736a63b6 Update 开源版与商业版特性对比.md
补充优化
2021-08-02 22:10:15 +08:00
EricZeng
fb4a9f9056 删除多余的‘在’
删除多余的‘在’
2021-07-26 09:28:16 +08:00
zengqiao
387d89d3af optimize code format by sonar-lint 2021-07-13 10:39:28 +08:00
EricZeng
65d9ca9d39 Merge pull request #336 from fengxsong/master
feat: update dockerfile and helm chart
2021-07-10 10:47:57 +08:00
Peng
8c842af4ba Update README.md
更新小尺寸logo
2021-07-09 12:46:18 +08:00
shirenchuang
4faf9262c9 配置文件漏了 加上 2021-07-09 11:55:14 +08:00
shirenchuang
be7724c67d 2021-07-09 11:21:20 +08:00
Peng
48d26347f7 Update README.md
替换logo
2021-07-09 11:01:18 +08:00
shirenchuang
bdb01ec8b5 2021-07-07 13:28:55 +08:00
mike.zhangliang
9047815799 Update README.md 2021-07-06 17:36:23 +08:00
EricZeng
05bd94a2cc Merge pull request #344 from didi/dev
删除钉钉群二维码
2021-07-05 12:15:52 +08:00
zengqiao
c9f7da84d0 删除钉钉群二维码 2021-07-05 12:14:37 +08:00
EricZeng
bcc124e86a Merge pull request #343 from Hongten/master
修复Converts#convert2OrderDO() 出现重复赋值
2021-07-04 21:35:34 +08:00
Hongten
48d2733403 Merge pull request #2 from didi/master
sync code
2021-07-04 18:04:55 +08:00
hongtenzone@foxmail.com
31fc6e4e56 remove duplicate operation 2021-07-04 17:59:36 +08:00
hongtenzone@foxmail.com
fcdeef0146 remove duplicate operation 2021-07-04 17:55:54 +08:00
EricZeng
1cd524c0cc Merge pull request #341 from didi/dev
Topic基本信息中增加retention.bytes信息
2021-07-02 18:34:56 +08:00
zengqiao
0f746917a7 Topic基本信息中增加retention.bytes信息 2021-07-02 16:41:57 +08:00
EricZeng
a2228d0169 Merge pull request #335 from didi/dev
bump jackson-databind version to 2.9.10.8
2021-06-24 18:04:57 +08:00
shirenchuang
e8a679d34b Merge branch 'master' into v2.4.3 2021-06-24 17:18:50 +08:00
fengxusong
1912a42091 fix: default config 2021-06-24 14:00:29 +08:00
fengxusong
ca81f96635 feat: update dockerfile and charts 2021-06-24 12:13:29 +08:00
zengqiao
eb3b8c4b31 bump jackson-databind version to 2.9.10.8 2021-06-23 21:31:43 +08:00
EricZeng
6740d6d60b Merge pull request #332 from didi/dev
修复poll异常时, 超时时间不生效问题
2021-06-23 20:24:50 +08:00
zengqiao
c46c35b248 修复poll异常时, 超时时间不生效问题 2021-06-23 10:11:38 +08:00
EricZeng
0b2dcec4bc Merge pull request #323 from didi/dev
fix jmx credentials
2021-06-03 10:22:12 +08:00
shirenchuang
f8e2a4aff4 修改km的打包方式
增加启动/关闭脚本
2021-06-02 18:13:58 +08:00
zengqiao
7256db8c4e fix jmx credentials 2021-06-02 13:59:18 +08:00
shirenchuang
b14d5d9bee 修改km的打包方式
增加启动/关闭脚本
2021-06-01 20:20:40 +08:00
shirenchuang
12e15c3e4b Merge branch 'shirc_dev' into dev_v2.5.0 2021-06-01 20:19:53 +08:00
shirenchuang
51911bf272 add distribution 2021-06-01 20:17:54 +08:00
shirenchuang
6dc8061401 add distribution 2021-06-01 16:32:16 +08:00
EricZeng
b8fa4f8797 Merge pull request #319 from didi/dev
optimize n9e's default port
2021-05-31 19:44:38 +08:00
zengqiao
cc0bea7f45 optimize n9e's default port 2021-05-31 19:43:03 +08:00
EricZeng
4e9124b244 Merge pull request #316 from didi/dev
Topic账单配置说明
2021-05-29 13:46:18 +08:00
zengqiao
f0eabef7b0 Topic账单配置说明 2021-05-28 17:36:36 +08:00
EricZeng
23e5557958 Merge pull request #315 from didi/master
kafka-gateway相关功能说明
2021-05-28 17:13:29 +08:00
EricZeng
b1d02afa85 Merge pull request #312 from lucasun/master
修复clipbord 2.0.6 打包问题
2021-05-28 11:34:01 +08:00
孙超
2edc380f47 修改package.json增加内存修复和clipbord版本 2021-05-28 11:30:33 +08:00
孙超
cea8295c09 clipbord升级版本 2021-05-28 11:21:12 +08:00
EricZeng
244bfc993a Merge pull request #310 from ZHAOYINRUI/master
补充FAQ开源版和商业版特性对比
2021-05-27 14:50:52 +08:00
ZHAOYINRUI
3a272a4493 Update faq.md 2021-05-27 14:45:51 +08:00
ZHAOYINRUI
a3300db770 Update faq.md 2021-05-27 14:22:59 +08:00
ZHAOYINRUI
b0394ce261 Delete Logi-KafkaManager开源版和商业版特性对比总结.pdf 2021-05-27 14:22:03 +08:00
ZHAOYINRUI
3123089790 Create 开源版与商业版特性对比.md 2021-05-27 14:21:48 +08:00
ZHAOYINRUI
f13cf66676 Delete 开源版与商业版特性对比.md 2021-05-27 12:08:15 +08:00
ZHAOYINRUI
0c8c4d87fb Delete 开源版与商业版特性对比.md 2021-05-27 12:08:01 +08:00
ZHAOYINRUI
066088fdeb Update faq.md 2021-05-27 12:06:51 +08:00
ZHAOYINRUI
cf641e41c7 Update faq.md 2021-05-27 12:05:41 +08:00
ZHAOYINRUI
5b48322e1b Update faq.md 2021-05-27 12:04:58 +08:00
ZHAOYINRUI
9d3f680d58 Update faq.md 2021-05-27 12:04:04 +08:00
ZHAOYINRUI
bed28d57e6 Update 开源版与商业版特性对比.md 2021-05-27 12:02:09 +08:00
ZHAOYINRUI
2538525103 Update 开源版与商业版特性对比.md 2021-05-27 12:01:51 +08:00
ZHAOYINRUI
6ed798db8c Create 开源版与商业版特性对比.md 2021-05-27 12:01:15 +08:00
ZHAOYINRUI
8e9d966829 Update 开源版与商业版特性对比.md 2021-05-27 12:00:26 +08:00
ZHAOYINRUI
be16640f92 Update 开源版与商业版特性对比.md 2021-05-27 11:59:41 +08:00
ZHAOYINRUI
0e1376dd2e Create 开源版与商业版特性对比.md 2021-05-27 11:57:56 +08:00
ZHAOYINRUI
0494575aa7 Update faq.md 2021-05-27 11:32:08 +08:00
ZHAOYINRUI
bed57534e0 Add files via upload 2021-05-27 11:22:47 +08:00
EricZeng
1862d631d1 Merge pull request #305 from didi/dev
heartbeat表的数据更新时间从MySQL自动生成调整为Logi-KM的时间
2021-05-25 13:44:04 +08:00
zengqiao
c977ce5690 heartbeat表的数据更新时间从MySQL自动生成调整为Logi-KM的时间 2021-05-25 10:27:27 +08:00
zengqiao
84df377516 bump version to v2.4.2 and add release notes 2021-05-21 10:45:10 +08:00
EricZeng
4d9a284f6e Merge pull request #303 from didi/dev
bump tomcat version to 8.5.66
2021-05-21 10:21:16 +08:00
zengqiao
da7ad8b44a bump tomcat version to 8.5.66 2021-05-21 10:20:09 +08:00
EricZeng
4164046323 Merge pull request #301 from didi/dev
fix title version
2021-05-20 20:36:14 +08:00
zengqiao
72e743dfd1 fix title version 2021-05-20 20:35:23 +08:00
EricZeng
7eb7edaf0a Merge pull request #300 from didi/dev
bump tomcat version to 8.5.56
2021-05-20 20:33:59 +08:00
zengqiao
49368aaf76 bump tomcat version to 8.5.56 2021-05-20 18:17:30 +08:00
zengqiao
b8c07a966f bump version to v2.4.1 2021-05-18 21:00:20 +08:00
EricZeng
c6bcc0e3aa Merge pull request #297 from didi/dev
split op util controller to topic controller and leader controller, and add authority controller, quota controller
2021-05-18 20:21:18 +08:00
zengqiao
7719339f23 split op util controller to topic controller and leader controller, and add authority controller, quota controller 2021-05-18 20:18:08 +08:00
EricZeng
8ad64722ed Merge pull request #296 from tcf1207239873/dev
迭代优化2.4.1-配额调整
2021-05-18 19:37:58 +08:00
tangcongfa_v@didichuxing.com
611f8b8865 调整配额与权限 2021-05-18 18:25:03 +08:00
tangcongfa_v@didichuxing.com
38bdc173e8 Merge remote-tracking branch 'origin/dev' into dev 2021-05-18 16:07:40 +08:00
tangcongfa_v@didichuxing.com
52244325d9 权限调整 2021-05-18 16:03:24 +08:00
tcf1207239873
3fd3d99b8c Merge pull request #4 from didi/dev
Dev-0518-2
2021-05-18 15:45:55 +08:00
tangcongfa_v@didichuxing.com
d4ee5e91a2 配额调整 2021-05-18 14:49:00 +08:00
EricZeng
c2ad2d7238 Merge pull request #295 from didi/dev
简化sd配置枚举类的长度
2021-05-18 14:36:29 +08:00
zengqiao
892e195f0e code format 2021-05-18 14:33:02 +08:00
zengqiao
c5b1bed7dc 简化sd配置枚举类的长度 2021-05-18 14:13:23 +08:00
EricZeng
0e388d7aa7 Merge pull request #294 from didi/dev
增加登录绕过的接口调用方式
2021-05-18 11:48:55 +08:00
zengqiao
c3a0dbbe48 增加登录绕过的接口调用方式 2021-05-18 11:45:15 +08:00
tangcongfa_v@didichuxing.com
8b95b3ffc7 配额调整 2021-05-18 11:25:41 +08:00
tcf1207239873
42b78461cd Merge pull request #3 from didi/dev
Dev-0518
2021-05-18 10:30:57 +08:00
EricZeng
9190a41ca5 Merge pull request #293 from didi/dev
增加监控指标说明
2021-05-17 18:57:58 +08:00
zengqiao
28a7251319 增加监控指标说明 2021-05-17 15:31:08 +08:00
EricZeng
20565866ef Merge pull request #292 from didi/master
merge master
2021-05-17 15:20:50 +08:00
EricZeng
246f10aee5 Merge pull request #258 from Huyueeer/patch-2
更新Topic资源治理部分
2021-05-17 09:58:09 +08:00
EricZeng
960017280d 增加Topic资源治理的配置说明
增加Topic资源治理的配置说明
2021-05-17 09:57:56 +08:00
tangcongfa_v@didichuxing.com
7218aaf52e 权限调整 2021-05-13 17:49:42 +08:00
zengqiao
62050cc7b6 Merge branch 'master' of https://github.com/didi/Logi-KafkaManager 2021-05-13 16:22:16 +08:00
zengqiao
f88a14ac0a 补充说明为什么删除Topic成功之后, 又立马出现的原因 2021-05-13 16:22:05 +08:00
EricZeng
9286761c30 更新体验地址
更新体验地址
2021-05-13 10:26:07 +08:00
zengqiao
07c3273247 增加v2.4.0更新内容 2021-05-12 20:16:47 +08:00
tangcongfa_v@didichuxing.com
eb8fe77582 配额调整 2021-05-12 11:31:41 +08:00
tangcongfa_v@didichuxing.com
b68ba0bff6 配额调整 2021-05-11 16:58:44 +08:00
tangcongfa_v@didichuxing.com
696657c09e 处理冲突 2021-05-11 10:40:58 +08:00
tcf1207239873
12bea9b60a Merge pull request #1 from didi/dev
Dev
2021-05-11 10:28:02 +08:00
tangcongfa_v@didichuxing.com
9334e9552f 处理冲突 2021-05-11 10:25:25 +08:00
tangcongfa_v@didichuxing.com
a43b04a98b 处理冲突 2021-05-11 10:23:25 +08:00
zengqiao
f359ff995d faq中增加应用下线失败的说明 2021-05-11 08:41:37 +08:00
tangcongfa_v@didichuxing.com
9185d2646b 权限申请 2021-05-10 16:22:58 +08:00
EricZeng
33e61c762c Merge pull request #285 from didi/dev
bump commons-beanutils version to 1.9.4 & faq补充app的使用说明
2021-05-08 13:51:26 +08:00
zengqiao
e342e646ff 优化因DB时区不对导致周期任务不能被触发的日志 2021-05-08 13:50:04 +08:00
zengqiao
ed163a80e0 bump commons-beanutils version to 1.9.4 2021-05-08 11:45:27 +08:00
zengqiao
b390df08b5 faq补充app的使用说明 2021-05-08 11:28:52 +08:00
tangcongfa_v@didichuxing.com
f0b3b9f7f4 扩分区 2021-05-08 11:23:06 +08:00
EricZeng
a67d732507 Merge pull request #284 from didi/dev
faq中补充说明heartbeat表的作用及集群删除是否影响物理集群的说明
2021-05-08 11:20:51 +08:00
zengqiao
ca0ebe0d75 faq中补充说明heartbeat表的作用及集群删除是否影响物理集群的说明 2021-05-08 11:15:25 +08:00
EricZeng
94d113cbe0 Merge pull request #283 from didi/master
merge master
2021-05-08 11:07:53 +08:00
Hongten
25c3aeaa5f Merge pull request #1 from Hongten/optimize/migration_task_name
optimize the migration task name
2021-05-07 19:26:54 +08:00
Xiang Hong Wei
736d5a00b7 optimize the migration task name 2021-05-07 19:06:47 +08:00
zengqiao
f1627b214c FAQ:新建告警组说明优化 2021-05-07 14:03:09 +08:00
EricZeng
d9265ec7ea Merge pull request #280 from didi/dev
连接信息写DB优化为批量写入
2021-05-07 13:52:43 +08:00
zengqiao
663e871bed 连接信息写DB优化为批量写入 2021-05-07 13:51:34 +08:00
shirenchuang
5c5eaddef7 readme add 社区km文章 2021-05-07 11:34:32 +08:00
shirenchuang
edaec4f1ae readme add 社区km文章 2021-05-07 11:34:18 +08:00
EricZeng
6d19acaa6c 更新FAQ,增加磁盘信息没有数据的说明
更新FAQ,增加磁盘信息没有数据的说明
2021-05-06 18:39:30 +08:00
EricZeng
d29a619fbf 优化页面无数据的FAQ
优化页面无数据的FAQ
2021-05-06 17:54:46 +08:00
EricZeng
b17808dd91 Merge pull request #275 from Huyueeer/master
修改maxMetricsSaveDays字段为Long类型
2021-04-30 18:48:54 +08:00
HuYueeer
c5321a3667 修改maxMetricsSaveDays字段为Long类型 2021-04-30 13:47:49 +08:00
EricZeng
8836691510 Merge pull request #273 from didi/dev
bump jetty-util version to 9.4.39.v20210325
2021-04-29 17:26:40 +08:00
zengqiao
6568f6525d bump jetty-util version to 9.4.39.v20210325 2021-04-29 17:25:08 +08:00
tangcongfa_v@didichuxing.com
473fc27b49 创建topic 2021-04-28 18:05:58 +08:00
EricZeng
74aeb55acb Merge pull request #266 from Liu-XinYuan/responsecode-fix
In case of authentication failure, add a clear response code
2021-04-28 17:25:55 +08:00
Liu-XinYuan
8efcf0529f In case of authentication failure, add a clear response code 2021-04-28 17:14:03 +08:00
EricZeng
06071c2f9c Merge pull request #264 from didi/dev_2.4.0
修复Topic工单选择Broker方式进行审批时展示的还是Region的问题
2021-04-28 14:19:26 +08:00
EricZeng
5eb4eca487 Merge pull request #263 from lucasun/master
修复Topic工单审批时,切换到Broker时展示还是Region的问题
2021-04-28 14:17:11 +08:00
孙超
33f6153e12 V2.4 bugfix 2021-04-28 14:09:54 +08:00
tangcongfa_v@didichuxing.com
df3283f526 删除topic 2021-04-28 11:22:53 +08:00
tangcongfa_v@didichuxing.com
b5901a2819 创建topic 2021-04-28 11:08:01 +08:00
EricZeng
6d5f1402fe Merge pull request #262 from didi/dev
增加App与Topic自动化审批开关
2021-04-28 10:51:12 +08:00
zengqiao
65e3782b2e 增加App与Topic自动化审批开关 2021-04-28 10:48:17 +08:00
EricZeng
135981dd30 Merge pull request #261 from didi/dev
开放接口集群ID开始调整, op对应的是物理集群ID, normal对应的是逻辑集群ID, 第一步, util接口调整
2021-04-28 10:15:42 +08:00
zengqiao
fe5cf2d922 开放接口集群ID开始调整, op对应的是物理集群ID, normal对应的是逻辑集群ID, 第一步, util接口调整 2021-04-28 10:10:16 +08:00
EricZeng
e15425cc2e 修复错别字
修复错别字
2021-04-27 17:36:44 +08:00
EricZeng
c3cb0a4e33 优化FAQ中告警组创建的说明
优化FAQ中告警组创建的说明
2021-04-27 17:30:18 +08:00
lucasun
cc32976bdd Merge pull request #259 from JokerQueue/master
用户管理页面,前端限制用户删除自己
2021-04-27 14:08:31 +08:00
Joker
bc08318716 用户管理页面,前端限制用户删除自己 2021-04-26 21:15:27 +08:00
tangcongfa_v@didichuxing.com
ee1ab30c2c 创建topic 2021-04-26 17:48:32 +08:00
HuYueeer
7fa1a66f7e 更新Topic资源治理部分 2021-04-26 14:51:12 +08:00
EricZeng
946bf37406 Merge pull request #257 from Huyueeer/patch-1
更新提问:heartbeat表相关
2021-04-26 14:27:10 +08:00
HuYueeer
8706f6931a Update faq.md
更新提问:heartbeat表相关
2021-04-26 14:15:32 +08:00
tangcongfa_v@didichuxing.com
f551674860 创建topic 2021-04-26 11:16:53 +08:00
EricZeng
d90fe0ef07 Merge pull request #253 from didi/master
merge master
2021-04-26 10:01:19 +08:00
EricZeng
bf979fa3b3 Merge pull request #252 from didi/dev_2.4.0
Dev 2.4.0
2021-04-26 09:55:30 +08:00
EricZeng
b3b88891e9 Merge pull request #251 from lucasun/master
v2.4.0
2021-04-25 21:01:33 +08:00
lucasun
01c5de60dc Merge branch 'master' into master 2021-04-25 20:54:10 +08:00
孙超
47b8fe5022 V2.4.1 FE 2021-04-25 20:43:20 +08:00
zengqiao
324b37b875 v2.4.0 be code 2021-04-25 18:11:52 +08:00
zengqiao
76e7e192d8 bump version to 2.4.0 2021-04-25 17:40:47 +08:00
EricZeng
f9f3c4d923 Merge pull request #240 from yangvipguang/docker-dev
Docker容器镜像优化
2021-04-25 17:23:36 +08:00
杨光
a476476bd1 Update Dockerfile
添加进程管理器tini 防止僵尸应用
升级基础镜像到Java 16 alpine 
默认使用官方jar 包
默认开启JMX 监控
2021-04-23 14:10:40 +08:00
杨光
82a60a884a Add files via upload 2021-04-23 14:06:55 +08:00
杨光
f17727de18 Merge pull request #1 from didi/master
同步提交
2021-04-23 11:31:35 +08:00
shirenchuang
f1f33c79f4 Merge branch 'shirc_dev' into dev 2021-04-23 10:26:43 +08:00
shirenchuang
d52eaafdbb 修正一下 用户手册中的 共享集群和独享集群的概念 2021-04-23 10:25:36 +08:00
shirenchuang
e7a3e50ed1 Merge branch 'shirc_dev' into dev 2021-04-23 10:20:01 +08:00
shirenchuang
2e09a87baa Merge remote-tracking branch 'origin/master' into shirc_dev 2021-04-23 10:18:30 +08:00
shirenchuang
b92ae7e47e 修正一个注释 2021-04-23 10:18:18 +08:00
EricZeng
f98446e139 Merge pull request #239 from Liu-XinYuan/i238
fix  create topic failed when not specify peak_byte_in
2021-04-22 19:02:48 +08:00
Liu-XinYuan
57a48dadaa modify from obj ==null to ValidateUtils.isNull 2021-04-22 18:44:34 +08:00
Liu-XinYuan
c65ec68e46 fix create topic failed when not specify peak_byte_in 2021-04-22 18:30:23 +08:00
zengqiao
d6559be3fc 部分后台任务获取Topic列表时不走缓存 2021-04-22 16:06:37 +08:00
shirenchuang
6fbf67f9a9 Merge branch 'dev' into shirc_dev 2021-04-22 14:13:32 +08:00
zengqiao
59df5b24fe broker元信息中增加Rack信息 2021-04-20 19:28:36 +08:00
zengqiao
3e1544294b 删除无效代码 2021-04-20 17:22:26 +08:00
EricZeng
a12c398816 Merge pull request #232 from didi/dev
应用下线功能权限列表获取优化
2021-04-20 13:54:49 +08:00
EricZeng
0bd3e28348 Merge pull request #228 from PengShuaixin/dev
应用下线审批功能优化
2021-04-20 13:51:32 +08:00
PengShuaixin
ad4e39c088 应用下线功能权限列表获取优化 2021-04-20 11:22:11 +08:00
PengShuaixin
2668d96e6a Merge remote-tracking branch 'origin/dev' into dev
# Conflicts:
#	kafka-manager-extends/kafka-manager-bpm/src/main/java/com/xiaojukeji/kafka/manager/bpm/order/impl/DeleteAppOrder.java
2021-04-20 11:19:49 +08:00
shirenchuang
357c496aad 下线应用的时候 判断先下线topic 2021-04-20 11:18:07 +08:00
shirenchuang
22a513ba22 升级mysql驱动;支持Mysql 8.0+ 2021-04-20 11:18:07 +08:00
zengqiao
e6dd1119be 通过获取类的RequestMapping注解来判断当前请求是否需要登录 2021-04-20 11:18:07 +08:00
EricZeng
2dbe454e04 Merge pull request #231 from didi/master
merge master
2021-04-20 10:46:03 +08:00
zengqiao
e3a59b76eb 修复数据删空之后, 缓存不能被更新的BUG 2021-04-19 20:31:40 +08:00
shirenchuang
01008acfcd 本地sql配置 2021-04-19 15:35:48 +08:00
shirenchuang
8bfde9fbaf Merge branch 'shirc_dev' into dev 2021-04-19 10:19:11 +08:00
shirenchuang
1fdecf8def 下线应用的时候 判断先下线topic 2021-04-19 10:17:29 +08:00
PengShuaixin
f41e29ab3a 应用下线功能优化 2021-04-14 12:29:59 +08:00
shirenchuang
5f1a839620 升级mysql驱动;支持Mysql 8.0+ 2021-04-06 12:09:52 +08:00
192 changed files with 3068 additions and 957 deletions

View File

@@ -1,20 +1,21 @@
---
![kafka-manager-logo](./docs/assets/images/common/logo_name.png)
![logikm_logo](https://user-images.githubusercontent.com/71620349/125024570-9e07a100-e0b3-11eb-8ebc-22e73e056771.png)
**一站式`Apache Kafka`集群指标监控与运维管控平台**
阅读本README文档您可以了解到滴滴Logi-KafkaManager的用户群体、产品定位等信息并通过体验地址快速体验Kafka集群指标监控与运维管控的全流程。<br>若滴滴Logi-KafkaManager已在贵司的生产环境进行使用并想要获得官方更好地支持和指导可以通过[`OCE认证`](http://obsuite.didiyun.com/open/openAuth),加入官方交流平台。
阅读本README文档您可以了解到滴滴Logi-KafkaManager的用户群体、产品定位等信息并通过体验地址快速体验Kafka集群指标监控与运维管控的全流程。
## 1 产品简介
滴滴Logi-KafkaManager脱胎于滴滴内部多年的Kafka运营实践经验是面向Kafka用户、Kafka运维人员打造的共享多租户Kafka云平台。专注于Kafka运维管控、监控告警、资源治理等核心场景经历过大规模集群、海量大数据的考验。内部满意度高达90%的同时,还与多家知名企业达成商业化合作。
### 1.1 快速体验地址
- 体验地址 http://117.51.146.109:8080 账号密码 admin/admin
- 体验地址 http://117.51.150.133:8080 账号密码 admin/admin
### 1.2 体验地图
相比较于同类产品的用户视角单一大多为管理员视角滴滴Logi-KafkaManager建立了基于分角色、多场景视角的体验地图。分别是**用户体验地图、运维体验地图、运营体验地图**
@@ -66,20 +67,23 @@
- [滴滴Logi-KafkaManager 开源之路](https://xie.infoq.cn/article/0223091a99e697412073c0d64)
- [滴滴Logi-KafkaManager 系列视频教程](https://mp.weixin.qq.com/s/9X7gH0tptHPtfjPPSdGO8g)
- [kafka实践十五滴滴开源Kafka管控平台 Logi-KafkaManager研究--A叶子叶来](https://blog.csdn.net/yezonggang/article/details/113106244)
- [kafka的灵魂伴侣Logi-KafkaManager系列文章专栏 --石臻](https://blog.csdn.net/u010634066/category_10977588.html)
## 3 滴滴Logi开源用户交流群
![image](https://user-images.githubusercontent.com/5287750/111266722-e531d800-8665-11eb-9242-3484da5a3099.png)
微信加群:关注公众号 Obsuite(官方公众号) 回复 "Logi加群"
微信加群:添加mike_zhangliang的微信号备注Logi加群或关注公众号 云原生可观测性 回复 "Logi加群"
![dingding_group](./docs/assets/images/common/dingding_group.jpg)
钉钉群ID32821440
## 4 知识星球
![image](https://user-images.githubusercontent.com/51046167/140718512-5ab1b336-5c48-46c0-90bd-44b5c7e168c8.png)
## 4 OCE认证
OCE是一个认证机制和交流平台为滴滴Logi-KafkaManager生产用户量身打造我们会为OCE企业提供更好的技术支持比如专属的技术沙龙、企业一对一的交流机会、专属的答疑群等如果贵司Logi-KafkaManager上了生产[快来加入吧](http://obsuite.didiyun.com/open/openAuth)
✅知识星球首个【Kafka中文社区】内测期免费加入https://z.didi.cn/5gSF9
有问必答~
互动有礼~
1600+群友一起共建国内最专业的【Kafka中文社区】
PS:提问请尽量把问题一次性描述清楚,并告知环境信息情况哦~!如使用版本、操作步骤、报错/警告信息等,方便嘉宾们快速解答~
## 5 项目成员
@@ -95,4 +99,4 @@ OCE是一个认证机制和交流平台为滴滴Logi-KafkaManager生产用户
## 6 协议
`kafka-manager`基于`Apache-2.0`协议进行分发和使用,更多信息参见[协议文件](./LICENSE)
`LogiKM`基于`Apache-2.0`协议进行分发和使用,更多信息参见[协议文件](./LICENSE)

View File

@@ -7,6 +7,50 @@
---
## v2.4.1+
版本上线时间2021-05-21
### 能力提升
- 增加直接增加权限和配额的接口(v2.4.1)
- 增加接口调用可绕过登录的功能(v2.4.1)
### 体验优化
- tomcat 版本提升至8.5.66(v2.4.2)
- op接口优化拆分util接口为topic、leader两类接口(v2.4.1)
- 简化Gateway配置的Key长度(v2.4.1)
### bug修复
- 修复页面展示版本错误问题(v2.4.2)
## v2.4.0
版本上线时间2021-05-18
### 能力提升
- 增加App与Topic自动化审批开关
- Broker元信息中增加Rack信息
- 升级MySQL 驱动支持MySQL 8+
- 增加操作记录查询界面
### 体验优化
- FAQ告警组说明优化
- 用户手册共享及 独享集群概念优化
- 用户管理界面,前端限制用户删除自己
### bug修复
- 修复op-util类中创建Topic失败的接口
- 周期同步Topic到DB的任务修复将Topic列表查询从缓存调整为直接查DB
- 应用下线审批失败的功能修复将权限为0(无权限)的数据进行过滤
- 修复登录及权限绕过的漏洞
- 修复研发角色展示接入集群、暂停监控等按钮的问题
## v2.3.0
版本上线时间2021-02-08

View File

@@ -1,72 +0,0 @@
#!/bin/bash
workspace=$(cd $(dirname $0) && pwd -P)
cd $workspace
## constant
OUTPUT_DIR=./output
KM_VERSION=2.3.1
APP_NAME=kafka-manager
APP_DIR=${APP_NAME}-${KM_VERSION}
MYSQL_TABLE_SQL_FILE=./docs/install_guide/create_mysql_table.sql
CONFIG_FILE=./kafka-manager-web/src/main/resources/application.yml
## function
function build() {
# 编译命令
mvn -U clean package -Dmaven.test.skip=true
local sc=$?
if [ $sc -ne 0 ];then
## 编译失败, 退出码为 非0
echo "$APP_NAME build error"
exit $sc
else
echo "$APP_NAME build ok"
fi
}
function make_output() {
# 新建output目录
rm -rf ${OUTPUT_DIR} &>/dev/null
mkdir -p ${OUTPUT_DIR}/${APP_DIR} &>/dev/null
# 填充output目录, output内的内容
(
cp -rf ${MYSQL_TABLE_SQL_FILE} ${OUTPUT_DIR}/${APP_DIR} && # 拷贝 sql 初始化脚本 至output目录
cp -rf ${CONFIG_FILE} ${OUTPUT_DIR}/${APP_DIR} && # 拷贝 application.yml 至output目录
# 拷贝程序包到output路径
cp kafka-manager-web/target/kafka-manager-web-${KM_VERSION}-SNAPSHOT.jar ${OUTPUT_DIR}/${APP_DIR}/${APP_NAME}.jar
echo -e "make output ok."
) || { echo -e "make output error"; exit 2; } # 填充output目录失败后, 退出码为 非0
}
function make_package() {
# 压缩output目录
(
cd ${OUTPUT_DIR} && tar cvzf ${APP_DIR}.tar.gz ${APP_DIR}
echo -e "make package ok."
) || { echo -e "make package error"; exit 2; } # 压缩output目录失败后, 退出码为 非0
}
##########################################
## main
## 其中,
## 1.进行编译
## 2.生成部署包output
## 3.生成tar.gz压缩包
##########################################
# 1.进行编译
build
# 2.生成部署包output
make_output
# 3.生成tar.gz压缩包
make_package
# 编译成功
echo -e "build done"
exit 0

View File

@@ -1,44 +1,28 @@
FROM openjdk:8-jdk-alpine3.9
FROM openjdk:16-jdk-alpine3.13
LABEL author="yangvipguang"
LABEL author="fengxsong"
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories && apk add --no-cache tini
ENV VERSION 2.1.0
ENV JAR_PATH kafka-manager-web/target
COPY $JAR_PATH/kafka-manager-web-$VERSION-SNAPSHOT.jar /tmp/app.jar
COPY $JAR_PATH/application.yml /km/
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories
RUN apk add --no-cache --virtual .build-deps \
font-adobe-100dpi \
ttf-dejavu \
fontconfig \
curl \
apr \
apr-util \
apr-dev \
tomcat-native \
&& apk del .build-deps
ENV VERSION 2.4.2
WORKDIR /opt/
ENV AGENT_HOME /opt/agent/
WORKDIR /tmp
COPY docker-depends/config.yaml $AGENT_HOME
COPY docker-depends/jmx_prometheus_javaagent-0.14.0.jar $AGENT_HOME
ENV JAVA_AGENT="-javaagent:$AGENT_HOME/jmx_prometheus_javaagent-0.14.0.jar=9999:$AGENT_HOME/config.yaml"
COPY docker-depends/jmx_prometheus_javaagent-0.15.0.jar $AGENT_HOME
ENV JAVA_AGENT="-javaagent:$AGENT_HOME/jmx_prometheus_javaagent-0.15.0.jar=9999:$AGENT_HOME/config.yaml"
ENV JAVA_HEAP_OPTS="-Xms1024M -Xmx1024M -Xmn100M "
ENV JAVA_OPTS="-verbose:gc \
-XX:+PrintGC -XX:+PrintGCDetails -XX:+PrintHeapAtGC -Xloggc:/tmp/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps \
-XX:MaxMetaspaceSize=256M -XX:+DisableExplicitGC -XX:+UseStringDeduplication \
-XX:+UseG1GC -XX:+HeapDumpOnOutOfMemoryError -XX:-UseContainerSupport"
#-Xlog:gc -Xlog:gc* -Xlog:gc+heap=trace -Xlog:safepoint
RUN wget https://github.com/didi/Logi-KafkaManager/releases/download/v${VERSION}/kafka-manager-${VERSION}.tar.gz && \
tar xvf kafka-manager-${VERSION}.tar.gz && \
mv kafka-manager-${VERSION}/kafka-manager.jar /opt/app.jar && \
rm -rf kafka-manager-${VERSION}*
EXPOSE 8080 9999
ENTRYPOINT ["sh","-c","java -jar $JAVA_HEAP_OPTS $JAVA_OPTS /tmp/app.jar --spring.config.location=/km/application.yml"]
## 默认不带Prometheus JMX监控需要可以自行取消以下注释并注释上面一行默认Entrypoint 命令。
## ENTRYPOINT ["sh","-c","java -jar $JAVA_AGENT $JAVA_HEAP_OPTS $JAVA_OPTS /tmp/app.jar --spring.config.location=/km/application.yml"]
ENTRYPOINT ["tini", "--"]
CMD [ "sh", "-c", "java -jar $JAVA_AGENT $JAVA_HEAP_OPTS $JAVA_OPTS app.jar --spring.config.location=application.yml"]

View File

@@ -0,0 +1,6 @@
dependencies:
- name: mysql
repository: https://charts.bitnami.com/bitnami
version: 8.6.3
digest: sha256:d250c463c1d78ba30a24a338a06a551503c7a736621d974fe4999d2db7f6143e
generated: "2021-06-24T11:34:54.625217+08:00"

View File

@@ -1,6 +1,6 @@
apiVersion: v2
name: didi-km
description: A Helm chart for Kubernetes
description: Logi-KafkaManager
# A chart can be either an 'application' or a 'library' chart.
#
@@ -21,4 +21,9 @@ version: 0.1.0
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: "1.16.0"
appVersion: "2.4.2"
dependencies:
- condition: mysql.enabled
name: mysql
repository: https://charts.bitnami.com/bitnami
version: 8.x.x

Binary file not shown.

View File

@@ -1,7 +1,17 @@
{{- define "datasource.mysql" -}}
{{- if .Values.mysql.enabled }}
{{- printf "%s-mysql" (include "didi-km.fullname" .) -}}
{{- else -}}
{{- printf "%s" .Values.externalDatabase.host -}}
{{- end -}}
{{- end -}}
apiVersion: v1
kind: ConfigMap
metadata:
name: km-cm
name: {{ include "didi-km.fullname" . }}-configs
labels:
{{- include "didi-km.labels" . | nindent 4 }}
data:
application.yml: |
server:
@@ -17,9 +27,9 @@ data:
name: kafkamanager
datasource:
kafka-manager:
jdbc-url: jdbc:mysql://xxxxx:3306/kafka-manager?characterEncoding=UTF-8&serverTimezone=GMT%2B8&useSSL=false
username: admin
password: admin
jdbc-url: jdbc:mysql://{{ include "datasource.mysql" . }}:3306/{{ .Values.mysql.auth.database }}?characterEncoding=UTF-8&serverTimezone=GMT%2B8&useSSL=false
username: {{ .Values.mysql.auth.username }}
password: {{ .Values.mysql.auth.password }}
driver-class-name: com.mysql.jdbc.Driver
main:
allow-bean-definition-overriding: true
@@ -54,7 +64,10 @@ data:
sync-topic-enabled: false # 未落盘的Topic定期同步到DB中
account:
# ldap settings
ldap:
enabled: false
authUserRegistration: false
kcm:
enabled: false

View File

@@ -42,6 +42,10 @@ spec:
protocol: TCP
resources:
{{- toYaml .Values.resources | nindent 12 }}
volumeMounts:
- name: configs
mountPath: /tmp/application.yml
subPath: application.yml
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
@@ -54,3 +58,7 @@ spec:
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
volumes:
- name: configs
configMap:
name: {{ include "didi-km.fullname" . }}-configs

View File

@@ -5,13 +5,14 @@
replicaCount: 1
image:
repository: docker.io/yangvipguang/km
repository: docker.io/fengxsong/logi-kafka-manager
pullPolicy: IfNotPresent
# Overrides the image tag whose default is the chart appVersion.
tag: "v18"
tag: "v2.4.2"
imagePullSecrets: []
nameOverride: ""
# fullnameOverride must set same as release name
fullnameOverride: "km"
serviceAccount:
@@ -59,10 +60,10 @@ resources:
# resources, such as Minikube. If you do want to specify resources, uncomment the following
# lines, adjust them as necessary, and remove the curly braces after 'resources:'.
limits:
cpu: 50m
cpu: 500m
memory: 2048Mi
requests:
cpu: 10m
cpu: 100m
memory: 200Mi
autoscaling:
@@ -77,3 +78,16 @@ nodeSelector: {}
tolerations: []
affinity: {}
# more configurations are set with configmap in file template/configmap.yaml
externalDatabase:
host: ""
mysql:
# if enabled is set to false, then you should manually specified externalDatabase.host
enabled: true
architecture: standalone
auth:
rootPassword: "s3cretR00t"
database: "logi_kafka_manager"
username: "logi_kafka_manager"
password: "n0tp@55w0rd"

View File

@@ -0,0 +1,16 @@
#!/bin/bash
cd `dirname $0`/../target
target_dir=`pwd`
pid=`ps ax | grep -i 'kafka-manager' | grep ${target_dir} | grep java | grep -v grep | awk '{print $1}'`
if [ -z "$pid" ] ; then
echo "No kafka-manager running."
exit -1;
fi
echo "The kafka-manager (${pid}) is running..."
kill ${pid}
echo "Send shutdown request to kafka-manager (${pid}) OK"

View File

@@ -0,0 +1,81 @@
error_exit ()
{
echo "ERROR: $1 !!"
exit 1
}
[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=$HOME/jdk/java
[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=/usr/java
[ ! -e "$JAVA_HOME/bin/java" ] && unset JAVA_HOME
if [ -z "$JAVA_HOME" ]; then
if $darwin; then
if [ -x '/usr/libexec/java_home' ] ; then
export JAVA_HOME=`/usr/libexec/java_home`
elif [ -d "/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home" ]; then
export JAVA_HOME="/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home"
fi
else
JAVA_PATH=`dirname $(readlink -f $(which javac))`
if [ "x$JAVA_PATH" != "x" ]; then
export JAVA_HOME=`dirname $JAVA_PATH 2>/dev/null`
fi
fi
if [ -z "$JAVA_HOME" ]; then
error_exit "Please set the JAVA_HOME variable in your environment, We need java(x64)! jdk8 or later is better!"
fi
fi
export WEB_SERVER="kafka-manager"
export JAVA_HOME
export JAVA="$JAVA_HOME/bin/java"
export BASE_DIR=`cd $(dirname $0)/..; pwd`
export CUSTOM_SEARCH_LOCATIONS=file:${BASE_DIR}/conf/
#===========================================================================================
# JVM Configuration
#===========================================================================================
JAVA_OPT="${JAVA_OPT} -server -Xms2g -Xmx2g -Xmn1g -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"
JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=${BASE_DIR}/logs/java_heapdump.hprof"
## jdk版本高的情况 有些 参数废弃了
JAVA_MAJOR_VERSION=$($JAVA -version 2>&1 | sed -E -n 's/.* version "([0-9]*).*$/\1/p')
if [[ "$JAVA_MAJOR_VERSION" -ge "9" ]] ; then
JAVA_OPT="${JAVA_OPT} -Xlog:gc*:file=${BASE_DIR}/logs/km_gc.log:time,tags:filecount=10,filesize=102400"
else
JAVA_OPT="${JAVA_OPT} -Djava.ext.dirs=${JAVA_HOME}/jre/lib/ext:${JAVA_HOME}/lib/ext"
JAVA_OPT="${JAVA_OPT} -Xloggc:${BASE_DIR}/logs/km_gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M"
fi
JAVA_OPT="${JAVA_OPT} -jar ${BASE_DIR}/target/${WEB_SERVER}.jar"
JAVA_OPT="${JAVA_OPT} --spring.config.additional-location=${CUSTOM_SEARCH_LOCATIONS}"
JAVA_OPT="${JAVA_OPT} --logging.config=${BASE_DIR}/conf/logback-spring.xml"
JAVA_OPT="${JAVA_OPT} --server.max-http-header-size=524288"
if [ ! -d "${BASE_DIR}/logs" ]; then
mkdir ${BASE_DIR}/logs
fi
echo "$JAVA ${JAVA_OPT}"
# check the start.out log output file
if [ ! -f "${BASE_DIR}/logs/start.out" ]; then
touch "${BASE_DIR}/logs/start.out"
fi
# start
echo -e "---- 启动脚本 ------\n $JAVA ${JAVA_OPT}" > ${BASE_DIR}/logs/start.out 2>&1 &
nohup $JAVA ${JAVA_OPT} >> ${BASE_DIR}/logs/start.out 2>&1 &
echo "${WEB_SERVER} is startingyou can check the ${BASE_DIR}/logs/start.out"

View File

@@ -0,0 +1,28 @@
## kafka-manager的配置文件该文件中的配置会覆盖默认配置
## 下面的配置信息基本就是jar中的 application.yml默认配置了;
## 可以只修改自己变更的配置,其他的删除就行了; 比如只配置一下mysql
server:
port: 8080
tomcat:
accept-count: 1000
max-connections: 10000
max-threads: 800
min-spare-threads: 100
spring:
application:
name: kafkamanager
profiles:
active: dev
datasource:
kafka-manager:
jdbc-url: jdbc:mysql://localhost:3306/logi_kafka_manager?characterEncoding=UTF-8&useSSL=false&serverTimezone=GMT%2B8
username: root
password: 123456
driver-class-name: com.mysql.cj.jdbc.Driver
main:
allow-bean-definition-overriding: true

View File

@@ -0,0 +1,116 @@
## kafka-manager的配置文件该文件中的配置会覆盖默认配置
## 下面的配置信息基本就是jar中的 application.yml默认配置了;
## 可以只修改自己变更的配置,其他的删除就行了; 比如只配置一下mysql
server:
port: 8080
tomcat:
accept-count: 1000
max-connections: 10000
max-threads: 800
min-spare-threads: 100
spring:
application:
name: kafkamanager
profiles:
active: dev
datasource:
kafka-manager:
jdbc-url: jdbc:mysql://localhost:3306/logi_kafka_manager?characterEncoding=UTF-8&useSSL=false&serverTimezone=GMT%2B8
username: root
password: 123456
driver-class-name: com.mysql.cj.jdbc.Driver
main:
allow-bean-definition-overriding: true
servlet:
multipart:
max-file-size: 100MB
max-request-size: 100MB
logging:
config: classpath:logback-spring.xml
custom:
idc: cn # 部署的数据中心, 忽略该配置, 后续会进行删除
jmx:
max-conn: 10 # 2.3版本配置不在这个地方生效
store-metrics-task:
community:
broker-metrics-enabled: true # 社区部分broker metrics信息收集开关, 关闭之后metrics信息将不会进行收集及写DB
topic-metrics-enabled: true # 社区部分topic的metrics信息收集开关, 关闭之后metrics信息将不会进行收集及写DB
didi:
app-topic-metrics-enabled: false # 滴滴埋入的指标, 社区AK不存在该指标因此默认关闭
topic-request-time-metrics-enabled: false # 滴滴埋入的指标, 社区AK不存在该指标因此默认关闭
topic-throttled-metrics: false # 滴滴埋入的指标, 社区AK不存在该指标因此默认关闭
save-days: 7 #指标在DB中保持的天数-1表示永久保存7表示保存近7天的数据
# 任务相关的开关
task:
op:
sync-topic-enabled: false # 未落盘的Topic定期同步到DB中
order-auto-exec: # 工单自动化审批线程的开关
topic-enabled: false # Topic工单自动化审批开关, false:关闭自动化审批, true:开启
app-enabled: false # App工单自动化审批开关, false:关闭自动化审批, true:开启
# ldap相关的配置
account:
ldap:
enabled: false
url: ldap://127.0.0.1:389/
basedn: dc=tsign,dc=cn
factory: com.sun.jndi.ldap.LdapCtxFactory
filter: sAMAccountName
security:
authentication: simple
principal: cn=admin,dc=tsign,dc=cn
credentials: admin
auth-user-registration: true
auth-user-registration-role: normal
# 集群升级部署相关的功能需要配合夜莺及S3进行使用
kcm:
enabled: false
s3:
endpoint: s3.didiyunapi.com
access-key: 1234567890
secret-key: 0987654321
bucket: logi-kafka
n9e:
base-url: http://127.0.0.1:8004
user-token: 12345678
timeout: 300
account: root
script-file: kcm_script.sh
# 监控告警相关的功能,需要配合夜莺进行使用
# enabled: 表示是否开启监控告警的功能, true: 开启, false: 不开启
# n9e.nid: 夜莺的节点ID
# n9e.user-token: 用户的密钥,在夜莺的个人设置中
# n9e.mon.base-url: 监控地址
# n9e.sink.base-url: 数据上报地址
# n9e.rdb.base-url: 用户资源中心地址
monitor:
enabled: false
n9e:
nid: 2
user-token: 1234567890
mon:
base-url: http://127.0.0.1:8000 # 夜莺v4版本默认端口统一调整为了8000
sink:
base-url: http://127.0.0.1:8000 # 夜莺v4版本默认端口统一调整为了8000
rdb:
base-url: http://127.0.0.1:8000 # 夜莺v4版本默认端口统一调整为了8000
notify: # 通知的功能
kafka: # 默认通知发送到kafka的指定Topic中
cluster-id: 95 # Topic的集群ID
topic-name: didi-kafka-notify # Topic名称
order: # 部署的KM的地址
detail-url: http://127.0.0.1

View File

@@ -210,11 +210,11 @@ CREATE TABLE `gateway_config` (
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_type_name` (`type`,`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='gateway配置';
INSERT INTO gateway_config(type, name, value, `version`) values('SERVICE_DISCOVERY_QUEUE_SIZE', 'SERVICE_DISCOVERY_QUEUE_SIZE', 100000000, 1);
INSERT INTO gateway_config(type, name, value, `version`) values('SERVICE_DISCOVERY_APPID_RATE', 'SERVICE_DISCOVERY_APPID_RATE', 100000000, 1);
INSERT INTO gateway_config(type, name, value, `version`) values('SERVICE_DISCOVERY_IP_RATE', 'SERVICE_DISCOVERY_IP_RATE', 100000000, 1);
INSERT INTO gateway_config(type, name, value, `version`) values('SERVICE_DISCOVERY_SP_RATE', 'app_01234567', 100000000, 1);
INSERT INTO gateway_config(type, name, value, `version`) values('SERVICE_DISCOVERY_SP_RATE', '192.168.0.1', 100000000, 1);
INSERT INTO gateway_config(type, name, value, `version`, `description`) values('SD_QUEUE_SIZE', 'SD_QUEUE_SIZE', 100000000, 1, '任意集群队列大小');
INSERT INTO gateway_config(type, name, value, `version`, `description`) values('SD_APP_RATE', 'SD_APP_RATE', 100000000, 1, '任意一个App限速');
INSERT INTO gateway_config(type, name, value, `version`, `description`) values('SD_IP_RATE', 'SD_IP_RATE', 100000000, 1, '任意一个IP限速');
INSERT INTO gateway_config(type, name, value, `version`, `description`) values('SD_SP_RATE', 'app_01234567', 100000000, 1, '指定App限速');
INSERT INTO gateway_config(type, name, value, `version`, `description`) values('SD_SP_RATE', '192.168.0.1', 100000000, 1, '指定IP限速');
--
-- Table structure for table `heartbeat`

View File

@@ -0,0 +1,215 @@
<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true" scanPeriod="10 seconds">
<contextName>logback</contextName>
<property name="log.path" value="./logs" />
<!-- 彩色日志 -->
<!-- 彩色日志依赖的渲染类 -->
<conversionRule conversionWord="clr" converterClass="org.springframework.boot.logging.logback.ColorConverter" />
<conversionRule conversionWord="wex" converterClass="org.springframework.boot.logging.logback.WhitespaceThrowableProxyConverter" />
<conversionRule conversionWord="wEx" converterClass="org.springframework.boot.logging.logback.ExtendedWhitespaceThrowableProxyConverter" />
<!-- 彩色日志格式 -->
<property name="CONSOLE_LOG_PATTERN" value="${CONSOLE_LOG_PATTERN:-%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(${LOG_LEVEL_PATTERN:-%5p}) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %m%n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}}"/>
<!--输出到控制台-->
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>info</level>
</filter>
<encoder>
<Pattern>${CONSOLE_LOG_PATTERN}</Pattern>
<charset>UTF-8</charset>
</encoder>
</appender>
<!--输出到文件-->
<!-- 时间滚动输出 level为 DEBUG 日志 -->
<appender name="DEBUG_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${log.path}/log_debug.log</file>
<!--日志文件输出格式-->
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg%n</pattern>
<charset>UTF-8</charset> <!-- 设置字符集 -->
</encoder>
<!-- 日志记录器的滚动策略,按日期,按大小记录 -->
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- 日志归档 -->
<fileNamePattern>${log.path}/log_debug_%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<!--日志文件保留天数-->
<maxHistory>7</maxHistory>
</rollingPolicy>
<!-- 此日志文件只记录debug级别的 -->
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>debug</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>DENY</onMismatch>
</filter>
</appender>
<!-- 时间滚动输出 level为 INFO 日志 -->
<appender name="INFO_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<!-- 正在记录的日志文件的路径及文件名 -->
<file>${log.path}/log_info.log</file>
<!--日志文件输出格式-->
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg%n</pattern>
<charset>UTF-8</charset>
</encoder>
<!-- 日志记录器的滚动策略,按日期,按大小记录 -->
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- 每天日志归档路径以及格式 -->
<fileNamePattern>${log.path}/log_info_%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<!--日志文件保留天数-->
<maxHistory>7</maxHistory>
</rollingPolicy>
<!-- 此日志文件只记录info级别的 -->
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>info</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>DENY</onMismatch>
</filter>
</appender>
<!-- 时间滚动输出 level为 WARN 日志 -->
<appender name="WARN_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<!-- 正在记录的日志文件的路径及文件名 -->
<file>${log.path}/log_warn.log</file>
<!--日志文件输出格式-->
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg%n</pattern>
<charset>UTF-8</charset> <!-- 此处设置字符集 -->
</encoder>
<!-- 日志记录器的滚动策略,按日期,按大小记录 -->
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${log.path}/log_warn_%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<!--日志文件保留天数-->
<maxHistory>7</maxHistory>
</rollingPolicy>
<!-- 此日志文件只记录warn级别的 -->
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>warn</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>DENY</onMismatch>
</filter>
</appender>
<!-- 时间滚动输出 level为 ERROR 日志 -->
<appender name="ERROR_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<!-- 正在记录的日志文件的路径及文件名 -->
<file>${log.path}/log_error.log</file>
<!--日志文件输出格式-->
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg%n</pattern>
<charset>UTF-8</charset> <!-- 此处设置字符集 -->
</encoder>
<!-- 日志记录器的滚动策略,按日期,按大小记录 -->
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${log.path}/log_error_%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<!--日志文件保留天数-->
<maxHistory>7</maxHistory>
</rollingPolicy>
<!-- 此日志文件只记录ERROR级别的 -->
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>ERROR</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>DENY</onMismatch>
</filter>
</appender>
<!-- Metrics信息收集日志 -->
<appender name="COLLECTOR_METRICS_LOGGER" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${log.path}/metrics/collector_metrics.log</file>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg%n</pattern>
<charset>UTF-8</charset>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${log.path}/metrics/collector_metrics_%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<maxHistory>3</maxHistory>
</rollingPolicy>
</appender>
<!-- Metrics信息收集日志 -->
<appender name="API_METRICS_LOGGER" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${log.path}/metrics/api_metrics.log</file>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg%n</pattern>
<charset>UTF-8</charset>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${log.path}/metrics/api_metrics_%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<maxHistory>3</maxHistory>
</rollingPolicy>
</appender>
<!-- Metrics信息收集日志 -->
<appender name="SCHEDULED_TASK_LOGGER" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${log.path}/metrics/scheduled_tasks.log</file>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg%n</pattern>
<charset>UTF-8</charset>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${log.path}/metrics/scheduled_tasks_%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<maxHistory>5</maxHistory>
</rollingPolicy>
</appender>
<logger name="COLLECTOR_METRICS_LOGGER" level="DEBUG" additivity="false">
<appender-ref ref="COLLECTOR_METRICS_LOGGER"/>
</logger>
<logger name="API_METRICS_LOGGER" level="DEBUG" additivity="false">
<appender-ref ref="API_METRICS_LOGGER"/>
</logger>
<logger name="SCHEDULED_TASK_LOGGER" level="DEBUG" additivity="false">
<appender-ref ref="SCHEDULED_TASK_LOGGER"/>
</logger>
<logger name="org.apache.ibatis" level="INFO" additivity="false" />
<logger name="org.mybatis.spring" level="INFO" additivity="false" />
<logger name="com.github.miemiedev.mybatis.paginator" level="INFO" additivity="false" />
<root level="info">
<appender-ref ref="CONSOLE" />
<appender-ref ref="DEBUG_FILE" />
<appender-ref ref="INFO_FILE" />
<appender-ref ref="WARN_FILE" />
<appender-ref ref="ERROR_FILE" />
<!--<appender-ref ref="METRICS_LOG" />-->
</root>
<!--生产环境:输出到文件-->
<!--<springProfile name="pro">-->
<!--<root level="info">-->
<!--<appender-ref ref="CONSOLE" />-->
<!--<appender-ref ref="DEBUG_FILE" />-->
<!--<appender-ref ref="INFO_FILE" />-->
<!--<appender-ref ref="ERROR_FILE" />-->
<!--<appender-ref ref="WARN_FILE" />-->
<!--</root>-->
<!--</springProfile>-->
</configuration>

64
distribution/pom.xml Normal file
View File

@@ -0,0 +1,64 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>kafka-manager</artifactId>
<groupId>com.xiaojukeji.kafka</groupId>
<version>${kafka-manager.revision}</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>distribution</artifactId>
<name>distribution</name>
<packaging>pom</packaging>
<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>kafka-manager-web</artifactId>
<version>${kafka-manager.revision}</version>
</dependency>
</dependencies>
<profiles>
<profile>
<id>release-kafka-manager</id>
<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>kafka-manager-web</artifactId>
<version>${kafka-manager.revision}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptors>
<descriptor>release-km.xml</descriptor>
</descriptors>
<tarLongFileMode>posix</tarLongFileMode>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>install</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
<finalName>kafka-manager</finalName>
</build>
</profile>
</profiles>
</project>

22
distribution/readme.md Normal file
View File

@@ -0,0 +1,22 @@
## 说明
### 1.创建mysql数据库文件
> conf/create_mysql_table.sql
### 2. 修改配置文件
> conf/application.yml.example
> 请将application.yml.example 复制一份改名为application.yml
> 并放在同级目录下(conf/); 并修改成自己的配置
> 这里的优先级比jar包内配置文件的默认值高;
>
### 3.启动/关闭kafka-manager
> sh bin/startup.sh 启动
>
> sh shutdown.sh 关闭
>
### 4.升级jar包
> 如果是升级, 可以看看文件 `upgrade_config.md` 的配置变更历史;
>

51
distribution/release-km.xml Executable file
View File

@@ -0,0 +1,51 @@
<?xml version="1.0" encoding="UTF-8"?>
<assembly>
<id>${project.version}</id>
<includeBaseDirectory>true</includeBaseDirectory>
<formats>
<format>dir</format>
<format>tar.gz</format>
<format>zip</format>
</formats>
<fileSets>
<fileSet>
<includes>
<include>conf/**</include>
</includes>
</fileSet>
<fileSet>
<includes>
<include>bin/*</include>
</includes>
<fileMode>0755</fileMode>
</fileSet>
</fileSets>
<files>
<file>
<source>readme.md</source>
<destName>readme.md</destName>
</file>
<file>
<source>upgrade_config.md</source>
<destName>upgrade_config.md</destName>
</file>
<file>
<!--打好的jar包名称和放置目录-->
<source>../kafka-manager-web/target/kafka-manager.jar</source>
<outputDirectory>target/</outputDirectory>
</file>
</files>
<moduleSets>
<moduleSet>
<useAllReactorProjects>true</useAllReactorProjects>
<includes>
<include>com.xiaojukeji.kafka:kafka-manager-web</include>
</includes>
</moduleSet>
</moduleSets>
</assembly>

View File

@@ -0,0 +1,42 @@
## 版本升级配置变更
> 本文件 从 V2.2.0 开始记录; 如果配置有变更则会填写到下文中; 如果没有,则表示无变更;
> 当您从一个很低的版本升级时候,应该依次执行中间有过变更的sql脚本
**一站式`Apache Kafka`集群指标监控与运维管控平台**
---
### 1.升级至`V2.2.0`版本
#### 1.mysql变更
`2.2.0`版本在`cluster`表及`logical_cluster`各增加了一个字段因此需要执行下面的sql进行字段的增加。
```sql
# cluster表中增加jmx_properties字段, 这个字段会用于存储jmx相关的认证以及配置信息
ALTER TABLE `cluster` ADD COLUMN `jmx_properties` TEXT NULL COMMENT 'JMX配置' AFTER `security_properties`;
# logical_cluster中增加identification字段, 同时数据和原先name数据相同, 最后增加一个唯一键.
# 此后, name字段还是表示集群名称, identification字段表示的是集群标识, 只能是字母数字及下划线组成,
# 数据上报到监控系统时, 集群这个标识采用的字段就是identification字段, 之前使用的是name字段.
ALTER TABLE `logical_cluster` ADD COLUMN `identification` VARCHAR(192) NOT NULL DEFAULT '' COMMENT '逻辑集群标识' AFTER `name`;
UPDATE `logical_cluster` SET `identification`=`name` WHERE id>=0;
ALTER TABLE `logical_cluster` ADD INDEX `uniq_identification` (`identification` ASC);
```
### 升级至`2.3.0`版本
#### 1.mysql变更
`2.3.0`版本在`gateway_config`表增加了一个描述说明的字段因此需要执行下面的sql进行字段的增加。
```sql
ALTER TABLE `gateway_config`
ADD COLUMN `description` TEXT NULL COMMENT '描述信息' AFTER `version`;
```

Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 785 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.5 MiB

View File

@@ -14,6 +14,8 @@
- 1、Topic定时同步任务
- 2、专家服务——Topic分区热点
- 3、专家服务——Topic分区不足
- 4、专家服务——Topic资源治理
- 5、账单配置
## 1、Topic定时同步任务
@@ -120,3 +122,47 @@ TOPIC_INSUFFICIENT_PARTITION_CONFIG
]
}
```
## 4、专家服务——Topic资源治理
首先我们认为在一定的时间长度内Topic的分区offset没有任何变化的Topic即没有数据写入的Topic为过期的Topic。
Topic分区不足相关的动态配置(页面在运维管控->平台管理->配置管理)
配置Key
```
EXPIRED_TOPIC_CONFIG
```
配置Value
```json
{
"minExpiredDay": 30, #过期时间大于此值才显示
"ignoreClusterIdList": [ # 忽略的集群
50
]
}
```
## 5、账单配置
Logi-KafkaManager除了作为Kafka运维管控平台之外实际上还会有一些资源定价相关的功能。
当前定价方式当月Topic的maxAvgDay天的峰值的均值流量作为Topic的使用额度。使用的额度 * 单价 * 溢价(预留buffer) 就等于当月的费用。
详细的计算逻辑见com.xiaojukeji.kafka.manager.task.dispatch.biz.CalKafkaTopicBill; 和 com.xiaojukeji.kafka.manager.task.dispatch.biz.CalTopicStatistics;
这块在计算Topic的费用的配置如下所示
配置Key
```
KAFKA_TOPIC_BILL_CONFIG
```
配置Value
```json
{
"maxAvgDay": 10, # 使用额度的计算规则
"quotaRatio": 1.5, # 溢价率
"priseUnitMB": 100 # 单价即单MB/s流量多少钱
}
```

View File

@@ -0,0 +1,53 @@
---
![kafka-manager-logo](../assets/images/common/logo_name.png)
**一站式`Apache Kafka`集群指标监控与运维管控平台**
---
# 如何增加上报监控系统指标?
## 0、前言
LogiKM是 **一站式`Apache Kafka`集群指标监控与运维管控平台** 当前会将消费LagTopic流量等指标上报到监控系统中从而方便用户在监控系统中对这些指标配置监控告警规则进而达到监控自身客户端是否正常的目的。
那么如果我们想增加一个新的监控指标应该如何做呢比如我们想监控Broker的流量监控Broker的存活信息监控集群Controller个数等等。
在具体介绍之前我们大家都知道Kafka监控相关的信息基本都存储于Broker、Jmx以及ZK中。当前LogiKM也已经具备从这三个地方获取数据的基本能力因此基于LogiKM我们再获取其他指标总体上还是非常方便的。
这里我们就以已经获取到的Topic流量信息为例看LogiKM如何实现Topic指标的获取并上报的。
---
## 1、确定指标位置
基于对Kafka的了解我们知道Topic流量信息这个指标是存储于Jmx中的因此我们需要从Jmx中获取。大家如果对于自己所需要获取的指标存储在何处不太清楚的可以加入我们维护的Kafka中文社区(README中有二维码)中今天沟通交流。
---
## 2、指标获取
Topic流量指标的获取详细见图中说明。
![Topic流量指标采集说明](./assets/increase_the_indicators_reported_to_monitor_system/collect_topic_metrics.jpg)
---
## 3、指标上报
上一步我们已经采集到Topic流量指标了下一步就是将该指标上报到监控系统这块只需要按照监控系统要求的格式将数据上报即可。
LogiKM中有一个monitor模块具体的如下图所示
![指标上报](./assets/increase_the_indicators_reported_to_monitor_system/sink_metrcis.png)
## 4、补充说明
监控系统对接的相关内容见:
[监控系统集成](./monitor_system_integrate_with_self.md)
[监控系统集成例子——集成夜莺](./monitor_system_integrate_with_n9e.md)

View File

@@ -58,6 +58,9 @@ custom:
task:
op:
sync-topic-enabled: false # 未落盘的Topic定期同步到DB中
order-auto-exec: # 工单自动化审批线程的开关
topic-enabled: false # Topic工单自动化审批开关, false:关闭自动化审批, true:开启
app-enabled: false # App工单自动化审批开关, false:关闭自动化审批, true:开启
account: # ldap相关的配置, 社区版本暂时支持不够完善,可以先忽略,欢迎贡献代码对这块做优化
ldap:

View File

@@ -31,17 +31,23 @@
**2、源代码进行打包**
下载好代码之后,进入`Logi-KafkaManager`的主目录,执行`sh build.sh`命令即可,执行完成之后会在`output/kafka-manager-xxx`目录下面生成一个jar包。
下载好代码之后,进入`Logi-KafkaManager`的主目录,执行`mvn -Prelease-kafka-manager -Dmaven.test.skip=true clean install -U `命令即可,
执行完成之后会在`distribution/target`目录下面生成一个`kafka-manager-*.tar.gz`
和一个`kafka-manager-*.zip` 文件,随便任意一个压缩包都可以;
当然此时同级目录有一个已经解压好的文件夹;
对于`windows`环境的用户,估计执行不了`sh build.sh`命令,因此可以直接执行`mvn install`,然后在`kafka-manager-web/target`目录下生成一个kafka-manager-web-xxx.jar的包。
获取到jar包之后我们继续下面的步骤。
---
## 3、MySQL-DB初始化
## 3. 解压安装包
解压完成后; 在文件目录中可以看到有`kafka-manager/conf/create_mysql_table.sql` 有个mysql初始化文件
先初始化DB
执行[create_mysql_table.sql](create_mysql_table.sql)中的SQL命令从而创建所需的MySQL库及表默认创建的库名是`logi_kafka_manager`
## 4、MySQL-DB初始化
执行[create_mysql_table.sql](../../distribution/conf/create_mysql_table.sql)中的SQL命令从而创建所需的MySQL库及表默认创建的库名是`logi_kafka_manager`
```
# 示例:
@@ -50,15 +56,38 @@ mysql -uXXXX -pXXX -h XXX.XXX.XXX.XXX -PXXXX < ./create_mysql_table.sql
---
## 4、启动
## 5.修该配置
请将`conf/application.yml.example` 文件复制一份出来命名为`application.yml` 放在同级目录:conf/application.yml ;
并且修改配置; 当然不修改的话 就会用默认的配置;
至少 mysql配置成自己的吧
```
# application.yml 是配置文件最简单的是仅修改MySQL相关的配置即可启动
nohup java -jar kafka-manager.jar --spring.config.location=./application.yml > /dev/null 2>&1 &
```
## 6、启动/关闭
解压包中有启动和关闭脚本
`kafka-manager/bin/shutdown.sh`
`kafka-manager/bin/startup.sh`
### 5、使用
执行 sh startup.sh 启动
执行 sh shutdown.sh 关闭
### 6、使用
本地启动的话,访问`http://localhost:8080`,输入帐号及密码(默认`admin/admin`)进行登录。更多参考:[kafka-manager 用户使用手册](../user_guide/user_guide_cn.md)
### 7. 升级
如果是升级版本,请查看文件 [kafka-manager 升级手册](../../distribution/upgrade_config.md)
在您下载的启动包(V2.5及其后)中也有记录,在 kafka-manager/upgrade_config.md 中
### 8. 在IDE中启动
> 如果想参与开发或者想在IDE中启动的话
> 先执行 `mvn -Dmaven.test.skip=true clean install -U `
>
> 然后这个时候可以选择去 [pom.xml](../../pom.xml) 中将`kafka-manager-console`模块注释掉;
> 注释是因为每次install的时候都会把前端文件`kafka-manager-console`重新打包进`kafka-manager-web`
>
> 完事之后,只需要直接用IDE启动运行`kafka-manager-web`模块中的
> com.xiaojukeji.kafka.manager.web.MainApplication main方法就行了

View File

@@ -0,0 +1,49 @@
---
![kafka-manager-logo](../assets/images/common/logo_name.png)
**一站式`Apache Kafka`集群指标监控与运维管控平台**
---
# 登录绕过
## 背景
现在除了开放出来的第三方接口,其他接口都需要走登录认证。
但是第三方接口不多,开放出来的能力有限,但是登录的接口又需要登录,非常的麻烦。
因此,新增了一个登录绕过的功能,为一些紧急临时的需求,提供一个调用不需要登录的能力。
## 使用方式
步骤一接口调用时在header中增加如下信息
```shell
# 表示开启登录绕过
Trick-Login-Switch : on
# 登录绕过的用户, 这里可以是admin, 或者是其他的, 但是必须在运维管控->平台管理->用户管理中设置了该用户。
Trick-Login-User : admin
```
&nbsp;
步骤二:在运维管控->平台管理->平台配置上,设置允许了该用户以绕过的方式登录
```shell
# 设置的key必须是这个
SECURITY.TRICK_USERS
# 设置的value是json数组的格式例如
[ "admin", "logi"]
```
&nbsp;
步骤三:解释说明
设置完成上面两步之后,就可以直接调用需要登录的接口了。
但是还有一点需要注意,绕过的用户仅能调用他有权限的接口,比如一个普通用户,那么他就只能调用普通的接口,不能去调用运维人员的接口。

View File

@@ -20,8 +20,16 @@
- 8、`topic biz data not exist`错误及处理方式
- 9、进程启动后如何查看API文档
- 10、如何创建告警组
- 11、连接信息、耗时信息为什么没有数据
- 11、连接信息、耗时信息、磁盘信息为什么没有数据?
- 12、逻辑集群申请审批通过之后为什么看不到逻辑集群
- 13、heartbeat表关联业务和使用场景是什么
- 14、集群的删除是否会真正的删除集群
- 15、APP(应用)如何被使用起来?
- 16、为什么下线应用提示operation forbidden
- 17、删除Topic成功为什么过一会儿之后又出现了
- 18、如何在不登录的情况下调用一些需要登录的接口
- 19、为什么无法看到连接信息、耗时信息等指标
- 20、AppID鉴权、生产消费配额不起作用
---
@@ -70,7 +78,7 @@
- 3、数据库时区问题。
检查MySQL的topic_metrics表查看是否有数据如果有数据那么再检查设置的时区是否正确
检查MySQL的cluster表的gmt_modify字段做一个update动作看这个gmt_modify时间是否是当前时间如果不是那么就是时区问题了。时区不对问题具体可以搜索一下看如何解决
---
@@ -113,11 +121,14 @@
### 10、如何创建告警组
这块需要配合监控系统进行使用,现在默认已经实现了夜莺的对接当然也可以对接自己内部的监控系统,不过需要实现一些接口。
告警组的创建需要到Logi-KM对接的监控系统中创建比如我们现在默认是对接了夜莺那么告警组需要到夜莺中创建如果没有安装夜莺那么需要安装一下夜莺并进行对接当然,这里也可以对接自己内部的监控系统,不过需要实现一些接口。
具体的文档可见:[监控功能对接夜莺](../dev_guide/monitor_system_integrate_with_n9e.md)、[监控功能对接其他系统](../dev_guide/monitor_system_integrate_with_self.md)
### 11、连接信息、耗时信息为什么没有数据
那么在夜莺中,如何创建告警组呢
需要前往夜莺平台-用户资源中心-团队管理中新建团队。新建过团队之后再次回到Logi-KM中刷新页面就可以在该下拉框中选择告警接收组了。
### 11、连接信息、耗时信息、磁盘信息为什么没有数据
这块需要结合滴滴内部的kafka-gateway一同使用才会有数据滴滴kafka-gateway暂未开源。
@@ -126,3 +137,79 @@
逻辑集群的申请与审批仅仅只是一个工单流程,并不会去实际创建逻辑集群,逻辑集群的创建还需要手动去创建。
具体的操作可见:[kafka-manager 接入集群](add_cluster/add_cluster.md)。
### 13、heartbeat表关联业务和使用场景是什么
做任务抢占用的。
KM支持HA的方式部署那么部署多台的时候就会出现每一台都可能去做指标收集的事情这块就使用heartbeat表做KM的存活性判断然后进行任务的抢占或者是均衡。
更多详细的内容可以看一下源码中heartbeat表在哪里被使用了。
### 14、集群的删除是否会真正的删除集群
Logi-KM的运维管控集群列表中的集群删除仅仅只是将该集群从Logi-KM中进行删除并不会对真正的物理集群做什么操作。
### 15、APP(应用)如何被使用起来?
app在Logi-KM中可以近似理解为租户或者是kafka里面的一个账号的概念。
界面中显示的app信息、权限信息等在平台层面仅仅只是控制Topic或集群在平台上的可见性如果使用的是社区版本的Kafka那么实际上是不能真正的管控到客户端对Topic的生产和消费。
但是如果是使用的滴滴的Kafka-Gateway那么是可以做到对客户端的生产和消费的权限管控。滴滴的Kafka-Gateway暂未开源属于企业服务具体的可以入群交流群地址在README中。
### 16、为什么下线应用提示operation forbidden
**原因一:**
该应用还存在对Topic的权限因此导致下线失败。具体查看的位置在"Topic管理-》应用管理-》详情"可以看到应用对哪些Topic还有权限。
只有当权限全部去除之后,才可以下线应用。
**原因二:**
使用的是2.4.0之前的旧版本旧版本存在缓存更新的BUG建议升级至最新的版本或者简单粗暴的就是重启一下KM。
### 17、删除Topic成功为什么过一会儿之后又出现了
**原因说明:**
Logi-KM会去请求Topic的endoffset信息要获取这个信息就需要发送metadata请求发送metadata请求的时候如果集群允许自动创建Topic那么当Topic不存在时就会自动将该Topic创建出来。
**问题解决:**
因为在Logi-KM上禁止Kafka客户端内部元信息获取这个动作非常的难做到因此短时间内这个问题不好从Logi-KM上解决。
当然对于不存在的TopicLogi-KM是不会进行元信息请求的因此也不用担心会莫名其妙的创建一个Topic出来。
但是另外一点对于开启允许Topic自动创建的集群建议是关闭该功能开启是非常危险的如果关闭之后Logi-KM也不会有这个问题。
最后这里举个开启这个配置后,非常危险的代码例子吧:
```java
for (int i= 0; i < 100000; ++i) {
// 如果是客户端类似这样写的那么一启动那么将创建10万个Topic出来集群元信息瞬间爆炸controller可能就不可服务了。
producer.send(new ProducerRecord<String, String>("logi_km" + i,"hello logi_km"));
}
```
### 18、如何在不登录的情况下调用一些需要登录的接口
具体见:[登录绕过](./call_api_bypass_login.md)
### 19、为什么无法看到连接信息、耗时信息等指标
连接信息、耗时信息等指标依赖于滴滴kafka-gateway和滴滴Kafka引擎通过gateway可获取到连接到该Topic的应用情况提高对Topic的管控能力。通过滴滴Kafka引擎的自带埋点可获取到耗时信息提升Topic生产消费时的可观测性。这部分内容是属于商业版的范畴暂未开源。如有需要可进行商业合作。
具体见:[滴滴Logi-KafkaManager开源版和商业版特性对比](../开源版与商业版特性对比.md)
### 20、AppID鉴权、生产消费配额不起作用
AppID鉴权、生产消费配额依赖于滴滴kafka-gateway通过gateway进行身份鉴权和生产消费限流可避免用户无限制的使用集群的流量流量大的用户会耗尽系统资源从而影响其他用户的使用造成集群的节点故障。这部分内容是属于商业版的范畴暂未开源。如有需要可进行商业合作。
具体见:[滴滴Logi-KafkaManager开源版和商业版特性对比](../开源版与商业版特性对比.md)

View File

@@ -4,11 +4,16 @@
---
## 报警策略-监控指标说明
| 指标 | 含义 |备注 |
| --- | --- | --- |
| online-kafka-consumer-lag | 消费时按照分区的维度进行监控lag数 | lag表示有多少数据没有被消费因为按照分区的维度监控所以告警时一般会有分区信息 |
| online-kafka-consumer-maxLag | 消费时按照整个Topic的维度监控Topic所有的分区里面的那个最大的lag | 比如每个分区的lag分别是3、5、7那么maxLag的值就是max(3,5,7)=7 |
| online-kafka-consumer-maxDelayTime | 消费时按照Topic维度监控预计的消费延迟 | 这块是按照lag和messagesIn之间的关系计算出来的可能会有误差 |
## 报警策略-报警函数介绍
| 类别 | 函数 | 含义 |函数文案 |备注 |
| --- | --- | --- | --- | --- |
| 发生次数 |alln | 最近$n个周期内全发生 | 连续发生(all) | |

View File

@@ -609,10 +609,10 @@ Lag表示该消费客户端是否有堆积等于 partition offset-consume
集群类型:选择创建的集群为“独享”还是“独立”。
* 独享集群意味着,您独自拥有一个
集群;
* 独享集群意味着, 您拥有一个集群中部分broker的使用权限。
* 独立集群意味着,您拥有一个集群中部分broker的使用权限。
* 独立集群意味着,您独自拥有一个
物理集群;
* <font color = red>共享集群意味着大家共用一个集群及其中broker。</font>

View File

@@ -0,0 +1,55 @@
---
![kafka-manager-logo](assets/images/common/logo_name.png)
**一站式`Apache Kafka`集群指标监控与运维管控平台**
---
**开源版、商业版对比**
纲要Logi-KafakManager的商业特性是强依赖于滴滴Kafka Gateway和滴滴kafka引擎。
滴滴KafkaGateway主要负责服务发现、安全管控身份鉴权、生产消费鉴权等、流量管控应用配额等
滴滴Kafka引擎主要负责更丰富的监控指标broker实时耗时、压缩指标、分区落盘等、磁盘过载保护等
备注两个版本的产品功能页面是一样的。区别在于开源版未使用滴滴KafkaGateway滴滴Kafka引擎部分产品功能/功能不起作用或者页面无数据
| 模块 |对比指标 |底层依赖 |开源版 |商业版 |备注 |
| --- | --- | --- | --- | --- | --- |
| 服务发现 | bootstrap地址变更对客户端无影响 | Gateway | | 是| |
| 安全管控 | 身份鉴权appID+password | Gateway | | 是 | |
| | 权限鉴权Topic+appID | Gateway | | 是 | |
| 指标监控 | Topic实时流量、历史流量 | | 是 | 是 | |
| | Broker实时耗时、历史耗时 | 引擎 | | 是 | |
| | 分区落盘 | 引擎 | | 是 | |
| | Topic里的数据压缩格式 | 引擎 | | 是 | |
| | 连接信息Topic上有哪些应用连接 | Gateway | | 是| |
| | 流量管控(应用配额、生产消费限流等) | Gateway | | 是 | |
| 监控报警 | | | 是 | 是 | 监控指标上报需对接外部监控系统夜莺or企业内部监控系统 |
| Topic运维 | 申请分区 | | 是 | 是 | |
| | 调整配额 | Gateway | | | 是 |
| | Topic数据采样 | | 是 | 是 | |
| | 消费组管理(重置消费偏移等) | | 是 | 是 | |
| 集群管理 | 集群接入(部署) | | 是 | 是 | 需手动部署集群,或借助外部的自动化部署系统(夜莺)来部署系统 |
| | 集群指标监控 | | 是 | 是 | |
| | 按照Region、逻辑集群进行管理 | | 是 | 是 | |
| | Topic迁移 | | 是 | 是| |
| | 集群任务(集群版本管理、升级、扩缩容、回滚等) | | 是 | 是 | 需借助夜莺或自动化部署系统来实现 |
| | 磁盘过载保护 | 引擎 | | 是 | |
| | 指定broker作为优选controller | Gateway | | 是 | |
| Gateway管理 | 管理 Gateway的配置文件 | Gateway | | 是 | |
| 资源治理 | 专家服务Topic分区热点、Topic分区不足、Topic长期未使用、Topic流量异常 | | 是 | 是 | 开源版:具备问题发现与基础的问题解决能力;商业版:可在此基础上,融入滴滴内部的资源治理经验,提供更加专家化的问题解决方法 |
| | 健康分 | | 是 | 是 | 开源版:具备基础的健康分算法;商业版:可融入更多的指标统计,及定制化的健康分算法 |
| 运营管理 | 资源审批应用申请、Topic申请、分区申请、配额申请、集群申请等都需要通过工单进行审批 | |是 | 是 | |
| | 账单体系根据流量核算Topic、集群费用 | | 是 | 是| |
**总结**
滴滴LogiKM的商业特性体现在滴滴Kafka Gateway、滴滴Kafka引擎、内部沉淀出的资源治理专家经验、可定制化的健康分算法。
从场景来看滴滴Logi-KafkaManager的开源版本在kafka集群运维、的Topic管理、监控告警、资源治理等kafka核心场景都充分开源用户的使用需求并且有着出色的表现。而商业版相较于开源版在安全管控、流量管控、更丰富的指标监控、资源治理专家经验的具有明显提升更加符合企业业务需求。
除此之外,商业版还可根据企业实际需求对平台源码进行定制化改造,并提供运维保障,稳定性保障,运营保障等服务。

View File

@@ -1,19 +0,0 @@
package com.xiaojukeji.kafka.manager.common.bizenum;
/**
* @author zengqiao
* @date 20/7/27
*/
public enum ApiLevelEnum {
LEVEL_0(0),
LEVEL_1(1),
LEVEL_2(2),
LEVEL_3(3)
;
private int level;
ApiLevelEnum(int level) {
this.level = level;
}
}

View File

@@ -24,18 +24,10 @@ public enum ConsumeHealthEnum {
return code;
}
public void setCode(Integer code) {
this.code = code;
}
public String getMessage() {
return message;
}
public void setMessage(String message) {
this.message = message;
}
@Override
public String toString() {
return "ConsumeHealthEnum{" +

View File

@@ -19,7 +19,10 @@ public enum DBStatusEnum {
return status;
}
public void setStatus(int status) {
this.status = status;
@Override
public String toString() {
return "DBStatusEnum{" +
"status=" + status +
'}';
}
}

View File

@@ -21,18 +21,10 @@ public enum IDCEnum {
return idc;
}
public void setIdc(String idc) {
this.idc = idc;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
@Override
public String toString() {
return "IDCEnum{" +

View File

@@ -21,10 +21,6 @@ public enum KafkaBrokerRoleEnum {
return role;
}
public void setRole(String role) {
this.role = role;
}
@Override
public String toString() {
return "KafkaBrokerRoleEnum{" +

View File

@@ -24,18 +24,10 @@ public enum KafkaClientEnum {
return code;
}
public void setCode(Integer code) {
this.code = code;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
@Override
public String toString() {
return "KafkaClientEnum{" +

View File

@@ -18,4 +18,11 @@ public enum OffsetResetTypeEnum {
public Integer getCode() {
return code;
}
@Override
public String toString() {
return "OffsetResetTypeEnum{" +
"code=" + code +
'}';
}
}

View File

@@ -46,7 +46,7 @@ public enum OperateEnum {
public static boolean validate(Integer code) {
if (code == null) {
return false;
return true;
}
for (OperateEnum state : OperateEnum.values()) {
if (state.getCode() == code) {

View File

@@ -27,4 +27,12 @@ public enum OperationStatusEnum {
public String getMessage() {
return message;
}
@Override
public String toString() {
return "OperationStatusEnum{" +
"code=" + code +
", message='" + message + '\'' +
'}';
}
}

View File

@@ -15,9 +15,9 @@ public enum PeakFlowStatusEnum {
;
public Integer code;
private Integer code;
public String message;
private String message;
PeakFlowStatusEnum(Integer code, String message) {
this.code = code;
@@ -28,18 +28,10 @@ public enum PeakFlowStatusEnum {
return code;
}
public void setCode(Integer code) {
this.code = code;
}
public String getMessage() {
return message;
}
public void setMessage(String message) {
this.message = message;
}
@Override
public String toString() {
return "PeakFlowStatusEnum{" +

View File

@@ -29,4 +29,12 @@ public enum RebalanceDimensionEnum {
public String getMessage() {
return message;
}
@Override
public String toString() {
return "RebalanceDimensionEnum{" +
"code=" + code +
", message='" + message + '\'' +
'}';
}
}

View File

@@ -43,18 +43,10 @@ public enum TaskStatusEnum {
return code;
}
public void setCode(Integer code) {
this.code = code;
}
public String getMessage() {
return message;
}
public void setMessage(String message) {
this.message = message;
}
@Override
public String toString() {
return "TaskStatusEnum{" +
@@ -64,9 +56,6 @@ public enum TaskStatusEnum {
}
public static Boolean isFinished(Integer code) {
if (code >= FINISHED.getCode()) {
return true;
}
return false;
return code >= FINISHED.getCode();
}
}

View File

@@ -45,11 +45,6 @@ public enum TaskStatusReassignEnum {
}
public static Boolean isFinished(Integer code) {
if (SUCCEED.getCode().equals(code)
|| FAILED.getCode().equals(code)
|| CANCELED.getCode().equals(code)) {
return true;
}
return false;
return SUCCEED.getCode().equals(code) || FAILED.getCode().equals(code) || CANCELED.getCode().equals(code);
}
}

View File

@@ -33,4 +33,12 @@ public enum TopicAuthorityEnum {
public String getMessage() {
return message;
}
@Override
public String toString() {
return "TopicAuthorityEnum{" +
"code=" + code +
", message='" + message + '\'' +
'}';
}
}

View File

@@ -29,4 +29,12 @@ public enum TopicExpiredStatusEnum {
public String getMessage() {
return message;
}
@Override
public String toString() {
return "TopicExpiredStatusEnum{" +
"status=" + status +
", message='" + message + '\'' +
'}';
}
}

View File

@@ -23,18 +23,10 @@ public enum TopicOffsetChangedEnum {
return code;
}
public void setCode(Integer code) {
this.code = code;
}
public String getMessage() {
return message;
}
public void setMessage(String message) {
this.message = message;
}
@Override
public String toString() {
return "TopicOffsetChangedEnum{" +

View File

@@ -5,11 +5,11 @@ package com.xiaojukeji.kafka.manager.common.bizenum.gateway;
* @date 20/7/28
*/
public enum GatewayConfigKeyEnum {
SD_CLUSTER_ID("SERVICE_DISCOVERY_CLUSTER_ID", "SERVICE_DISCOVERY_CLUSTER_ID"),
SD_QUEUE_SIZE("SERVICE_DISCOVERY_QUEUE_SIZE", "SERVICE_DISCOVERY_QUEUE_SIZE"),
SD_APP_ID_RATE("SERVICE_DISCOVERY_APPID_RATE", "SERVICE_DISCOVERY_APPID_RATE"),
SD_IP_RATE("SERVICE_DISCOVERY_IP_RATE", "SERVICE_DISCOVERY_IP_RATE"),
SD_SP_RATE("SERVICE_DISCOVERY_SP_RATE", "SERVICE_DISCOVERY_SP_RATE"),
SD_CLUSTER_ID("SD_CLUSTER_ID", "SD_CLUSTER_ID"),
SD_QUEUE_SIZE("SD_QUEUE_SIZE", "SD_QUEUE_SIZE"),
SD_APP_RATE("SD_APP_RATE", "SD_APP_RATE"),
SD_IP_RATE("SD_IP_RATE", "SD_IP_RATE"),
SD_SP_RATE("SD_SP_RATE", "SD_SP_RATE"),
;
@@ -26,18 +26,10 @@ public enum GatewayConfigKeyEnum {
return configType;
}
public void setConfigType(String configType) {
this.configType = configType;
}
public String getConfigName() {
return configName;
}
public void setConfigName(String configName) {
this.configName = configName;
}
@Override
public String toString() {
return "GatewayConfigKeyEnum{" +

View File

@@ -12,4 +12,7 @@ public class ApiLevelContent {
public static final int LEVEL_NORMAL_3 = 3;
public static final int LEVEL_DEFAULT_4 = 4;
private ApiLevelContent() {
}
}

View File

@@ -20,6 +20,15 @@ public class ApiPrefix {
// open
public static final String API_V1_THIRD_PART_PREFIX = API_V1_PREFIX + "third-part/";
// 开放给OP的接口, 后续对 应的接口的集群都需要是物理集群
public static final String API_V1_THIRD_PART_OP_PREFIX = API_V1_THIRD_PART_PREFIX + "op/";
// 开放给Normal的接口, 后续对应的接口的集群,都需要是逻辑集群
public static final String API_V1_THIRD_PART_NORMAL_PREFIX = API_V1_THIRD_PART_PREFIX + "normal/";
// gateway
public static final String GATEWAY_API_V1_PREFIX = "/gateway" + API_V1_PREFIX;
private ApiPrefix() {
}
}

View File

@@ -30,4 +30,7 @@ public class ConfigConstant {
public static final String BROKER_CAPACITY_LIMIT_CONFIG_KEY = "BROKER_CAPACITY_LIMIT_CONFIG";
public static final String KAFKA_CLUSTER_DO_CONFIG_KEY = "KAFKA_CLUSTER_DO_CONFIG";
private ConfigConstant() {
}
}

View File

@@ -45,4 +45,7 @@ public class Constant {
public static final Integer DEFAULT_MAX_CAL_TOPIC_EXPIRED_DAY = 90;
public static final Integer INVALID_CODE = -1;
private Constant() {
}
}

View File

@@ -16,4 +16,7 @@ public class KafkaConstant {
public static final String CLIENT_VERSION_NAME_UNKNOWN = "unknown";
public static final String RETENTION_MS_KEY = "retention.ms";
private KafkaConstant() {
}
}

View File

@@ -39,4 +39,7 @@ public class KafkaMetricsCollections {
* Broker信息
*/
public static final int BROKER_VERSION = 400;
private KafkaMetricsCollections() {
}
}

View File

@@ -10,4 +10,7 @@ public class LogConstant {
public static final String API_METRICS_LOGGER = "API_METRICS_LOGGER";
public static final String SCHEDULED_TASK_LOGGER = "SCHEDULED_TASK_LOGGER";
private LogConstant() {
}
}

View File

@@ -11,4 +11,7 @@ public class LoginConstant {
public static final String COOKIE_CHINESE_USERNAME_KEY = "chineseName";
public static final Integer COOKIE_OR_SESSION_MAX_AGE_UNIT_MS = 24 * 60 * 60 * 1000;
private LoginConstant() {
}
}

View File

@@ -6,4 +6,7 @@ package com.xiaojukeji.kafka.manager.common.constant;
*/
public class SystemCodeConstant {
public static final String KAFKA_MANAGER = "kafka-manager";
private SystemCodeConstant() {
}
}

View File

@@ -25,6 +25,8 @@ public class TopicCreationConstant {
public static final String TOPIC_RETENTION_TIME_KEY_NAME = "retention.ms";
public static final String TOPIC_RETENTION_BYTES_KEY_NAME = "retention.bytes";
public static final Long DEFAULT_QUOTA = 3 * 1024 * 1024L;
public static Properties createNewProperties(Long retentionTime) {
@@ -54,4 +56,7 @@ public class TopicCreationConstant {
* 单次自动化审批, 最多允许的通过单子
*/
public static final Integer MAX_PASSED_ORDER_NUM_PER_TASK = 200;
private TopicCreationConstant() {
}
}

View File

@@ -16,4 +16,7 @@ public class TopicSampleConstant {
public static final Integer MAX_TIMEOUT_UNIT_MS = 10000;
public static final Integer POLL_TIME_OUT_UNIT_MS = 2000;
public static final Integer MAX_DATA_LENGTH_UNIT_BYTE = 2048;
private TopicSampleConstant() {
}
}

View File

@@ -0,0 +1,24 @@
package com.xiaojukeji.kafka.manager.common.constant;
public class TrickLoginConstant {
/**
* HTTP Header key
*/
public static final String TRICK_LOGIN_SWITCH = "Trick-Login-Switch";
public static final String TRICK_LOGIN_USER = "Trick-Login-User";
/**
* 配置允许 trick 登录用户名单
*/
public static final String TRICK_LOGIN_LEGAL_USER_CONFIG_KEY = "SECURITY.TRICK_USERS";
/**
* 开关状态值
*/
public static final String TRICK_LOGIN_SWITCH_ON = "on";
public static final String TRICK_LOGIN_SWITCH_OFF = "off";
private TrickLoginConstant() {
}
}

View File

@@ -25,6 +25,9 @@ public enum ResultStatus {
CHANGE_ZOOKEEPER_FORBIDDEN(1405, "change zookeeper forbidden"),
APP_OFFLINE_FORBIDDEN(1406, "先下线topic才能下线应用"),
TOPIC_OPERATION_PARAM_NULL_POINTER(1450, "参数错误"),
TOPIC_OPERATION_PARTITION_NUM_ILLEGAL(1451, "分区数错误"),
TOPIC_OPERATION_BROKER_NUM_NOT_ENOUGH(1452, "Broker数不足错误"),

View File

@@ -1,5 +1,7 @@
package com.xiaojukeji.kafka.manager.common.entity.ao.gateway;
import com.xiaojukeji.kafka.manager.common.entity.dto.gateway.TopicQuotaDTO;
/**
* @author zhongyuankai
* @date 2020/4/27
@@ -65,4 +67,15 @@ public class TopicQuota {
", consumeQuota=" + consumeQuota +
'}';
}
public static TopicQuota buildFrom(TopicQuotaDTO dto) {
TopicQuota topicQuota = new TopicQuota();
topicQuota.setAppId(dto.getAppId());
topicQuota.setClusterId(dto.getClusterId());
topicQuota.setTopicName(dto.getTopicName());
topicQuota.setProduceQuota(dto.getProduceQuota());
topicQuota.setConsumeQuota(dto.getConsumeQuota());
return topicQuota;
}
}

View File

@@ -25,6 +25,8 @@ public class MineTopicSummary {
private Integer access;
private String description;
public Long getLogicalClusterId() {
return logicalClusterId;
}
@@ -105,6 +107,14 @@ public class MineTopicSummary {
this.access = access;
}
public String getDescription() {
return description;
}
public void setDescription(String description) {
this.description = description;
}
@Override
public String toString() {
return "MineTopicSummary{" +

View File

@@ -37,6 +37,8 @@ public class TopicBasicDTO {
private Long retentionTime;
private Long retentionBytes;
public Long getClusterId() {
return clusterId;
}
@@ -157,6 +159,14 @@ public class TopicBasicDTO {
this.retentionTime = retentionTime;
}
public Long getRetentionBytes() {
return retentionBytes;
}
public void setRetentionBytes(Long retentionBytes) {
this.retentionBytes = retentionBytes;
}
@Override
public String toString() {
return "TopicBasicDTO{" +
@@ -166,7 +176,7 @@ public class TopicBasicDTO {
", principals='" + principals + '\'' +
", topicName='" + topicName + '\'' +
", description='" + description + '\'' +
", regionNameList='" + regionNameList + '\'' +
", regionNameList=" + regionNameList +
", score=" + score +
", topicCodeC='" + topicCodeC + '\'' +
", partitionNum=" + partitionNum +
@@ -175,6 +185,7 @@ public class TopicBasicDTO {
", modifyTime=" + modifyTime +
", createTime=" + createTime +
", retentionTime=" + retentionTime +
", retentionBytes=" + retentionBytes +
'}';
}
}

View File

@@ -0,0 +1,47 @@
package com.xiaojukeji.kafka.manager.common.entity.dto.gateway;
import com.xiaojukeji.kafka.manager.common.entity.dto.ClusterTopicDTO;
import com.xiaojukeji.kafka.manager.common.utils.ValidateUtils;
import io.swagger.annotations.ApiModel;
import io.swagger.annotations.ApiModelProperty;
@ApiModel(description = "配额调整")
public class TopicQuotaDTO extends ClusterTopicDTO {
@ApiModelProperty(value = "appId")
private String appId;
@ApiModelProperty(value = "发送数据速率B/s")
private Long produceQuota;
@ApiModelProperty(value = "消费数据速率B/s")
private Long consumeQuota;
public String getAppId() {
return appId;
}
public void setAppId(String appId) {
this.appId = appId;
}
public Long getProduceQuota() {
return produceQuota;
}
public void setProduceQuota(Long produceQuota) {
this.produceQuota = produceQuota;
}
public Long getConsumeQuota() {
return consumeQuota;
}
public void setConsumeQuota(Long consumeQuota) {
this.consumeQuota = consumeQuota;
}
@Override
public boolean paramLegal() {
return !ValidateUtils.isNullOrLessThanZero(clusterId) && !ValidateUtils.isBlank(topicName) && !ValidateUtils.isBlank(appId);
}
}

View File

@@ -40,6 +40,9 @@ public class TopicCreationDTO extends ClusterTopicDTO {
@ApiModelProperty(value = "Topic属性列表")
private Properties properties;
@ApiModelProperty(value = "最大写入字节数")
private Long peakBytesIn;
public String getAppId() {
return appId;
}
@@ -104,6 +107,14 @@ public class TopicCreationDTO extends ClusterTopicDTO {
this.properties = properties;
}
public Long getPeakBytesIn() {
return peakBytesIn;
}
public void setPeakBytesIn(Long peakBytesIn) {
this.peakBytesIn = peakBytesIn;
}
@Override
public String toString() {
return "TopicCreationDTO{" +

View File

@@ -81,11 +81,6 @@ public class OperateRecordDTO {
}
public boolean legal() {
if (!ModuleEnum.validate(moduleId) ||
(!ValidateUtils.isNull(operateId) && OperateEnum.validate(operateId))
) {
return false;
}
return true;
return !ValidateUtils.isNull(moduleId) && ModuleEnum.validate(moduleId) && OperateEnum.validate(operateId);
}
}

View File

@@ -1,6 +1,7 @@
package com.xiaojukeji.kafka.manager.common.entity.pojo;
import com.xiaojukeji.kafka.manager.common.entity.dto.op.topic.TopicCreationDTO;
import com.xiaojukeji.kafka.manager.common.utils.ValidateUtils;
import java.util.Date;
@@ -95,6 +96,7 @@ public class TopicDO {
topicDO.setClusterId(dto.getClusterId());
topicDO.setTopicName(dto.getTopicName());
topicDO.setDescription(dto.getDescription());
topicDO.setPeakBytesIn(ValidateUtils.isNull(dto.getPeakBytesIn()) ? -1L : dto.getPeakBytesIn());
return topicDO;
}
}

View File

@@ -33,7 +33,7 @@ public class BrokerOverviewVO {
@ApiModelProperty(value = "分区数")
private Integer partitionCount;
@ApiModelProperty(value = "已同步副本")
@ApiModelProperty(value = "失效副本分区的个")
private Integer underReplicatedPartitions;
@ApiModelProperty(value = "未同步")

View File

@@ -27,8 +27,11 @@ public class OrderVO {
@ApiModelProperty(value = "工单状态, 0:待审批, 1:通过, 2:拒绝, 3:取消")
private Integer status;
@ApiModelProperty(value = "申请/审核时间")
private Date gmtTime;
@ApiModelProperty(value = "申请时间")
private Date gmtCreate;
@ApiModelProperty(value = "审核时间")
private Date gmtHandle;
public Long getId() {
return id;
@@ -70,12 +73,20 @@ public class OrderVO {
this.status = status;
}
public Date getGmtTime() {
return gmtTime;
public Date getGmtCreate() {
return gmtCreate;
}
public void setGmtTime(Date gmtTime) {
this.gmtTime = gmtTime;
public void setGmtCreate(Date gmtCreate) {
this.gmtCreate = gmtCreate;
}
public Date getGmtHandle() {
return gmtHandle;
}
public void setGmtHandle(Date gmtHandle) {
this.gmtHandle = gmtHandle;
}
public String getApplicant() {
@@ -95,7 +106,7 @@ public class OrderVO {
", applicant='" + applicant + '\'' +
", description='" + description + '\'' +
", status=" + status +
", gmtTime=" + gmtTime +
", gmtTime=" + gmtCreate +
'}';
}
}

View File

@@ -33,6 +33,9 @@ public class TopicBasicVO {
@ApiModelProperty(value = "存储时间(ms)")
private Long retentionTime;
@ApiModelProperty(value = "单分区数据保存大小(Byte)")
private Long retentionBytes;
@ApiModelProperty(value = "创建时间")
private Long createTime;
@@ -62,12 +65,20 @@ public class TopicBasicVO {
this.clusterId = clusterId;
}
public String getTopicCodeC() {
return topicCodeC;
public String getAppId() {
return appId;
}
public void setTopicCodeC(String topicCodeC) {
this.topicCodeC = topicCodeC;
public void setAppId(String appId) {
this.appId = appId;
}
public String getAppName() {
return appName;
}
public void setAppName(String appName) {
this.appName = appName;
}
public Integer getPartitionNum() {
@@ -86,22 +97,6 @@ public class TopicBasicVO {
this.replicaNum = replicaNum;
}
public Long getModifyTime() {
return modifyTime;
}
public void setModifyTime(Long modifyTime) {
this.modifyTime = modifyTime;
}
public Long getCreateTime() {
return createTime;
}
public void setCreateTime(Long createTime) {
this.createTime = createTime;
}
public String getPrincipals() {
return principals;
}
@@ -110,30 +105,6 @@ public class TopicBasicVO {
this.principals = principals;
}
public String getDescription() {
return description;
}
public void setDescription(String description) {
this.description = description;
}
public void setAppId(String appId) {
this.appId = appId;
}
public void setBootstrapServers(String bootstrapServers) {
this.bootstrapServers = bootstrapServers;
}
public String getAppId() {
return appId;
}
public String getBootstrapServers() {
return bootstrapServers;
}
public Long getRetentionTime() {
return retentionTime;
}
@@ -142,12 +113,28 @@ public class TopicBasicVO {
this.retentionTime = retentionTime;
}
public String getAppName() {
return appName;
public Long getRetentionBytes() {
return retentionBytes;
}
public void setAppName(String appName) {
this.appName = appName;
public void setRetentionBytes(Long retentionBytes) {
this.retentionBytes = retentionBytes;
}
public Long getCreateTime() {
return createTime;
}
public void setCreateTime(Long createTime) {
this.createTime = createTime;
}
public Long getModifyTime() {
return modifyTime;
}
public void setModifyTime(Long modifyTime) {
this.modifyTime = modifyTime;
}
public Integer getScore() {
@@ -158,6 +145,30 @@ public class TopicBasicVO {
this.score = score;
}
public String getTopicCodeC() {
return topicCodeC;
}
public void setTopicCodeC(String topicCodeC) {
this.topicCodeC = topicCodeC;
}
public String getDescription() {
return description;
}
public void setDescription(String description) {
this.description = description;
}
public String getBootstrapServers() {
return bootstrapServers;
}
public void setBootstrapServers(String bootstrapServers) {
this.bootstrapServers = bootstrapServers;
}
public List<String> getRegionNameList() {
return regionNameList;
}
@@ -176,6 +187,7 @@ public class TopicBasicVO {
", replicaNum=" + replicaNum +
", principals='" + principals + '\'' +
", retentionTime=" + retentionTime +
", retentionBytes=" + retentionBytes +
", createTime=" + createTime +
", modifyTime=" + modifyTime +
", score=" + score +

View File

@@ -36,6 +36,9 @@ public class TopicMineVO {
@ApiModelProperty(value = "状态, 0:无权限, 1:可消费 2:可发送 3:可消费发送 4:可管理")
private Integer access;
@ApiModelProperty(value = "备注")
private String description;
public Long getClusterId() {
return clusterId;
}
@@ -108,6 +111,14 @@ public class TopicMineVO {
this.access = access;
}
public String getDescription() {
return description;
}
public void setDescription(String description) {
this.description = description;
}
@Override
public String toString() {
return "TopicMineVO{" +

View File

@@ -1,4 +1,4 @@
package com.xiaojukeji.kafka.manager.openapi.common.vo;
package com.xiaojukeji.kafka.manager.common.entity.vo.normal.topic;
import io.swagger.annotations.ApiModel;
import io.swagger.annotations.ApiModelProperty;
@@ -14,7 +14,6 @@ public class TopicStatisticMetricsVO {
public TopicStatisticMetricsVO(Double peakBytesIn) {
this.peakBytesIn = peakBytesIn;
}
public Double getPeakBytesIn() {

View File

@@ -0,0 +1,20 @@
package com.xiaojukeji.kafka.manager.common.utils;
public class BackoffUtils {
private BackoffUtils() {
}
public static void backoff(long timeUnitMs) {
if (timeUnitMs <= 0) {
return;
}
try {
Thread.sleep(timeUnitMs);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} catch (Exception e) {
// ignore
}
}
}

View File

@@ -2,6 +2,7 @@ package com.xiaojukeji.kafka.manager.common.utils;
import com.xiaojukeji.kafka.manager.common.constant.Constant;
import com.xiaojukeji.kafka.manager.common.constant.LoginConstant;
import com.xiaojukeji.kafka.manager.common.constant.TrickLoginConstant;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.BeansException;
@@ -53,13 +54,6 @@ public class SpringTool implements ApplicationContextAware, DisposableBean {
return getApplicationContext().getBeansOfType(type);
}
// /**
// * 从静态变量applicationContext中去的Bean自动转型为所复制对象的类型
// */
// public static <T> T getBean(Class<T> requiredType) {
// return (T) applicationContext.getBean(requiredType);
// }
/**
* 清除SpringContextHolder中的ApplicationContext为Null
*/
@@ -87,10 +81,18 @@ public class SpringTool implements ApplicationContextAware, DisposableBean {
}
public static String getUserName(){
HttpServletRequest request =
((ServletRequestAttributes) RequestContextHolder.getRequestAttributes()).getRequest();
HttpServletRequest request = ((ServletRequestAttributes) RequestContextHolder.getRequestAttributes()).getRequest();
String username = null;
if (TrickLoginConstant.TRICK_LOGIN_SWITCH_ON.equals(request.getHeader(TrickLoginConstant.TRICK_LOGIN_SWITCH))) {
// trick登录方式的获取用户
username = request.getHeader(TrickLoginConstant.TRICK_LOGIN_USER);
} else {
// 走页面登录方式登录的获取用户
HttpSession session = request.getSession();
String username = (String) session.getAttribute(LoginConstant.SESSION_USERNAME_KEY);
username = (String) session.getAttribute(LoginConstant.SESSION_USERNAME_KEY);
}
if (ValidateUtils.isNull(username)) {
return Constant.DEFAULT_USER_NAME;
}

View File

@@ -1,7 +1,7 @@
package com.xiaojukeji.kafka.manager.common.utils.factory;
import com.alibaba.fastjson.JSONObject;
import com.xiaojukeji.kafka.manager.common.entity.pojo.ClusterDO;
import com.xiaojukeji.kafka.manager.common.utils.JsonUtils;
import com.xiaojukeji.kafka.manager.common.utils.ValidateUtils;
import org.apache.commons.pool2.BasePooledObjectFactory;
import org.apache.commons.pool2.PooledObject;
@@ -16,7 +16,7 @@ import java.util.Properties;
* @author zengqiao
* @date 20/8/24
*/
public class KafkaConsumerFactory extends BasePooledObjectFactory<KafkaConsumer> {
public class KafkaConsumerFactory extends BasePooledObjectFactory<KafkaConsumer<String, String>> {
private ClusterDO clusterDO;
public KafkaConsumerFactory(ClusterDO clusterDO) {
@@ -25,17 +25,17 @@ public class KafkaConsumerFactory extends BasePooledObjectFactory<KafkaConsumer>
@Override
public KafkaConsumer create() {
return new KafkaConsumer(createKafkaConsumerProperties(clusterDO));
return new KafkaConsumer<String, String>(createKafkaConsumerProperties(clusterDO));
}
@Override
public PooledObject<KafkaConsumer> wrap(KafkaConsumer obj) {
return new DefaultPooledObject<KafkaConsumer>(obj);
public PooledObject<KafkaConsumer<String, String>> wrap(KafkaConsumer<String, String> obj) {
return new DefaultPooledObject<>(obj);
}
@Override
public void destroyObject(final PooledObject<KafkaConsumer> p) throws Exception {
KafkaConsumer kafkaConsumer = p.getObject();
public void destroyObject(final PooledObject<KafkaConsumer<String, String>> p) throws Exception {
KafkaConsumer<String, String> kafkaConsumer = p.getObject();
if (ValidateUtils.isNull(kafkaConsumer)) {
return;
}
@@ -57,7 +57,7 @@ public class KafkaConsumerFactory extends BasePooledObjectFactory<KafkaConsumer>
if (ValidateUtils.isBlank(clusterDO.getSecurityProperties())) {
return properties;
}
properties.putAll(JSONObject.parseObject(clusterDO.getSecurityProperties(), Properties.class));
properties.putAll(JsonUtils.stringToObj(clusterDO.getSecurityProperties(), Properties.class));
return properties;
}
}

View File

@@ -1,5 +1,6 @@
package com.xiaojukeji.kafka.manager.common.utils.jmx;
import com.xiaojukeji.kafka.manager.common.utils.BackoffUtils;
import com.xiaojukeji.kafka.manager.common.utils.ValidateUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -79,7 +80,8 @@ public class JmxConnectorWrap {
try {
Map<String, Object> environment = new HashMap<String, Object>();
if (!ValidateUtils.isBlank(this.jmxConfig.getUsername()) && !ValidateUtils.isBlank(this.jmxConfig.getPassword())) {
environment.put(JMXConnector.CREDENTIALS, Arrays.asList(this.jmxConfig.getUsername(), this.jmxConfig.getPassword()));
// fixed by riyuetianmu
environment.put(JMXConnector.CREDENTIALS, new String[]{this.jmxConfig.getUsername(), this.jmxConfig.getPassword()});
}
if (jmxConfig.isOpenSSL() != null && this.jmxConfig.isOpenSSL()) {
environment.put(Context.SECURITY_PROTOCOL, "ssl");
@@ -145,18 +147,16 @@ public class JmxConnectorWrap {
long now = System.currentTimeMillis();
while (true) {
try {
if (System.currentTimeMillis() - now > 60000) {
break;
}
int num = atomicInteger.get();
if (num <= 0) {
Thread.sleep(2);
continue;
BackoffUtils.backoff(2);
}
if (atomicInteger.compareAndSet(num, num - 1)) {
if (atomicInteger.compareAndSet(num, num - 1) || System.currentTimeMillis() - now > 6000) {
break;
}
} catch (Exception e) {
// ignore
}
}
}

View File

@@ -119,4 +119,7 @@ public class ZkPathUtil {
public static String getControllerCandidatePath(Integer brokerId) {
return D_CONTROLLER_CANDIDATES + ZOOKEEPER_SEPARATOR + brokerId;
}
private ZkPathUtil() {
}
}

View File

@@ -1,8 +1,5 @@
package com.xiaojukeji.kafka.manager.common.zookeeper.znode.brokers;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.util.List;
/**
@@ -18,12 +15,11 @@ import java.util.List;
* "host":null,
* "timestamp":"1546632983233",
* "port":-1,
* "version":4
* "version":4,
* "rack": "CY"
* }
*/
public class BrokerMetadata implements Cloneable {
private final static Logger LOGGER = LoggerFactory.getLogger(TopicMetadata.class);
private long clusterId;
private int brokerId;
@@ -43,6 +39,8 @@ public class BrokerMetadata implements Cloneable {
private long timestamp;
private String rack;
public long getClusterId() {
return clusterId;
}
@@ -107,14 +105,12 @@ public class BrokerMetadata implements Cloneable {
this.timestamp = timestamp;
}
@Override
public Object clone() {
try {
return super.clone();
} catch (CloneNotSupportedException var3) {
LOGGER.error("clone BrokerMetadata failed.", var3);
public String getRack() {
return rack;
}
return null;
public void setRack(String rack) {
this.rack = rack;
}
@Override
@@ -128,6 +124,7 @@ public class BrokerMetadata implements Cloneable {
", jmxPort=" + jmx_port +
", version='" + version + '\'' +
", timestamp=" + timestamp +
", rack='" + rack + '\'' +
'}';
}
}

View File

@@ -1,18 +1,18 @@
{
"name": "logi-kafka",
"version": "2.3.1",
"version": "2.5.0",
"description": "",
"scripts": {
"start": "webpack-dev-server",
"daily-build": "cross-env NODE_ENV=production webpack",
"pre-build": "cross-env NODE_ENV=production webpack",
"prod-build": "cross-env NODE_ENV=production webpack"
"prod-build": "cross-env NODE_ENV=production webpack",
"fix-memory": "cross-env LIMIT=4096 increase-memory-limit"
},
"author": "",
"license": "ISC",
"devDependencies": {
"@hot-loader/react-dom": "^16.8.6",
"@types/clipboard": "^2.0.1",
"@types/echarts": "^4.4.1",
"@types/lodash.debounce": "^4.0.6",
"@types/react": "^16.8.8",
@@ -21,12 +21,13 @@
"@types/spark-md5": "^3.0.2",
"antd": "^3.26.15",
"clean-webpack-plugin": "^3.0.0",
"clipboard": "2.0.6",
"clipboard": "^2.0.8",
"cross-env": "^7.0.2",
"css-loader": "^2.1.0",
"echarts": "^4.5.0",
"file-loader": "^5.0.2",
"html-webpack-plugin": "^3.2.0",
"increase-memory-limit": "^1.0.7",
"less": "^3.9.0",
"less-loader": "^4.1.0",
"mini-css-extract-plugin": "^0.6.0",

Binary file not shown.

After

Width:  |  Height:  |  Size: 36 KiB

View File

Before

Width:  |  Height:  |  Size: 125 KiB

After

Width:  |  Height:  |  Size: 125 KiB

View File

@@ -60,6 +60,22 @@ export class ChartWithDatePicker extends React.Component<IChartProps> {
public changeChartOptions(options: any) {
const noData = options.series.length ? false : true;
this.setState({ noData });
options.tooltip.formatter = (params: any) => {
var res =
"<div style='margin-bottom:5px;padding:0 12px;width:100%;height:24px;line-height:24px;border-radius:3px;'><p>" +
params[0].data.time +
" </p></div>";
for (var i = 0; i < params.length; i++) {
res += `<div key=${params[i].seriesName} style="color: #fff;padding:0 12px;line-height: 24px">
<span style="display:inline-block;margin-right:5px;border-radius:50%;width:10px;height:10px;background-color:${[
params[i].color,
]};"></span>
${params[i].seriesName}
${params[i].data[params[i].seriesName]}
</div>`;
}
return res;
}
this.chart.setOption(options, true);
}

View File

@@ -68,8 +68,8 @@ export class StatusGraghCom<T extends IFlowInfo> extends React.Component {
public render() {
const statusData = this.getData();
const loading = this.getLoading();
if (!statusData) return null;
const data: any[] = [];
if (!statusData) return <Table columns={flowColumns} dataSource={data} />;
Object.keys(statusData).map((key) => {
if (statusData[key]) {
const v = key === 'byteIn' || key === 'byteOut' ? statusData[key].map(i => i && (i / 1024).toFixed(2)) :

View File

@@ -1,4 +1,4 @@
.ant-input-number {
.ant-input-number, .ant-form-item-children .ant-select {
width: 314px
}

View File

@@ -59,6 +59,10 @@ export const adminMenu = [{
href: `/admin/bill`,
i: 'k-icon-renwuliebiao',
title: '用户账单',
},{
href: `/admin/operation-record`,
i: 'k-icon-operationrecord',
title: '操作记录',
}] as ILeftMenu[];
export const expertMenu = [{

View File

@@ -172,7 +172,7 @@ export class ClusterTopic extends SearchAndFilterContainer {
key: 'appName',
// width: '10%',
render: (val: string, record: IClusterTopics) => (
<Tooltip placement="bottomLeft" title={record.appId} >
<Tooltip placement="bottomLeft" title={val} >
{val}
</Tooltip>
),

View File

@@ -314,8 +314,7 @@ export class ExclusiveCluster extends SearchAndFilterContainer {
>
<div className="region-prompt">
<span>
Region已被逻辑集群 {this.state.logicalClusterName} 使
Region与逻辑集群的关系
Region已被逻辑集群 {this.state.logicalClusterName} 使Region与逻辑集群的关系
</span>
</div>
</Modal>

View File

@@ -16,7 +16,7 @@
.traffic-table {
margin: 10px 0;
min-height: 450px;
min-height: 330px;
.traffic-header {
width: 100%;
height: 44px;

View File

@@ -4,6 +4,7 @@ import { wrapper } from 'store';
import { observer } from 'mobx-react';
import { IXFormWrapper, IMetaData, IRegister } from 'types/base-type';
import { admin } from 'store/admin';
import { users } from 'store/users';
import { registerCluster, createCluster, pauseMonitoring } from 'lib/api';
import { SearchAndFilterContainer } from 'container/search-filter';
import { cluster } from 'store/cluster';
@@ -78,34 +79,34 @@ export class ClusterList extends SearchAndFilterContainer {
disabled: item ? true : false,
},
},
{
key: 'idc',
label: '数据中心',
defaultValue: region.regionName,
rules: [{ required: true, message: '请输入数据中心' }],
attrs: {
placeholder: '请输入数据中心',
disabled: true,
},
},
{
key: 'mode',
label: '集群类型',
type: 'select',
options: cluster.clusterModes.map(ele => {
return {
label: ele.message,
value: ele.code,
};
}),
rules: [{
required: true,
message: '请选择集群类型',
}],
attrs: {
placeholder: '请选择集群类型',
},
},
// {
// key: 'idc',
// label: '数据中心',
// defaultValue: region.regionName,
// rules: [{ required: true, message: '请输入数据中心' }],
// attrs: {
// placeholder: '请输入数据中心',
// disabled: true,
// },
// },
// {
// key: 'mode',
// label: '集群类型',
// type: 'select',
// options: cluster.clusterModes.map(ele => {
// return {
// label: ele.message,
// value: ele.code,
// };
// }),
// rules: [{
// required: true,
// message: '请选择集群类型',
// }],
// attrs: {
// placeholder: '请选择集群类型',
// },
// },
{
key: 'kafkaVersion',
label: 'kafka版本',
@@ -148,7 +149,7 @@ export class ClusterList extends SearchAndFilterContainer {
attrs: {
placeholder: `请输入JMX认证例如
{
"maxConn": 10, #KM对单台Broker最大连接数
"maxConn": 10, #KM对单台Broker最大jmx连接数
"username": "xxxxx", #用户名
"password": "xxxxx", #密码
"openSSL": true, #开启SSLtrue表示开启SSLfalse表示关闭
@@ -276,10 +277,13 @@ export class ClusterList extends SearchAndFilterContainer {
public getColumns = () => {
const cols = getAdminClusterColumns();
const role = users.currentUser.role;
const col = {
title: '操作',
render: (value: string, item: IMetaData) => (
<>
{
role && role === 2 ? <>
<a
onClick={this.createOrRegisterCluster.bind(this, item)}
className="action-button"
@@ -291,7 +295,7 @@ export class ClusterList extends SearchAndFilterContainer {
cancelText="取消"
okText="确认"
>
<Tooltip title="暂停监控将无法正常监控指标信息,建议开启监控">
<Tooltip placement="left" title="暂停监控将无法正常监控指标信息,建议开启监控">
<a
className="action-button"
>
@@ -302,6 +306,12 @@ export class ClusterList extends SearchAndFilterContainer {
<a onClick={this.showMonitor.bind(this, item)}>
</a>
</> : <Tooltip placement="left" title="该功能只对运维人员开放">
<a style={{ color: '#a0a0a0' }} className="action-button"></a>
<a className="action-button" style={{ color: '#a0a0a0' }}>{item.status === 1 ? '暂停监控' : '开始监控'}</a>
<a style={{ color: '#a0a0a0' }}></a>
</Tooltip>
}
</>
),
};
@@ -310,6 +320,7 @@ export class ClusterList extends SearchAndFilterContainer {
}
public renderClusterList() {
const role = users.currentUser.role;
return (
<>
<div className="container">
@@ -318,7 +329,14 @@ export class ClusterList extends SearchAndFilterContainer {
{this.renderSearch('', '请输入集群名称')}
<li className="right-btn-1">
<a style={{ display: 'inline-block', marginRight: '20px' }} href={indexUrl.cagUrl} target="_blank"></a>
{
role && role === 2 ?
<Button type="primary" onClick={this.createOrRegisterCluster.bind(this, null)}></Button>
:
<Tooltip placement="left" title="该功能只对运维人员开放" trigger='hover'>
<Button disabled type="primary"></Button>
</Tooltip>
}
</li>
</ul>
</div>

View File

@@ -28,6 +28,7 @@ export const getUserColumns = () => {
<span className="table-operation">
<a onClick={() => showApplyModal(record)}></a>
<a onClick={() => showApplyModalModifyPassword(record)}></a>
{record.username == users.currentUser.username ? "" :
<Popconfirm
title="确定删除?"
onConfirm={() => users.deleteUser(record.username)}
@@ -36,6 +37,7 @@ export const getUserColumns = () => {
>
<a></a>
</Popconfirm>
}
</span>);
},
},

View File

@@ -11,3 +11,5 @@ export * from './operation-management/migration-detail';
export * from './configure-management';
export * from './individual-bill';
export * from './bill-detail';
export * from './operation-record';

View File

@@ -0,0 +1,134 @@
import * as React from 'react';
import { cellStyle } from 'constants/table';
import { Tooltip } from 'antd';
import { admin } from 'store/admin';
import moment = require('moment');
const moduleList = [
{ moduleId: 0, moduleName: 'Topic' },
{ moduleId: 1, moduleName: '应用' },
{ moduleId: 2, moduleName: '配额' },
{ moduleId: 3, moduleName: '权限' },
{ moduleId: 4, moduleName: '集群' },
{ moduleId: 5, moduleName: '分区' },
{ moduleId: 6, moduleName: 'Gateway配置' },
]
export const operateList = {
0: '新增',
1: '删除',
2: '修改'
}
// [
// { operate: '新增', operateId: 0 },
// { operate: '删除', operateId: 1 },
// { operate: '修改', operateId: 2 },
// ]
export const getJarFuncForm: any = (props: any) => {
const formMap = [
{
key: 'moduleId',
label: '模块',
type: 'select',
attrs: {
style: {
width: '130px'
},
placeholder: '请选择模块',
},
options: moduleList.map(item => {
return {
label: item.moduleName,
value: item.moduleId
}
}),
formAttrs: {
initialvalue: 0,
},
},
{
key: 'operator',
label: '操作人',
type: 'input',
attrs: {
style: {
width: '170px'
},
placeholder: '请输入操作人'
},
getvaluefromevent: (event: any) => {
return event.target.value.replace(/\s+/g, '')
},
},
// {
// key: 'resource',
// label: '资源名称',
// type: 'input',
// attrs: {
// style: {
// width: '170px'
// },
// placeholder: '请输入资源名称'
// },
// },
// {
// key: 'content',
// label: '操作内容',
// type: 'input',
// attrs: {
// style: {
// width: '170px'
// },
// placeholder: '请输入操作内容'
// },
// },
]
return formMap;
}
export const getOperateColumns = () => {
const columns: any = [
{
title: '模块',
dataIndex: 'module',
key: 'module',
align: 'center',
width: '12%'
},
{
title: '资源名称',
dataIndex: 'resource',
key: 'resource',
align: 'center',
width: '12%'
},
{
title: '操作内容',
dataIndex: 'content',
key: 'content',
align: 'center',
width: '25%',
onCell: () => ({
style: {
maxWidth: 350,
...cellStyle,
},
}),
render: (text: string, record: any) => {
return (
<Tooltip placement="topLeft" title={text} >{text}</Tooltip>);
},
},
{
title: '操作人',
dataIndex: 'operator',
align: 'center',
width: '12%'
},
];
return columns
}

View File

@@ -0,0 +1,130 @@
import * as React from 'react';
import { observer } from 'mobx-react';
import { SearchAndFilterContainer } from 'container/search-filter';
import { IXFormWrapper, IMetaData, IRegister } from 'types/base-type';
import { admin } from 'store/admin';
import { customPagination, cellStyle } from 'constants/table';
import { Table, Tooltip } from 'component/antd';
import { timeFormat } from 'constants/strategy';
import { SearchFormComponent } from '../searchForm';
import { getJarFuncForm, operateList, getOperateColumns } from './config'
import moment = require('moment');
import { tableFilter } from 'lib/utils';
@observer
export class OperationRecord extends SearchAndFilterContainer {
public state: any = {
searchKey: '',
filteredInfo: null,
sortedInfo: null,
};
public getData<T extends IMetaData>(origin: T[]) {
let data: T[] = origin;
let { searchKey } = this.state;
searchKey = (searchKey + '').trim().toLowerCase();
data = searchKey ? origin.filter((item: IMetaData) =>
(item.clusterName !== undefined && item.clusterName !== null) && item.clusterName.toLowerCase().includes(searchKey as string),
) : origin;
return data;
};
public searchForm = (params: any) => {
// this.props.setFuncSubValue(params)
// getSystemFuncList(params).then(res => {
// this.props.setSysFuncList(res.data)
// this.props.setPagination(res.pagination)
// })
const { operator, moduleId } = params || {}
operator ? admin.getOperationRecordData(params) : admin.getOperationRecordData({ moduleId })
// getJarList(params).then(res => {
// this.props.setJarList(res.data)
// this.props.setPagination(res.pagination)
// })
}
public clearAll = () => {
this.setState({
filteredInfo: null,
sortedInfo: null,
});
};
public setHandleChange = (pagination: any, filters: any, sorter: any) => {
this.setState({
filteredInfo: filters,
sortedInfo: sorter,
});
}
public renderOperationRecordList() {
let { sortedInfo, filteredInfo } = this.state;
sortedInfo = sortedInfo || {};
filteredInfo = filteredInfo || {};
const operatingTime = Object.assign({
title: '操作时间',
dataIndex: 'modifyTime',
key: 'modifyTime',
align: 'center',
sorter: (a: any, b: any) => a.modifyTime - b.modifyTime,
render: (t: number) => moment(t).format(timeFormat),
width: '15%',
sortOrder: sortedInfo.columnKey === 'modifyTime' && sortedInfo.order,
});
const operatingPractice = Object.assign({
title: '行为',
dataIndex: 'operate',
key: 'operate',
align: 'center',
width: '12%',
filters: tableFilter<any>(this.getData(admin.oRList), 'operateId', operateList),
// filteredValue: filteredInfo.operate || null,
onFilter: (value: any, record: any) => {
return record.operateId === value
}
}, this.renderColumnsFilter('modifyTime'))
const columns = getOperateColumns()
columns.splice(0, 0, operatingTime);
columns.splice(3, 0, operatingPractice);
return (
<>
<div className="container">
<div className="table-operation-panel">
<SearchFormComponent
formMap={getJarFuncForm()}
onSubmit={(params: any) => this.searchForm(params)}
clearAll={() => this.clearAll()}
isReset={true}
/>
</div>
<div className="table-wrapper">
<Table
rowKey="key"
loading={admin.loading}
dataSource={this.getData(admin.oRList)}
columns={columns}
pagination={customPagination}
bordered
onChange={this.setHandleChange}
/>
</div>
</div>
</>
)
};
componentDidMount() {
admin.getOperationRecordData({ moduleId: 0 });
}
render() {
return <div>
{
this.renderOperationRecordList()
}
</div>
}
}

View File

@@ -0,0 +1,120 @@
import * as React from 'react';
import { Select, Input, InputNumber, Form, Switch, Checkbox, DatePicker, Radio, Upload, Button, Icon, Tooltip } from 'component/antd';
// import './index.less';
const Search = Input.Search;
export interface IFormItem {
key: string;
label: string;
type: string;
value?: string;
// 内部组件属性注入
attrs?: any;
// form属性注入
formAttrs?: any;
defaultValue?: string | number | any[];
rules?: any[];
invisible?: boolean;
getvaluefromevent: Function;
}
interface SerachFormProps {
formMap: IFormItem[];
// formData: any;
form: any;
onSubmit: Function;
isReset?: boolean;
clearAll: Function;
layout?: 'inline' | 'horizontal' | 'vertical';
}
export interface IFormSelect extends IFormItem {
options: Array<{ key?: string | number, value: string | number, label: string }>;
}
class SearchForm extends React.Component<SerachFormProps>{
public onSubmit = () => {
// this.props.onSubmit()
//
}
public renderFormItem(item: IFormItem) {
switch (item.type) {
default:
case 'input':
return <Input key={item.key} {...item.attrs} />;
case 'select':
return (
<Select
// size="small"
key={item.key}
{...item.attrs}
invisibleValue={item.formAttrs.invisibleValue}
>
{(item as IFormSelect).options && (item as IFormSelect).options.map((v, index) => (
<Select.Option
key={v.value || v.key || index}
value={v.value}
>
{v.label}
{/* <Tooltip placement='left' title={v.value}>
{v.label}
</Tooltip> */}
</Select.Option>
))}
</Select>
);
}
}
public theQueryClick = (value: any) => {
this.props.onSubmit(value)
this.props.clearAll()
// this.props.form.resetFields()
}
public resetClick = () => {
this.props.form.resetFields()
this.props.clearAll()
this.theQueryClick(this.props.form.getFieldsValue())
}
public render() {
const { form, formMap, isReset } = this.props;
const { getFieldDecorator, getFieldsValue } = form;
return (
<Form layout='inline' onSubmit={this.onSubmit}>
{
formMap.map(formItem => {
// const { initialValue, valuePropName } = this.handleFormItem(formItem, formData);
// const getFieldValue = {
// initialValue,
// rules: formItem.rules || [{ required: false, message: '' }],
// valuePropName,
// };
return (
<Form.Item
key={formItem.key}
label={formItem.label}
{...formItem.formAttrs}
>
{getFieldDecorator(formItem.key, {
initialValue: formItem.formAttrs?.initialvalue,
getValueFromEvent: formItem?.getvaluefromevent,
})(
this.renderFormItem(formItem),
)}
</Form.Item>
);
})
}
<Form.Item>
{
isReset && <Button style={{ width: '80px', marginRight: '20px' }} type="primary" onClick={() => this.resetClick()}></Button>
}
<Button style={{ width: '80px' }} type="primary" onClick={() => this.theQueryClick(getFieldsValue())}></Button>
</Form.Item>
</Form>
);
}
}
export const SearchFormComponent = Form.create<SerachFormProps>({ name: 'search-form' })(SearchForm);

View File

@@ -54,7 +54,7 @@ export class AlarmSelect extends React.Component<IAlarmSelectProps> {
<a
className="icon-color"
target="_blank"
href="https://github.com/didi/kafka-manager"
href="https://github.com/didi/Logi-KafkaManager/blob/master/docs/user_guide/faq.md"
>
</a>

View File

@@ -149,9 +149,9 @@ export class DynamicSetFilter extends React.Component<IDynamicProps> {
public handleSelectChange = (e: string, type: 'topic' | 'consumerGroup' | 'location') => {
switch (type) {
case 'topic':
if (!this.clusterId) {
return message.info('请选择集群');
}
// if (!this.clusterId) {
// return message.info('请选择集群');
// }
this.topicName = e;
const type = this.dealMonitorType();
if (['kafka-consumer-maxLag', 'kafka-consumer-maxDelayTime', 'kafka-consumer-lag'].indexOf(type) > -1) {

View File

@@ -52,8 +52,7 @@ export class CommonAppList extends SearchAndFilterContainer {
},
}),
render: (text: string, record: IAppItem) => {
return (
<Tooltip placement="bottomLeft" title={record.name}>{text}</Tooltip>);
return (<Tooltip placement="bottomLeft" title={record.name}>{text}</Tooltip>);
},
},
{
@@ -103,7 +102,7 @@ export class CommonAppList extends SearchAndFilterContainer {
}
public getOnlineConnect(record: IAppItem) {
modal.showOfflineAppModal(record.appId);
modal.showOfflineAppNewModal(record.appId);
}
public getData<T extends IAppItem>(origin: T[]) {

Some files were not shown because too many files have changed in this diff Show More