Compare commits

...

777 Commits

Author SHA1 Message Date
zengqiao
66e3da5d2f 产品prd 2023-02-28 10:59:12 +08:00
zengqiao
5f2adfe74e 产品PRD 2023-02-28 10:58:48 +08:00
zengqiao
cecfde906a 清空代码 2023-02-28 10:37:11 +08:00
zengqiao
258385dc9a 升级至3.3.0版本 2023-02-24 11:12:31 +08:00
zengqiao
65238231f0 补充3.3.0版本升级信息 2023-02-24 11:11:12 +08:00
zengqiao
cb22e02fbe 补充3.3.0版本变更信息 2023-02-24 11:10:42 +08:00
erge
aa0bec1206 [Optimize]package.json锁定lerna版本,更新package-lock.json文件(#957) 2023-02-23 20:14:04 +08:00
wyb
793c780015 [Bugfix]修复mm2列表请求超时(#949)
调整代码结构
2023-02-23 11:17:48 +08:00
erge
ec6f063450 [Optimize] 去除package.json 出现内部地址(#939) 2023-02-22 17:08:21 +08:00
zengqiao
f25c65b98b [Doc]补充贡献者信息 2023-02-22 14:00:52 +08:00
Luckywustone
2d99aae779 [Bugfix]ZK健康巡检日志不清晰导致问题难定位 #904
[Bugfix]ZK健康巡检日志不清晰导致问题难定位 #904
2023-02-22 13:41:02 +08:00
erge
a8847dc282 [Bugfix] 修复打包不成功(#940) 2023-02-22 11:58:33 +08:00
zengqiao
4852c01c88 [Feature]补充贡献代码相关文档(#947)
1、补充贡献者名单,如有遗漏,辛苦告知;
2、补充贡献指南;
2023-02-22 11:53:00 +08:00
zengqiao
3d6f405b69 [Bugfix]订正失效的邮箱地址(#944)
[Bugfix]订正语句(#944)
2023-02-22 11:52:40 +08:00
erge
18e3fbf41d [Optimize] 健康检查项时间和结果显示(didi#930) 2023-02-21 10:41:49 +08:00
erge
ae8cc3092b [Optimize] 新增/编辑MM2 Topic 由当前集群获取改为对应的sourceKafka集群获取& 新增/编辑MM2入参优化(#894) 2023-02-21 10:41:44 +08:00
erge
5c26e8947b [Optimize] JSON新增MM2 Drawer Title文案变更(#894) 2023-02-21 10:41:37 +08:00
erge
fbe6945d3b [Bugfix]zookeeper页面leader节点显示异常(#873) 2023-02-21 10:41:25 +08:00
zengqiao
7dc8f2dc48 [Bugfix]修复Connector列表和MM2列表搜索不生效的问题(#928) 2023-02-21 10:40:05 +08:00
zengqiao
91c60ce72c [Bugfix]修复新接入的集群,Controller-Host不显示的问题(#927)
问题原因:
1、新接入的集群,DB中暂未存储Broker信息,因此在存储Controller至DB时,查询DB中的Broker会查询为空。

解决方式:
1、存储Controller至DB前,主动获取一次Broker的信息。
2023-02-21 10:39:46 +08:00
zengqiao
687eea80c8 补充3.3.0版本变更信息 2023-02-16 14:51:43 +08:00
zengqiao
9bfe3fd1db 设置为AGPL协议 2023-02-15 17:53:46 +08:00
shizeying
03f81bc6de [Bugfix]删除idx_cluster_phy_id 索引并新增idx_cluster_update_time索引(#918) 2023-02-15 17:45:53 +08:00
slhu
eed9571ffa [Bugfix]解决在解析命令执行后返回指标的值时发生的数据类型转换错误与指标存储上报时报空指针的问题(#912)
1.zk_min_latency、zk_max_latency指标数据类型变更为float
2.使用ConvertUtil.string2Float()方法进行string到float到类型转换
2023-02-15 16:20:39 +08:00
edengyuan_v
e4651ef749 [Optimize]新增Topic时清理策略区分单选多选(#770) 2023-02-15 11:18:33 +08:00
zengqiao
f715cf7a8d 补充 3.3.0 版本变更信息 2023-02-13 11:57:51 +08:00
wyb
fad9ddb9a1 fix: 更新登录页文案 2023-02-13 11:49:00 +08:00
wyb
b6e4f50849 fix: 健康状态详情优化 & Connector 样式优化 & 无MM2任务指标兜底页 2023-02-13 11:49:00 +08:00
wyb
5c6911e398 [Optimize]Overview指标卡片展示逻辑 2023-02-13 11:49:00 +08:00
wyb
a0371ab88b feat: 新增Topic 复制功能 2023-02-13 11:49:00 +08:00
wyb
fa2abadc25 feat: 新增Mirror Maker 2.0(MM2) 2023-02-13 11:49:00 +08:00
zengqiao
f03460f3cd [Bugfix]修复 Broker Similar Config 显示错误的问题(#872) 2023-02-13 11:22:13 +08:00
zengqiao
b5683b73c2 [Optimize]优化 MySQL & ES 测试容器的初始化(#906)
主要的变更
1、knowstreaming/knowstreaming-manager 容器;
2、knowstreaming/knowstreaming-mysql 容器调整为使用 mysql:5.7 容器;
3、初始化 mysql:5.7 容器后,增加初始化 MySQL 表及数据的动作;

被影响的变更:
1、移动 km-dist/init/sql 下的MySQL初始化脚本至 km-persistence/src/main/resource/sql 下,以便项目测试时加载到所需的初始化 SQL;
2、删除无用的 km-dist/init/template 目录;
3、因为 km-dist/init/sql 和 km-dist/init/template 目录的调整,因此也调整 ReleaseKnowStreaming.xml 内的文件内容;
2023-02-13 10:33:40 +08:00
zengqiao
c062586c7e [Optimize]删除无用&多余的打包配置文件 2023-02-10 16:51:32 +08:00
fengqiongfeng
98a5c7b776 [Optimize]健康检查日志优化(#869) 2023-02-10 11:02:24 +08:00
zengqiao
e204023b1f [Feature]增加支持Topic复制的集群列表接口(#899) 2023-02-09 17:03:28 +08:00
zengqiao
4c5ffccc45 [Optimize]删除无效代码 2023-02-09 17:00:50 +08:00
zengqiao
fbcf58e19c [Feature]MM2管理-Connector元信息管理优化(#894) 2023-02-09 16:59:38 +08:00
zengqiao
e5c6d00438 [Feature]MM2管理-补充集群Group列表信息(#894) 2023-02-09 16:59:38 +08:00
zengqiao
ab6a4d7099 [Feature]MM2管理-MM2管理相关接口类(#894) 2023-02-09 16:59:38 +08:00
zengqiao
78b2b8a45e [Feature]MM2管理-MM2管理相关业务类(#894) 2023-02-09 16:59:38 +08:00
zengqiao
add2af4f3f [Feature]MM2管理-MM2管理相关服务类(#894) 2023-02-09 16:59:38 +08:00
zengqiao
235c0ed30e [Feature]MM2管理-MM2管理相关实体类(#894) 2023-02-09 16:59:38 +08:00
zengqiao
5bd93aa478 [Bugfix]修复正常情况下,集群状态统计错误的问题(#865) 2023-02-09 16:44:26 +08:00
zengqiao
f95be2c1b3 [Optimize]TaskResult增加返回任务分组信息 2023-02-09 16:36:19 +08:00
zengqiao
5110b30f62 [Feature]MM2管理-MM2健康巡检(#894) 2023-02-09 15:36:35 +08:00
zengqiao
861faa5df5 [Feature]HA-镜像Topic管理(#899)
1、底层Kafka需要是滴滴版本的Kafka;
2、新增镜像Topic的增删改查;
3、新增镜像Topic的指标查看;
2023-02-09 15:21:23 +08:00
zengqiao
efdf624c67 [Feature]HA-滴滴Kafka版本信息兼容(#899) 2023-02-09 15:21:23 +08:00
zengqiao
caccf9cef5 [Feature]MM2管理-采集MM2指标任务(#894) 2023-02-09 14:58:34 +08:00
zengqiao
6ba3dceb84 [Feature]MM2管理-采集MM2指标(#894) 2023-02-09 14:58:34 +08:00
zengqiao
9b7c41e804 [Feature]MM2管理-读写ES中的MM2指标(#894) 2023-02-09 14:58:34 +08:00
zengqiao
346aee8fe7 [Bugfix]修复Topic指标大盘获取TopN指标存在错误的问题(#896)
1、将ES排序调整为基于本地cache的排序;
2、将database的本地cache从core模块移动到persistence模块;
2023-02-09 14:20:02 +08:00
zengqiao
353d781bca [Feature]补充MM2相关索引及数据库表信息(#894) 2023-02-09 13:44:40 +08:00
EricZeng
3ce4bf231a 修复条件判断错误问题
Co-authored-by: haoqi123 <49672871+haoqi123@users.noreply.github.com>
2023-02-09 11:28:26 +08:00
EricZeng
d046cb8bf4 修复条件判断错误问题
Co-authored-by: haoqi123 <49672871+haoqi123@users.noreply.github.com>
2023-02-09 11:28:26 +08:00
zengqiao
da95c63503 [Optimize]优化TestContainers相关依赖(#892)
1、去除对mysql-connector-j的依赖;
2、整理代码;
2023-02-09 11:28:26 +08:00
haoqi
915e48de22 [Optimize]补充Testcontainers的使用说明(#890) 2023-02-09 11:05:44 +08:00
_haoqi
256f770971 [Feature]Support running tests with testcontainers(#870) 2023-02-08 14:56:44 +08:00
zengqiao
16e251cbe8 调整开源协议 2023-02-08 14:10:37 +08:00
zengqiao
67743b859a [Optimize]补充Ldap登录的配置说明(#888) 2023-02-08 13:51:45 +08:00
congchen0321
c275b42632 Update faq.md 2023-02-08 13:41:08 +08:00
zengqiao
a02760417b [Optimize]ZK-Overview页面补充默认展示的指标(#874) 2023-01-30 13:18:06 +08:00
zengqiao
0e50bfc5d4 优化PR模版 2023-01-13 16:04:25 +08:00
wuyouwuyoulian
eab988e18f For #781, Fix "The partition display is incomplete" bug 2023-01-12 11:03:30 +08:00
zengqiao
dd6004b9d4 [Bugfix]修复采集副本指标时,参数传递错误问题(#867) 2023-01-11 18:00:21 +08:00
zengqiao
ac7c32acd5 [Optimize]优化ES索引及模版的初始化文档(#832)
1、订正不同地方索引模版的shard数存在不一致的问题;
2、删除多余的template.sh,统一使用init_es_template.sh;
3、init_es_template.sh中,增加connect相关索引模版的初始化脚本,删除replica 和 zookeper索引模版的初始化脚本;
2023-01-09 15:18:41 +08:00
zengqiao
f4a219ceef [Optimize]去除Replica指标从ES读写的相关代码(#862) 2023-01-09 14:57:38 +08:00
zengqiao
a8b56fb613 [Bugfix]修复用户信息修改后,用户列表会抛出空指针异常的问题(#860) 2023-01-09 14:57:23 +08:00
zengqiao
2925a20e8e [Bugfix]修复查看消息时,选择分区不生效问题(#858) 2023-01-09 13:38:10 +08:00
zengqiao
6b3eb05735 [Bugfix]修复对ZK客户端进行配置后不生效的问题(#694)
1、修复在ks_km_physical_cluster表的zk_properties字段填写ZK 客户端的相关配置后,不生效的问题。
2、删除zk_properties字段中,暂时无需使用的jmxConfig字段。
2023-01-09 10:44:35 +08:00
zengqiao
17e0c39f83 [Optimize]优化Topic健康巡检的日志(#855) 2023-01-06 14:42:08 +08:00
zengqiao
4994639111 [Optimize]无ZK模块时,巡检详情忽略对ZK的展示(#764) 2023-01-04 10:32:18 +08:00
wyb
c187b5246f [Bugfix]修复connector指标筛选缺少指标的问题(#846) 2022-12-23 16:19:34 +08:00
wyb
6ed6d5ec8a [Bugfix]修复用户更新失败问题(#840) 2022-12-22 15:56:48 +08:00
wyb
0735b332a8 [Bugfix]修复函数映射错误(#842) 2022-12-22 08:48:59 +08:00
wyb
344cec19fe [Bugfix]connector指标采集算最大值错误(#836) 2022-12-20 09:50:42 +08:00
zengqiao
6ef365e201 bump version to 3.2.0 2022-12-16 13:58:40 +08:00
zengqiao
edfa6a9f71 调整v3.2版本容器化部署信息 2022-12-16 13:39:51 +08:00
孙超
860d0b92e2 V3.2 2022-12-16 13:27:09 +08:00
zengqiao
5bceed7105 [Optimize]缩小ES索引默认shard数 2022-12-15 14:44:18 +08:00
zengqiao
44a2fe0398 增加3.2.0版本升级信息 2022-12-14 14:14:35 +08:00
zengqiao
218459ad1b 增加3.2.0版本变更信息 2022-12-14 14:14:20 +08:00
zengqiao
7db757bc12 [Optimize]优化Connector创建时的入参
1、增加config.action.reload的默认值;
2、增加errors.tolerance的默认值;
2022-12-14 14:12:32 +08:00
zengqiao
896a943587 [Optimize]缩短ES索引默认保存时间为15天 2022-12-14 14:10:46 +08:00
zengqiao
cd2c388e68 [Optimize]优化Sonar代码扫描结果 2022-12-14 14:07:30 +08:00
wyb
4543a339b7 [Bugfix]修复job更新中的数组越界报错(#744) 2022-12-14 13:56:29 +08:00
zengqiao
1c4fbef9f2 [Feature]支持拆分API服务和Job服务部署(#829)
1、JMX检查功能是每一个KS都必须要有的,因此从Task模块移动到Core模块;
2、application.yml中补充Task模块任务的整体开关字段;
2022-12-09 16:11:03 +08:00
zengqiao
b2f0f69365 [Optimize]Overview页面的TopN查询ES流程优化(#823)
1、复用线程池,同时支持线程池的线程数可配置;
2、优化查询TopN指标时,可能会出现重复查询的问题;
3、处理代码扫描(SonarLint)反馈的问题;
2022-12-09 14:39:17 +08:00
wyb
c4fb18a73c [Bugfix]修复迁移任务状态不一致问题(#815) 2022-12-08 17:13:14 +08:00
zengqiao
5cad7b4106 [Bugfix]修复集群Topic列表页面白屏问题(#819)
集群Topic列表健康状态对应关系存在问题,导致当健康状态指标存在时,会出现白屏。
2022-12-07 16:27:27 +08:00
zengqiao
f3c4133cd2 [Bugfix]分批从ES查询Topic最近一条指标(#817) 2022-12-07 16:15:01 +08:00
zengqiao
d9c59cb3d3 增加Connect Rest接口 2022-12-07 10:20:02 +08:00
zengqiao
7a0db7161b 增加Connect 业务层方法 2022-12-07 10:20:02 +08:00
zengqiao
6aefc16fa0 增加Connect相关任务 2022-12-07 10:20:02 +08:00
zengqiao
186dcd07e0 增加3.2版本升级信息 2022-12-07 10:20:02 +08:00
zengqiao
e8652d5db5 Connect相关代码 2022-12-07 10:20:02 +08:00
zengqiao
fb5964af84 补充kafka-connect相关包 2022-12-07 10:20:02 +08:00
zengqiao
249fe7c700 调整ES相关文件位置 & 补充connectESDAO相关类 2022-12-07 10:20:02 +08:00
zengqiao
cc2a590b33 新增自定义的KSPartialKafkaAdminClient
由于原生的KafkaAdminClient在解析Group时,会将Connect集群的Group过滤掉,因此自定义KSPartialKafkaAdminClient,使其具备获取Connect Group的能力
2022-12-07 10:20:02 +08:00
zengqiao
5b3f3e5575 移动指标入ES的代码 2022-12-07 10:20:02 +08:00
wyb
36cf285397 [Bug]修复logi-securiy模块数据库选择错误(#808) 2022-12-06 20:02:49 +08:00
zengqiao
4386563c2c 调整指标采集的默认耗时值,以便在查看Top指标时即可看到 2022-12-06 16:47:53 +08:00
zengqiao
0123ce4a5a 优化Broker列表JMX端口的返回值 2022-12-06 16:47:07 +08:00
zengqiao
c3d47d3093 池化KafkaAdminClient,避免KafkaAdminClient出现性能问题 2022-12-06 16:46:11 +08:00
zengqiao
9735c4f885 删除重复采集的指标 2022-12-06 16:41:27 +08:00
zengqiao
3a3141a361 调整ZK指标的采集时间 2022-12-06 16:40:52 +08:00
zengqiao
ac30436324 [Bugfix]修复更新健康巡检结果时出现死锁的问题(#728) 2022-12-05 16:30:37 +08:00
zengqiao
7176e418f5 [Optimize]优化健康巡检相关指标的计算(#726)
1、增加缓存,减少健康状态指标计算时的IO;
2、健康巡检调整为按照资源维度并发处理;
3、明确HealthCheckResultService和HealthStateService的功能边界;
2022-12-05 16:26:31 +08:00
zengqiao
ca794f507e [Optimize]规范日志输出格式(#800)
修改log输出配置,使其输出的日志中自带class={className}的信息,后续书写代码时,就无需书写该部分内容。
2022-12-05 14:27:02 +08:00
zengqiao
0f8be4fadc [Optimize]优化日志输出 & 本地缓存统一管理(#800) 2022-12-05 14:04:19 +08:00
zengqiao
7066246e8f [Optimize]错开采集任务触发时间,降低Offset信息获取时超时情况的发生(#726)
当前指标采集任务都是整分钟触发执行的,导致会同时向Kafka请求分区Offset信息,会导致:
1、请求过多,从而出现超时;
2、同时进行,可能会导致分区重复获取Offset信息;

因此将其错开。
2022-12-05 13:49:35 +08:00
zengqiao
7d1bb48b59 [Optimize]ZK四字命令解析日志优化(#805)
增加遗漏的指标名的处理,减少warn日志该部分的信息
2022-12-05 13:39:26 +08:00
limaiwang
dd0d519677 [Optimize]更新Zookeeper详情目录结构搜索文案(#793) 2022-12-05 12:15:03 +08:00
zengqiao
4293d05fca [Optimize]优化Topic元信息更新策略(#806) 2022-12-04 17:55:27 +08:00
zengqiao
2c82baf9fc [Optimize]指标采集性能优化-part1(#726) 2022-12-04 15:41:48 +08:00
zengqiao
921161d6d0 [Bugfix]修复ReplicaMetricCollector编译失败问题(#802) 2022-12-03 14:34:38 +08:00
zengqiao
e632c6c13f [Optimize]优化Sonar扫描结果 2022-12-02 15:34:28 +08:00
zengqiao
5833a8644c [Optimize]关闭errorLogger,去除无用输出(#801) 2022-12-02 15:29:17 +08:00
zengqiao
fab41e892f [Optimize]日志统一格式&优化输出内容-part3(#800) 2022-12-02 15:14:21 +08:00
zengqiao
7a52cf67b0 [Optimize]日志统一格式&优化输出内容-part2(#800) 2022-12-02 15:01:24 +08:00
zengqiao
175b8d643a [Optimize]统一日志格式-part1(#800) 2022-12-02 14:39:57 +08:00
zengqiao
6241eb052a [Bugfix]修复KafkaJMXClient类中logger错误的问题(#794) 2022-11-30 11:15:00 +08:00
zengqiao
c2fd0a8410 [Optimize]优化Sonar扫描出的不规范代码 2022-11-29 20:54:41 +08:00
zengqiao
5127b600ec [Optimize]优化ESClient的并发访问控制(#787) 2022-11-29 10:47:57 +08:00
zengqiao
feb03aede6 [Optimize]优化线程池的名称(#789) 2022-11-28 15:11:54 +08:00
duanxiaoqiu
47b6c5d86a [Bugfix]修复创建topic选择过期策略(kafka版本0.10.1.0之前)compact和delete只能二选一(didi#770) 2022-11-27 14:18:50 +08:00
SimonTeo58
c4a81613f4 [Optimize]更新Topic-Messages抽屉文案(#771) 2022-11-24 21:54:29 +08:00
limaiwang
daeb5c4cec [Bugfix]修复集群配置不写时,校验参数报错的问题 2022-11-24 15:30:01 +08:00
WangYaobo
38def45ad6 [Doc]增加无数据排查文档(#773) 2022-11-24 10:44:37 +08:00
pen4
4b29a2fdfd update org.springframework:spring-context 5.3.18 to 5.3.19 2022-11-23 11:38:11 +08:00
zengqiao
a165ecaeef [Bugfix]修复Broker&Topic修改时,版本设置错误问题(#762)
Kafka v2.3增加了增量修改配置的功能,但是KS中错误的将其配置为0.11.0版本就具备该能力,因此对其进行调整。
2022-11-21 15:56:33 +08:00
night.liang
6637ba4ccc [Optimize] optimize zk OutstandingRequests checker’s exception log (#738) 2022-11-18 17:12:07 +08:00
duanxiaoqiu
2f807eec2b [Feat]Topic列表健康分修改为健康状态(#758) 2022-11-18 13:56:27 +08:00
石臻臻的杂货铺
636c2c6a83 Update README.md 2022-11-17 13:33:40 +08:00
zengqiao
898a55c703 [Bugfix]修复Broker列表LogSize指标存储时名称错误的问题(#759) 2022-11-17 13:27:45 +08:00
zengqiao
8ffe7e7101 [Bugfix]修复Prometheus中Group部分指标缺少的问题(#756) 2022-11-14 13:33:16 +08:00
zengqiao
7661826ea5 [Optimize]健康巡检增加ClusterParam, 从而拆分Kafka和Connect相关的巡检任务 2022-11-10 16:24:39 +08:00
zengqiao
e456be91ef [Bugfix]集群JMX配置发生变更时,进行JMX的重新加载 2022-11-10 16:04:40 +08:00
zengqiao
da0a97cabf [Optimize] 调整Task代码结构为Connector功能做准备 2022-11-09 10:28:52 +08:00
zengqiao
c1031a492a [Optimize]增加ES索引删除的功能 2022-11-09 10:28:52 +08:00
zengqiao
3c8aaf528c [Bugfix] 修复因为指标缺失导致返回的集群数错误的问题 (#741) 2022-11-09 10:28:52 +08:00
黄海婷
70ff20a2b0 styles:cardBar卡片标题图标hover样式 2022-11-07 10:38:28 +08:00
黄海婷
6918f4babe styles:job列表自定义列按钮新增hover背景色 2022-11-07 10:38:28 +08:00
黄海婷
805a704d34 styles:部分icon在hover的时候,需要有背景色 2022-11-07 10:38:28 +08:00
黄海婷
c69c289bc4 styles:部分icon在hover的时候,需要有背景色 2022-11-07 10:38:28 +08:00
zengqiao
dd5869e246 [Optimize] 调整代码结构,为Connect功能做准备 2022-11-07 10:13:26 +08:00
Richard
b51ffb81a3 [Bugfix] No thread-bound request found. (#743) 2022-11-07 10:06:54 +08:00
黄海婷
ed0efd6bd2 styles:字体颜色#adb5bc变更为#74788D 2022-11-03 16:49:35 +08:00
黄海婷
39d2fe6195 styles:消息大小测试弹窗下方提示字体加粗 2022-11-03 16:49:35 +08:00
黄海婷
7471d05c20 styles:消息大小测试弹框字符数显示字体调整 2022-11-03 16:49:35 +08:00
黄海婷
3492688733 feat:Consumer列表刷新按钮新增hover提示 2022-11-01 17:37:37 +08:00
Sean
a603783615 [Optimize].gitignore 中添加 flatten.xml 过滤,为后续引入flatten 做准备(#732) 2022-11-01 14:16:53 +08:00
night.liang
5c9096d564 [Bugfix] fix replica dsl (#708) 2022-11-01 10:45:59 +08:00
zengqiao
c27786a257 bump version to 3.1.0 2022-10-31 14:55:50 +08:00
zengqiao
81910d1958 [Hotfix] 修复新接入集群时,健康状态信息页面出现空指针问题 2022-10-31 14:55:22 +08:00
zengqiao
55d5fc4bde 增加v3.1.0版本的变更项 2022-10-31 14:05:42 +08:00
GraceWalk
f30586b150 fix: 依赖安装默认采用 taobao 镜像 2022-10-29 13:55:36 +08:00
GraceWalk
37037c19f0 fix: 更新版本信息获取方式 2022-10-29 13:55:36 +08:00
GraceWalk
1a5e2c7309 fix: 错误页面优化 2022-10-29 13:55:36 +08:00
GraceWalk
941dd4fd65 feat: 支持 Zookeeper 模块 2022-10-29 13:55:36 +08:00
GraceWalk
5f6df3681c feat: 健康状态展示优化 2022-10-29 13:55:36 +08:00
zengqiao
7d045dbf05 补充ZK健康巡检任务 2022-10-29 13:55:07 +08:00
zengqiao
4ff4accdc3 补充3.1.0版本升级信息 2022-10-29 13:55:07 +08:00
zengqiao
bbe967c4a8 补充多集群健康状态概览信息 2022-10-29 13:55:07 +08:00
zengqiao
b101cec6fa 健康分调整为健康状态 2022-10-29 13:55:07 +08:00
zengqiao
e98ec562a2 Znode信息中,补充当前节点路径信息 2022-10-29 13:55:07 +08:00
zengqiao
0e71ecc587 延长健康检查结果过期时间 2022-10-29 13:55:07 +08:00
zengqiao
0f11a65df8 补充获取ZK的namespace的方法 2022-10-29 13:55:07 +08:00
zengqiao
da00c8c877 还原消费组重置失败的提示文案 2022-10-29 13:55:07 +08:00
hongtenzone@foxmail.com
8b177877bb Add release notes 2022-10-28 15:35:26 +08:00
hongtenzone@foxmail.com
ea199dca8d Add release notes 2022-10-28 15:35:26 +08:00
renxiangde
88b5833f77 [Bugfix] 修复新建Topic后,立即查看Topic-Messages信息会提示Topic不存在的问题 (#697) 2022-10-27 11:04:26 +08:00
zwen
127b5be651 [fix]Repair that preferredReplicaElection is not called as expected 2022-10-27 10:15:15 +08:00
Mengqi777
80f001cdd5 [ISSUE #723]Ignore error and continue to package km-rest if no git directory 2022-10-26 10:14:14 +08:00
zengqiao
30d297cae1 bump version to 3.1.0-SNAPSHOT 2022-10-21 17:13:02 +08:00
zengqiao
a96853db90 bump version to v3.0.1 2022-10-21 15:02:09 +08:00
zengqiao
c1502152c0 Revert "bump version to 3.1.0"
This reverts commit 7b5c2d80
2022-10-21 14:59:42 +08:00
GraceWalk
afda292796 fix: typescript 版本更新 2022-10-21 14:47:01 +08:00
GraceWalk
163cab78ae fix: 部分文案 & 样式优化 2022-10-21 14:47:01 +08:00
GraceWalk
8f4ff36c09 fix: 优化 Topic 扩分区名称 & 描述展示 2022-10-21 14:47:01 +08:00
GraceWalk
47b6b3577a fix: Broker 列表 jmxPort 列支持展示连接状态 2022-10-21 14:47:01 +08:00
GraceWalk
f3eca3b214 fix: ConsumerGroup 列表 & 详情页重构 2022-10-21 14:47:00 +08:00
GraceWalk
62f7d3f72f fix: 图表逻辑 & 展示优化 2022-10-21 14:47:00 +08:00
GraceWalk
26e60d8a64 fix: 优化全局 Message & Notification 展示效果 2022-10-21 14:47:00 +08:00
zengqiao
df655a250c 增加v3.0.1变更内容 2022-10-21 14:36:29 +08:00
zengqiao
811fc9b400 补充v3.0.1版本升级信息 2022-10-21 14:32:57 +08:00
zengqiao
83df02783c 安装包中,去除docs相关的文档 2022-10-21 14:32:07 +08:00
zengqiao
6a5efce874 [Bugfix] 修复指标版本信息list转map时出现key冲突从而抛出异常的问题 2022-10-21 12:06:22 +08:00
zengqiao
fa0ae5e474 [Optimize] 集群Broker列表中,补充Jmx是否成功连接的信息
1、当前页面无数据时,一部分的原因是JMX连接失败导致;
2、Broker列表中增加是否连接成功的信息,便于问题的排查;
2022-10-21 12:03:19 +08:00
zengqiao
cafd665a2d [Optimize] 删除Replica指标采集任务
1、当集群存在较多副本时,指标采集的性能会严重降低;
2、Replica的指标基本上都是在实时获取时才需要,因此当前先将Replica指标采集任务关闭,后续依据产品需要再看是否开启;
2022-10-21 11:49:58 +08:00
zengqiao
e8f77a456b [Optimize] 优化ZK指标的获取,减少重复采集的出现 (#709)
1、避免不同集群,相同的ZK地址时,指标重复获取的情况;
2、避免集群某个ZK地址获取指标失败时,下一个周期还会继续尝试从该地址获取指标;
2022-10-21 11:26:07 +08:00
_haoqi
4510c62ebd [ISSUE #677] 重启会导致部分信息采集抛出空指针 2022-10-20 15:36:32 +08:00
zengqiao
79864955e1 [Feature] 集群Group列表按照Group维度进行展示 (#580) 2022-10-20 13:29:43 +08:00
Richard
ff26a8d46c fix issue:
* [issue #700] Adjust the prompt and replace the Arrays.asList() with the Collections.singletonList()
2022-10-19 15:19:43 +08:00
dianyang12138
cc226d552e fix:修复es模版错误 2022-10-19 11:44:00 +08:00
EricZeng
962f89475b Merge pull request #699 from silent-night-no-trace/dev
[ISSUE #683]  fix ldap bug
2022-10-19 10:23:47 +08:00
night.liang
ec204a1605 fix ldap bug 2022-10-18 20:16:40 +08:00
早晚会起风
58d7623938 Merge pull request #696 from chenzhongyu11/dev
[ISSUE #672] 修复健康巡检结果时间展示错误的问题
2022-10-18 10:41:47 +08:00
EricZeng
8f4ecfcdc0 Merge pull request #691 from didi/dev
补充Kafka-Group表
2022-10-17 20:30:32 +08:00
zengqiao
ef719cedbc 补充Kafka-Group表 2022-10-17 10:34:21 +08:00
EricZeng
b7856c892b Merge pull request #690 from didi/master
合并默认分支
2022-10-17 10:30:18 +08:00
EricZeng
7435a78883 Merge pull request #689 from didi/dev
优化健康检查结果替换时出现死锁问题
2022-10-17 10:26:11 +08:00
chenzy
f49206b316 修复时间展示有误的bug,由原先的12小时制改为24小时制 2022-10-16 22:57:50 +08:00
EricZeng
7d500a0721 Merge pull request #684 from RichardZhengkay/dev
fix issue: [#662]
2022-10-15 14:39:37 +08:00
EricZeng
98a519f20b Merge pull request #682 from haoqi123/fix_678
[ISSUE #678] zk-Latency avg为多位小数会抛出空指针
2022-10-15 14:17:23 +08:00
Richard
39b655bb43 fix issue:
* [issue #662] Fix deadlocks caused by adding data using MySQL's REPLACE method
2022-10-14 14:03:16 +08:00
_haoqi
78d56a49fe 修改zk-Latency avg为小数时的数值转换异常问题 2022-10-14 11:53:48 +08:00
EricZeng
d2e9d1fa01 Merge pull request #673 from didi/dev
fix [ISSUE-666] Error in ks_km_zookeeper table role type #666
2022-10-13 18:57:06 +08:00
zengqiao
41ff914dc3 修复ZK元信息表role字段类型错误问题 2022-10-13 18:50:41 +08:00
shirenchuang
3ba447fac2 update readme 2022-10-13 18:49:06 +08:00
shirenchuang
e9cc380a2e update readme 2022-10-13 18:30:13 +08:00
EricZeng
017cac9bbe Merge pull request #670 from RichardZhengkay/dev
fix issue: [#666]
2022-10-13 18:25:15 +08:00
Richard
9ad72694af fix issue:
* [issue #666] Fix the type of role phase in ks_km_zookeeper table
2022-10-13 18:00:43 +08:00
shirenchuang
e8f9821870 Merge remote-tracking branch 'origin/master' 2022-10-13 16:31:03 +08:00
shirenchuang
bb167b9f8d update readme 2022-10-13 15:31:34 +08:00
石臻臻的杂货铺
28fbb5e130 Merge pull request #665 from zwOvO/patch-1
[ISSUE #664]关于'JMX-连接失败问题解决'的超链接修复
2022-10-13 10:17:29 +08:00
EricZeng
16101e81e8 Merge pull request #661 from didi/dev
合并开发分支
2022-10-13 10:16:14 +08:00
赤月
aced504d2a Update faq.md 2022-10-12 22:08:29 +08:00
shirenchuang
abb064d9d1 update readme add who's using know streaming 2022-10-12 19:15:19 +08:00
zengqiao
dc1899a1cd 修复集群ZK列表中缺少返回服务状态字段的问题 2022-10-12 16:45:47 +08:00
zengqiao
442f34278c 指标信息中,增加返回ZK的指标信息 2022-10-12 16:44:07 +08:00
zengqiao
a6dcbcd35b 删除未被使用的import 2022-10-12 16:43:16 +08:00
zengqiao
2b600e96eb 健康检查任务优化 2022-10-12 16:41:27 +08:00
zengqiao
177bb80f31 application.yml文件中增加ES用户名密码的配置项 2022-10-12 16:36:04 +08:00
zengqiao
63fbe728c4 增加ZK指标上报普罗米修斯 2022-10-12 11:11:25 +08:00
EricZeng
b33020840b ZookeeperService中增加服务存活统计方法(#659) 2022-10-12 11:07:52 +08:00
zengqiao
c5caf7c0d6 ZookeeperService中增加服务存活统计方法 2022-10-12 11:02:41 +08:00
EricZeng
0f0473db4c 增加float转integer方法(#658)
增加float转integer方法
2022-10-12 10:09:16 +08:00
zengqiao
beadde3e06 增加float转integer方法 2022-10-11 18:46:16 +08:00
EricZeng
a423a20480 修复获取TopN的Broker指标时,会出现部分指标缺失的问题(#657)
修复获取TopN的Broker指标时,会出现部分指标缺失的问题
2022-10-11 18:44:02 +08:00
shirenchuang
79f0a23813 update contribuer document 2022-10-11 17:38:15 +08:00
zengqiao
780fdea2cc 修复获取TopN的Broker指标时,会出现部分指标缺失的问题 2022-10-11 16:54:39 +08:00
shirenchuang
1c0fda1adf Merge remote-tracking branch 'origin/master' 2022-10-11 10:39:08 +08:00
EricZeng
9cf13e9b30 Broker增加服务是否存活接口(#654)
Broker增加服务是否存活接口
2022-10-10 19:56:12 +08:00
zengqiao
87cd058fd8 Broker增加服务是否存活接口 2022-10-10 19:54:47 +08:00
EricZeng
81b1ec48c2 调整贡献者名单(#653)
调整贡献者名单
2022-10-10 19:52:50 +08:00
zengqiao
66dd82f4fd 调整贡献者名单 2022-10-10 19:49:22 +08:00
EricZeng
ce35b23911 修复DSL错误导致ZK指标查询失败问题(#652)
修复DSL错误导致ZK指标查询失败问题
2022-10-10 19:27:48 +08:00
zengqiao
e79342acf5 修复DSL错误导致ZK指标查询失败问题 2022-10-10 19:19:05 +08:00
EricZeng
3fc9f39d24 Merge pull request #651 from didi/master
合并主分支
2022-10-10 19:10:48 +08:00
shirenchuang
0221fb3a4a 贡献者相关文档 2022-10-10 18:02:19 +08:00
shirenchuang
f009f8b7ba 贡献者相关文档 2022-10-10 17:21:21 +08:00
shirenchuang
b76959431a 贡献者相关文档 2022-10-10 16:55:33 +08:00
shirenchuang
975370b593 贡献者相关文档 2022-10-10 15:57:07 +08:00
shirenchuang
7275030971 贡献者相关文档 2022-10-10 15:50:16 +08:00
shirenchuang
99b0be5a95 Merge branch 'master' into docs_only 2022-10-10 15:01:00 +08:00
石臻臻的杂货铺
edd3f95fc4 Update CONTRIBUTING.md 2022-10-10 14:22:24 +08:00
石臻臻的杂货铺
479f983b09 Update CONTRIBUTING.md 2022-10-10 13:58:35 +08:00
石臻臻的杂货铺
7650332252 Update CONTRIBUTING.md 2022-10-10 13:50:55 +08:00
shirenchuang
8f1a021851 readme 2022-10-10 13:46:14 +08:00
shirenchuang
ce4df4d5fd Merge remote-tracking branch 'origin/master' 2022-10-10 13:00:28 +08:00
shirenchuang
bd43ae1b5d Issue 模板 2022-10-10 12:57:53 +08:00
石臻臻的杂货铺
8fa34116b9 Merge pull request #648 from didi/docs_only
PR 模板
2022-10-10 12:39:38 +08:00
shirenchuang
7e92553017 PR 模板 2022-10-10 11:42:04 +08:00
shirenchuang
b7e243a693 Merge remote-tracking branch 'origin/master' 2022-10-09 17:23:16 +08:00
shirenchuang
35d4888afb 贡献者规约文档 2022-10-09 17:03:46 +08:00
EricZeng
b3e8a4f0f6 Merge pull request #647 from didi/dev
合并DEV分支
2022-10-09 16:54:45 +08:00
shirenchuang
321125caee issue template 2022-10-09 15:47:13 +08:00
shirenchuang
e01427aa4f issue template 2022-10-09 15:42:40 +08:00
shirenchuang
14652e7f7a issue template 2022-10-09 15:39:20 +08:00
shirenchuang
7c05899dbd issue template 2022-10-09 15:26:57 +08:00
shirenchuang
56726b703f issue template 2022-10-09 13:56:44 +08:00
shirenchuang
6237b0182f issue template 2022-10-09 12:27:27 +08:00
EricZeng
be5b662f65 Merge pull request #645 from didi/dev_feature_zk_kerberos
如何修改代码支持ZK-Kerberos认证
2022-10-09 10:39:26 +08:00
EricZeng
224698355c 恢复为原先代码
恢复为原先代码
2022-10-09 10:38:36 +08:00
EricZeng
8f47138ecd Merge pull request #643 from didi/dev_3.1
监控Kafka的ZK
2022-10-08 17:22:03 +08:00
zengqiao
d159746391 调整接入带Kerberos认证的ZK集群的文档 2022-10-08 17:00:08 +08:00
EricZeng
63df93ea5e Merge pull request #608 from luhea/dev_feature_zk_kerberos
Add zk supported kerberos
2022-10-08 16:11:37 +08:00
EricZeng
38948c0daa Merge pull request #644 from didi/master
合并主分支
2022-10-08 16:09:40 +08:00
zengqiao
6c610427b6 ZK-增加ZK信息查询接口 2022-10-08 15:46:18 +08:00
zengqiao
b4cc31c459 ZK-指标采集入ES 2022-10-08 15:31:59 +08:00
zengqiao
7d781712c9 ZK-同步ZK元信息至DB 2022-10-08 15:19:09 +08:00
zengqiao
dd61ce9b2a ZK-增加配置的默认值 2022-10-08 14:58:28 +08:00
zengqiao
69a7212986 ZK-增加四字命令信息的获取 2022-10-08 14:52:17 +08:00
EricZeng
ff05a951fd Merge pull request #642 from didi/master
合并主分支
2022-10-08 14:42:37 +08:00
EricZeng
89d5357b40 Merge pull request #641 from didi/dev
删除无效的健康分计算代码
2022-10-08 14:41:27 +08:00
zengqiao
7ca3d65c42 删除无效的健康分计算代码 2022-10-08 14:15:20 +08:00
zengqiao
7b5c2d800f bump version to 3.1.0 2022-09-29 15:13:41 +08:00
EricZeng
f414b47a78 1、补充升级至v3.0.0信息;2、增加v3.0.0变更内容;(#636)
1、补充升级至v3.0.0信息;2、增加v3.0.0变更内容;(#)
2022-09-29 13:08:32 +08:00
EricZeng
44f4e2f0f9 Merge pull request #635 from didi/dev
合并前端调整的内容
2022-09-29 11:50:25 +08:00
zengqiao
2361008bdf Merge branch 'dev' of github.com:didi/KnowStreaming into dev 2022-09-29 11:49:00 +08:00
zengqiao
7377ef3ec5 增加v3.0.0变更内容 2022-09-29 11:45:29 +08:00
lucasun
a28d064b7a Merge pull request #634 from GraceWalk/dev
前端 bug 修复 & 问题优化
2022-09-29 11:23:25 +08:00
GraceWalk
e2e57e8575 fix: 依赖版本更新 2022-09-29 11:15:47 +08:00
zengqiao
9d90bd2835 补充升级至v3.0.0信息 2022-09-29 11:04:49 +08:00
EricZeng
7445e68df4 Merge pull request #632 from didi/master
合并主分支
2022-09-29 10:53:54 +08:00
GraceWalk
ab42625ad2 fix: 数字展示格式化 2022-09-29 10:52:31 +08:00
GraceWalk
18789a0a53 fix: IconFont 组件改为从独立包引入 2022-09-29 10:51:52 +08:00
zengqiao
68a37bb56a Merge branch 'master' of github.com:didi/KnowStreaming 2022-09-29 10:49:46 +08:00
GraceWalk
3b33652c47 fix: Rebalance 卡片 icon 颜色调整 2022-09-29 10:48:52 +08:00
GraceWalk
1e0c4c3904 feat: Topic 详情消息 Value 列支持复制 2022-09-29 10:48:09 +08:00
zengqiao
04e223de16 修改协议文案 2022-09-29 10:48:00 +08:00
GraceWalk
c4a691aa8a fix: 多集群列表兼容集群无 ZK 情况 2022-09-29 10:44:28 +08:00
GraceWalk
ff9dde163a feat: 图表支持存储拖拽排序 & 补点逻辑优化 2022-09-29 10:42:44 +08:00
EricZeng
eb7efbd1a5 增加字段校验注解(#631)
增加字段校验注解(#631)
2022-09-29 09:59:02 +08:00
zengqiao
8c8c362c54 Merge branch 'dev' of github.com:didi/KnowStreaming into dev 2022-09-28 20:19:35 +08:00
zengqiao
66e119ad5d 增加字段校验注解 2022-09-28 20:16:06 +08:00
EricZeng
6dedc04a05 Merge pull request #630 from didi/dev
合并开发分支
2022-09-28 20:14:51 +08:00
EricZeng
0cf8bad0df Merge pull request #629 from didi/master
合并主分支
2022-09-28 20:06:26 +08:00
zengqiao
95c9582d8b 优化消费组详情指标为实时获取 2022-09-28 20:03:23 +08:00
EricZeng
7815126ff5 1、修复Group指标防重复不生效问题;2、修复自动创建ES索引模版失败问题; (#628)
* 修复自动创建ES索引模版失败问题

* 修复Group指标防重复不生效问题

Co-authored-by: zengqiao <zengqiao@didiglobal.com>
2022-09-28 19:55:30 +08:00
zengqiao
a5fa9de54b 修复Group指标防重复不生效问题 2022-09-28 19:52:11 +08:00
zengqiao
95f1a2c630 修复自动创建ES索引模版失败问题 2022-09-28 19:46:07 +08:00
zengqiao
1e256ae1fd 修复自动创建ES索引模版失败问题 2022-09-28 19:44:33 +08:00
zengqiao
9fc9c54fa1 bump version to 3.0.0 2022-09-28 11:20:16 +08:00
zengqiao
1b362b1e02 Merge branch 'master' of github.com:didi/KnowStreaming 2022-09-28 11:16:54 +08:00
EricZeng
04e3172cca [ISSUE-624]过滤掉不存在的Topic(#625)
[ISSUE-624]过滤掉不存在的Topic(#625)
2022-09-28 11:13:15 +08:00
EricZeng
1caab7f3f7 [ISSUE-624]过滤掉不存在的Topic(#624)
[ISSUE-624]过滤掉不存在的Topic(#624)
2022-09-28 10:41:39 +08:00
zengqiao
9d33c725ad [ISSUE-624]过滤掉不存在的Topic(#624)
同步Group元信息时,如果Topic已经不存在了,则过滤掉该Group+Topic信息
2022-09-28 10:39:33 +08:00
EricZeng
6ed1d38106 [ISSUE-598]Fix start_time not set when create reassign job in MySQL-8 (#623)
[ISSUE-598]Fix start_time not set when create reassign job in MySQL-8 (#623 )
2022-09-28 10:26:56 +08:00
zengqiao
0f07ddedaf [ISSUE-598]Fix start_time not set when create reassign job in MySQL-8 2022-09-28 10:24:32 +08:00
EricZeng
289945b471 Merge pull request #622 from didi/dev
后端补充Kafka集群运行模式字段信息
2022-09-28 10:08:17 +08:00
zengqiao
f331a6d144 后端补充Kafka集群运行模式字段信息 2022-09-27 18:43:22 +08:00
EricZeng
0c8c12a651 Merge pull request #621 from didi/dev
指标发送ES类按照指标类别拆分
2022-09-27 18:38:05 +08:00
zengqiao
028c3bb2fa 指标发送ES类按照指标类别拆分 2022-09-27 10:19:18 +08:00
EricZeng
d7a5a0d405 健康巡检任务按照类型进行拆分
健康巡检任务按照类型进行拆分
2022-09-27 10:17:12 +08:00
zengqiao
5ef5f6e531 健康巡检任务按照类型进行拆分 2022-09-26 20:10:49 +08:00
EricZeng
1d205734b3 Merge pull request #619 from didi/dev
集群信息中,补充ZK配置字段
2022-09-26 19:50:26 +08:00
Peng
5edd43884f Update README.md 2022-09-26 18:43:25 +08:00
zengqiao
c1992373bc 集群信息中,补充ZK配置字段 2022-09-26 11:10:38 +08:00
EricZeng
ed562f9c8a Merge pull request #618 from didi/dev
DB中Group信息的更新方式,由replace调整为insert或update
2022-09-26 10:02:24 +08:00
zengqiao
b4d44ef8c7 DB中Group信息的更新方式,由replace调整为insert或update 2022-09-23 17:02:25 +08:00
EricZeng
ad0c16a1b4 升级Helm版本及增加Docker相关文件
升级Helm版本及增加Docker相关文件
2022-09-23 16:17:00 +08:00
wangdongfang-aden
7eabe66853 Merge pull request #616 from wangdongfang-aden/dev
添加docker-compose部署和更新helm
2022-09-23 14:50:22 +08:00
wangdongfang-aden
3983d73695 Update Chart.yaml 2022-09-23 14:47:40 +08:00
wangdongfang-aden
161d4c4562 Update 单机部署手册.md 2022-09-23 14:46:27 +08:00
wangdongfang-aden
9a1e89564e Update 单机部署手册.md 2022-09-23 14:44:49 +08:00
wangdongfang-aden
0c18c5b4f6 Update 单机部署手册.md 2022-09-23 14:43:23 +08:00
wangdongfang-aden
3e12ba34f7 Update docker-compose.yml 2022-09-23 14:33:05 +08:00
wangdongfang-aden
e71e29391b Delete ks-start.sh 2022-09-23 14:26:24 +08:00
wangdongfang-aden
9b7b9a7af0 Delete es_template_create.sh 2022-09-23 14:26:16 +08:00
wangdongfang-aden
a23819c308 Create ks-start.sh 2022-09-23 14:19:35 +08:00
wangdongfang-aden
6cb1825d96 Create es_template_create.sh 2022-09-23 14:19:10 +08:00
wangdongfang-aden
77b8c758dc Create initsql 2022-09-23 14:18:17 +08:00
wangdongfang-aden
e5a582cfad Create my.cnf 2022-09-23 14:17:25 +08:00
wangdongfang-aden
ec83db267e Create init.sh 2022-09-23 14:17:02 +08:00
wangdongfang-aden
bfd026cae7 Create dockerfile 2022-09-23 14:16:28 +08:00
wangdongfang-aden
35f1dd8082 Create dockerfile 2022-09-23 14:14:47 +08:00
wangdongfang-aden
7ed0e7dd23 Create dockerfile 2022-09-23 14:14:02 +08:00
wangdongfang-aden
1a3cbf7a9d Create knowstreaming.conf 2022-09-23 14:07:04 +08:00
wangdongfang-aden
d9e4abc3de Create ks-start.sh 2022-09-23 14:05:59 +08:00
wangdongfang-aden
a4186085d3 Create es_template_create.sh 2022-09-23 14:05:05 +08:00
wangdongfang-aden
26b1846bb4 Create docker-compose.yml 2022-09-23 14:03:14 +08:00
wangdongfang-aden
1aa89527a6 helm update 3.0.0-beta.3 2022-09-23 11:36:46 +08:00
wangdongfang-aden
eac76d7ad0 helm update 3.0.0-beta.3 2022-09-23 11:36:01 +08:00
wangdongfang-aden
cea0cd56f6 Merge pull request #607 from haoqi123/dev
[单机部署手册.md]docker-compose部署方式添加注释描述
2022-09-23 10:27:04 +08:00
EricZeng
c4b897f282 bump version to 3.0.0-beta.4
bump version to 3.0.0-beta.4
2022-09-23 10:24:52 +08:00
zengqiao
47389dbabb bump version to 3.0.0-beta.4 2022-09-23 10:17:58 +08:00
haoqi
a2f8b1a851 1. [单机部署手册.md]docker-compose部署方式添加注释描述 2022-09-22 19:46:21 +08:00
EricZeng
feac0a058f Merge pull request #613 from didi/dev
补充v3.0.0-beta.2变更信息
2022-09-22 17:30:35 +08:00
zengqiao
27eeac9fd4 补充v3.0.0-beta.2变更信息 2022-09-22 17:28:51 +08:00
EricZeng
a14db4b194 Merge pull request #612 from didi/dev
合并开发分支
2022-09-22 17:28:09 +08:00
lucasun
54ee271a47 Merge pull request #611 from GraceWalk/dev
修复前端bug和体验问题
2022-09-22 15:51:46 +08:00
GraceWalk
a3a9be4f7f fix: 更正前端本地环境接口代理地址 2022-09-22 15:37:24 +08:00
GraceWalk
d4f0a832f3 fix: 样式更新 2022-09-22 15:31:52 +08:00
GraceWalk
7dc533372c fix: 更正文件引用路径 2022-09-22 15:31:34 +08:00
GraceWalk
1737d87713 fix: 修复配置无法删除的问题 2022-09-22 15:31:13 +08:00
GraceWalk
dbb98dea11 fix: 更新登录页图片 2022-09-22 15:21:04 +08:00
GraceWalk
802b382b36 fix: Topic Messages 详情提示优化 2022-09-22 15:20:31 +08:00
GraceWalk
fc82999d45 fix: 消费测试 Message 限制最大值 2022-09-22 15:19:56 +08:00
GraceWalk
08aa000c07 refactor: 接入/编辑集群优化 2022-09-22 15:19:03 +08:00
GraceWalk
39015b5100 feat: 多集群管理列表页增加手动刷新功能 2022-09-22 15:18:13 +08:00
GraceWalk
0d635ad419 refactor: webpack 配置结构调整 2022-09-22 15:13:25 +08:00
EricZeng
9133205915 Merge pull request #610 from didi/dev
合并开发分支
2022-09-22 14:51:23 +08:00
zengqiao
725ac10c3d 1、调整KafkaZKDao位置;2、offset信息获取时,过滤掉无leader分区;3、调整验证ZK是否合法时的session超时时间 2022-09-22 11:30:46 +08:00
zengqiao
2b76358c8f Overview页面,后端增加排序信息 2022-09-22 11:24:13 +08:00
zengqiao
833c360698 bump oshi-core version to 5.6.1 2022-09-22 11:17:59 +08:00
zengqiao
7da1e67b01 FAQ补充权限识别失败问题说明 2022-09-22 11:13:54 +08:00
GraceWalk
7eb86a47dd fix: 部分依赖更新 2022-09-21 16:22:45 +08:00
GraceWalk
d67e383c28 feat: 系统管理列表增加手动刷新功能 2022-09-21 16:21:57 +08:00
GraceWalk
8749d3e1f5 fix: config 子应用 axios 配置错误兼容 2022-09-21 16:21:07 +08:00
GraceWalk
30fba21c48 fix: 生产测试单词发送消息数限制为 0~1000 2022-09-21 16:15:19 +08:00
GraceWalk
d83d35aee9 fix: 样式 & 文案优化 2022-09-21 16:12:13 +08:00
GraceWalk
1d3caeea7d feat: Cluster 图表去掉放大功能 2022-09-21 16:11:14 +08:00
luhe
c8806dbb4d 修改代码支持ZK-Kerberos认证与配置文档 2022-09-21 16:09:04 +08:00
luhe
e5802c7f50 修改代码支持ZK-Kerberos认证与配置文档 2022-09-21 16:02:38 +08:00
luhe
590f684d66 修改代码支持ZK-Kerberos认证与配置文档 2022-09-21 15:59:31 +08:00
luhe
8e5a67f565 修改代码支持ZK-Kerberos认证 2022-09-21 15:58:59 +08:00
luhe
8d2fbce11e 修改代码支持ZK-Kerberos认证 2022-09-21 15:54:30 +08:00
haoqi
26916f6632 1. [单机部署手册.md]docker-compose部署方式添加注释描述
2. 更改docker-compose中ui对外访问port为80
2022-09-21 12:55:43 +08:00
EricZeng
fbfa0d2d2a Merge pull request #600 from haoqi123/dev
docker-compose addition
2022-09-21 10:49:08 +08:00
haoqi
e626b99090 1. 删除km-dist/docker文件夹,以[单机部署手册.md]为准 2022-09-20 19:30:20 +08:00
haoqi123
203859b71b Merge branch 'didi:dev' into dev 2022-09-20 19:25:12 +08:00
haoqi
9a25c22f3a 1. 调整docker-compose.yml中各个服务的镜像
2. 经过@wangdongfang-aden大哥的调试将helm与docker镜像合二为一,于是删减掉各个镜像的Dockerfile与启动脚本,后续也不需要额外维护
2022-09-20 19:23:18 +08:00
zengqiao
0a03f41a7c 后端增加指标摆放顺序功能 2022-09-20 14:42:22 +08:00
zengqiao
56191939c8 Merge branch 'dev' of github.com:didi/KnowStreaming into dev 2022-09-20 14:23:09 +08:00
zengqiao
beb754aaaa 修复JMX连接被关闭,抛出IOException后,未进行连接重建的问题 2022-09-20 14:22:06 +08:00
EricZeng
f234f740ca Merge pull request #603 from didi/dev
合并开发分支
2022-09-20 10:51:39 +08:00
EricZeng
e14679694c Merge pull request #602 from f1558/dev
fix issue
2022-09-20 10:31:16 +08:00
zengqiao
e06712397e 修复因DB中Broker信息不存在导致TotalLogSize指标获取时抛空指针问题 2022-09-20 10:27:30 +08:00
Richard
b6c6df7ffc fix issue
* SQL specification comments to avoid direct operation failure
2022-09-20 09:42:42 +08:00
zengqiao
375c6f56c9 修改GroupOffsetResetEnum类名为OffsetTypeEnum 2022-09-19 13:55:59 +08:00
EricZeng
0bf85c97b5 Merge pull request #555 from superspeedone/dev
Dev
2022-09-19 11:18:28 +08:00
EricZeng
630e582321 Merge pull request #593 from Mengqi777/mengqi-dev
fix: adjust os judgment method with uname
2022-09-19 10:34:16 +08:00
EricZeng
a89fe23bdd Merge pull request #597 from WYAOBO/dev
文档更新
2022-09-19 10:15:38 +08:00
haoqi
a7a5fa9a31 1. 调整docker-compose.yml中networks配置
2. ks-manager添加健康检查
3. 更新单机部署手册
2022-09-18 19:10:22 +08:00
_haoqi
c73a7eee2f 1. 调整docker-compose服务,容器名称 2022-09-16 20:03:58 +08:00
_haoqi
121f8468d5 1. 调整文件格式LF
2. 调整docker-compose服务,容器名称
2022-09-16 17:33:19 +08:00
haoqi
7b0b6936e0 1. 调整docker-compose.yml中容器名称 2022-09-16 15:54:34 +08:00
Peng
597ea04a96 Update README.md 2022-09-16 15:20:04 +08:00
Peng
f7f90aeaaa Update README.md 2022-09-16 15:18:29 +08:00
_haoqi
227479f695 1. 修改dockerfile
2. 删除无用配置文件
2022-09-16 15:13:18 +08:00
WYAOBO
6477fb3fe0 Merge branch 'didi:dev' into dev 2022-09-16 14:50:13 +08:00
wangdongfang-aden
4223f4f3c4 Merge pull request #596 from wangdongfang-aden/dev
helm update 3.0.0-beta.2
2022-09-16 14:45:43 +08:00
wangdongfang-aden
7288874d72 helm update 3.0.0-beta.2 2022-09-16 14:44:14 +08:00
wangdongfang-aden
68f76f2daf helm update 3.0.0-beta.2 2022-09-16 14:42:34 +08:00
wyb
fe6ddebc49 文档更新 2022-09-16 14:41:45 +08:00
wangdongfang-aden
12b5acd073 helm update 3.0.0-beta.2 2022-09-16 14:41:40 +08:00
wangdongfang-aden
a6f1fe07b3 helm update 3.0.0-beta.2 2022-09-16 14:41:02 +08:00
wangdongfang-aden
85e3f2a946 helm update 3.0.0-beta.2 2022-09-16 14:40:34 +08:00
pokemeng
d4f416de14 fix: adjust os judgment method with uname 2022-09-16 11:34:03 +08:00
haoqi
0d9a6702c1 1. 更改es初始化脚本输出追加为重定向 2022-09-15 17:13:58 +08:00
haoqi
d11285cdbf Merge branch 'master' into dev
# Conflicts:
#	km-dist/init/sql/ddl-logi-security.sql
2022-09-15 17:01:39 +08:00
EricZeng
5f1f33d2b9 Merge pull request #591 from didi/master
合并主分支
2022-09-15 16:59:11 +08:00
zengqiao
474daf752d bump version to 3.0.0-beta.3 2022-09-15 16:54:52 +08:00
haoqi
27d1b92690 1. 添加init容器,只用于初始化es索引 2022-09-15 16:22:51 +08:00
zengqiao
993afa4c19 默认用户名密码调整说明 2022-09-15 16:20:13 +08:00
EricZeng
028d891c32 Merge pull request #588 from didi/dev_v3.0.0-beta.2
合并v3.0.0 beta.2
2022-09-15 15:46:58 +08:00
zengqiao
0df55ec22d 更新3.0.0-beta.2升级手册 2022-09-15 15:23:29 +08:00
zengqiao
579f64774d 更新3.0.0-beta.2变更说明 2022-09-15 15:20:50 +08:00
haoqi
792f8d939d 1. 更改Dockerfile 2022-09-15 15:06:19 +08:00
EricZeng
e4fb02fcda Merge pull request #587 from didi/dev
合并开发分支
2022-09-15 14:35:00 +08:00
haoqi
0c14c641d0 1. 添加docker-compose部署方式
2. 更改manage服务初始化方式
3. 更改es初始化方式
2022-09-15 14:26:45 +08:00
EricZeng
dba671fd1e Merge pull request #586 from GraceWalk/dev
Dev
2022-09-15 13:49:04 +08:00
GraceWalk
80d1693722 fix: 修复单集群详情引导步骤定位错误的问题 2022-09-15 13:39:09 +08:00
GraceWalk
26014a11b2 feat: 补充前端打包构建部分文档说明 2022-09-15 13:36:09 +08:00
GraceWalk
848fddd55a fix: 切换依赖安装源为 taobao 镜像 2022-09-15 13:34:15 +08:00
EricZeng
97f5f05f1a Merge pull request #585 from didi/dev
更新单机部署文档
2022-09-15 13:05:14 +08:00
zengqiao
25b82810f2 更新单机部署文档 2022-09-15 13:01:33 +08:00
EricZeng
9b1e506fa7 Merge pull request #584 from didi/dev
修复日志表字段过短问题
2022-09-15 12:56:49 +08:00
zengqiao
7a42996e97 修复日志表字段过短问题 2022-09-15 12:55:06 +08:00
EricZeng
dbfcebcf67 Merge pull request #583 from didi/dev
合并开发分支
2022-09-15 12:38:11 +08:00
zengqiao
37c3f69a28 修复类型转化失败问题 2022-09-15 11:32:44 +08:00
zengqiao
5d412890b4 调整超时时间配置 2022-09-15 11:31:25 +08:00
zengqiao
1e318a4c40 修改默认的用户名密码 2022-09-15 11:31:03 +08:00
EricZeng
d4549176ec Merge pull request #566 from lomodays207/master
解决 java.lang.NumberFormatException: For input string: "{"value":0,"relation":"eq"}" 问题
2022-09-15 10:05:26 +08:00
haoqi
61efdf492f 添加docker-compose部署方式 2022-09-13 23:20:41 +08:00
lucasun
67ea4d44c8 Merge pull request #575 from GraceWalk/dev
同步前端代码
2022-09-13 15:13:02 +08:00
GraceWalk
fdae05a4aa fix: 登录页文案修改 2022-09-13 14:46:42 +08:00
GraceWalk
5efb837ee8 fix: 单集群详情样式优化 2022-09-13 14:46:29 +08:00
GraceWalk
584b626d93 fix: 修复 Broker Card 返回数据后依旧展示加载态的问题 2022-09-13 14:45:56 +08:00
GraceWalk
de25a4ed8e fix: 修复 Broker Card 返回数据后依旧展示加载态的问题 2022-09-13 14:45:27 +08:00
GraceWalk
2e852e5ca6 fix: 修复用户登出后回退还可以访问系统的问题 2022-09-13 14:44:18 +08:00
GraceWalk
b11000715a 修复 Topic Config 编辑表单不能正确回显当前值的问题 2022-09-13 14:43:35 +08:00
GraceWalk
b3f8b46f0f fix: 修复扩缩/迁移副本无法选中默认 Topic 的问题 & 迁移副本 Topic 迁移时间单位支持分钟粒度 2022-09-13 14:42:21 +08:00
GraceWalk
8d22a0664a fix: Broker 列表标识当前 Controller 2022-09-13 14:37:20 +08:00
GraceWalk
20756a3453 fix: 重置 Offset 部分 partationId 修改为 Select & Offset 数值限制 2022-09-13 14:35:23 +08:00
GraceWalk
c9b4d45a64 fix: 修复 Job 扩缩副本任务明细错误的问题 2022-09-13 14:31:45 +08:00
GraceWalk
83f7f5468b fix: 均衡历史列表样式重构 & 周期均衡场景化 & 立即均衡默认带入周期均衡参数 2022-09-13 14:30:03 +08:00
GraceWalk
59c042ad67 fix: Topic 列表趋势图优化 & 相关文案调整 2022-09-13 14:26:12 +08:00
GraceWalk
d550fc5068 fix: 修复 Consume 点击 Stop 后未停止请求发送的问题 2022-09-13 14:24:30 +08:00
GraceWalk
6effba69a0 feat: 补充 ReBalance 和 Topic 部分权限项 2022-09-13 14:22:50 +08:00
GraceWalk
9b46956259 fix: Topic 详情 Partition Tab 卡片模式展示优化 2022-09-13 14:18:17 +08:00
GraceWalk
b5a4a732da fix: 健康分设置问题修复 2022-09-13 14:15:15 +08:00
GraceWalk
487862367e feat: 多集群列表支持编辑 & 代码结构优化 2022-09-13 14:14:15 +08:00
GraceWalk
5b63b9ce67 feat: 左侧栏内容调整 2022-09-13 14:12:34 +08:00
GraceWalk
afbcd3e1df fix: Broker/Topic 图表详情 bugfix & 体验优化 2022-09-13 14:09:57 +08:00
GraceWalk
12b82c1395 fix: 图表展示 bugifx & 优化 2022-09-13 14:09:03 +08:00
GraceWalk
863b765e0d feat: 新增 RenderEmpty 组件 2022-09-13 14:04:55 +08:00
GraceWalk
731429c51c fix: 系统管理子应用补充返回 code 码拦截逻辑 2022-09-13 11:44:47 +08:00
GraceWalk
66f3bc61fe fix: 创建/编辑角色优化 2022-09-13 11:44:08 +08:00
GraceWalk
4efe35dd51 fix: 项目打包构建流程优化 & 补充说明 2022-09-13 11:43:30 +08:00
EricZeng
c92461ef93 Merge pull request #565 from didi/dev
合并开发分支
2022-09-12 05:53:34 +08:00
superspeedone
405e6e0c1d Topic消息查询支持Timestamp排序,接口支持按指定日期查询 2022-09-09 18:56:45 +08:00
superspeedone
0d227aef49 Topic消息查询支持Timestamp排序,接口支持按指定日期查询 2022-09-09 17:29:22 +08:00
superspeedone
0e49002f42 Topic消息查询支持Timestamp排序,接口支持按指定日期查询 2022-09-09 15:45:31 +08:00
wangdongfang-aden
2e016800e0 Merge pull request #568 from wangdongfang-aden/dev
使用3.0.0-beta.1镜像
2022-09-09 15:18:36 +08:00
wangdongfang-aden
09f317b991 使用3.0.0-beta.1镜像 2022-09-09 15:17:02 +08:00
wangdongfang-aden
5a48cb1547 使用3.0.0-beta.1镜像 2022-09-09 15:16:33 +08:00
wangdongfang-aden
f632febf33 Update Chart.yaml 2022-09-09 15:15:56 +08:00
wangdongfang-aden
3c53467943 使用3.0.0-beta.1镜像 2022-09-09 15:15:24 +08:00
qiubo
d358c0f4f7 修复ES total 查询转换异常问题 2022-09-09 10:22:26 +08:00
zengqiao
de977a5b32 加快添加集群后的信息获取的速度 2022-09-08 14:21:26 +08:00
zengqiao
703d685d59 Task任务分为metrics,common,metaddata三类,每一类任务的执行对应一个线程池,减少对Job模块线程池的依赖 2022-09-08 14:17:15 +08:00
zengqiao
31a5f17408 修复旧副本数为NULL的问题 2022-09-08 13:53:41 +08:00
zengqiao
c40ae3c455 增加副本变更任务结束后,进行优先副本选举的操作 2022-09-08 13:52:51 +08:00
zengqiao
b71a34279e 调整默认的权限 2022-09-08 13:50:08 +08:00
zengqiao
8f8c0c4eda 删除无效文件 2022-09-08 13:49:07 +08:00
zengqiao
3a384f0e34 优化重置Offset时的错误信息 2022-09-08 13:47:21 +08:00
zengqiao
cf7bc11cbd 增加登录系统对接文档 2022-09-08 13:46:45 +08:00
EricZeng
be60ae8399 Merge pull request #560 from didi/dev
合并开发分支
2022-09-07 14:20:04 +08:00
superspeedone
8e50d145d5 Topic消息查询支持Timestamp排序,支持查询最新消息或最早消息 #534 2022-09-07 11:17:59 +08:00
zengqiao
7a3d15525c 支持Ldap登录认证 2022-09-06 15:25:27 +08:00
zengqiao
64f32d8b24 bump logi-security version to 2.10.13 and logi-elasticsearch-client version to 1.0.24 2022-09-06 15:24:05 +08:00
zengqiao
949d6ba605 集群Broker列表,增加Controller角色信息 2022-09-06 15:22:57 +08:00
zengqiao
ceb8db09f4 优化查询Topic信息,Topic不存在时的错误提示 2022-09-06 15:21:53 +08:00
zengqiao
ed05a0ebb8 修复集群Group列表搜索反馈结果错误问题 2022-09-06 15:20:50 +08:00
zengqiao
a7cbb76655 修复Offset单位错误问题 2022-09-06 15:19:29 +08:00
zengqiao
93cbfa0b1f 后端增加页面权限点 2022-09-06 15:18:54 +08:00
zengqiao
6120613a98 Merge branch 'dev' of github.com:didi/KnowStreaming into dev 2022-09-06 15:15:14 +08:00
EricZeng
dbd00db159 Merge pull request #559 from didi/master
合并主分支
2022-09-06 15:14:18 +08:00
zengqiao
befde952f5 补充KS连接特定JMX IP的说明 2022-09-06 15:13:03 +08:00
zengqiao
1aa759e5be bump version to 3.0.0-beta.2 2022-09-06 10:25:14 +08:00
EricZeng
2de27719c1 Merge pull request #557 from didi/dev
合并开发分支
2022-09-05 17:07:51 +08:00
EricZeng
21db57b537 Merge pull request #556 from GraceWalk/dev
fix: 锁定 tree-changes 版本
2022-09-05 16:55:22 +08:00
GraceWalk
dfe8d09477 fix: 锁定 tree-changes 版本 2022-09-05 16:53:49 +08:00
EricZeng
90dfa22c64 Merge pull request #554 from WYAOBO/master
修复jmx,ip错误
2022-09-05 16:08:03 +08:00
superspeedone
0f35427645 Merge branch 'dev' of https://github.com/superspeedone/KnowStreaming into dev 2022-09-05 15:41:08 +08:00
wangyaobo
7909f60ff8 修复jmx ip选择错误 2022-09-05 15:40:01 +08:00
WYAOBO
9a1a8a4c30 Merge pull request #1 from didi/dev
Dev
2022-09-05 15:31:34 +08:00
yanweiwen
fa7ad64140 Topic消息查询支持Timestamp排序,支持查询最新消息或最早消息 #534 2022-09-05 14:46:40 +08:00
EricZeng
8a0c23339d Merge pull request #551 from didi/dev
合并开发分支
2022-09-05 11:31:40 +08:00
zengqiao
e7ab3aff16 增加升级至v3.0.0-beta.1的内容 2022-09-05 11:29:38 +08:00
zengqiao
d0948797b9 增加v3.0.0-beta.1信息 2022-09-05 11:28:51 +08:00
EricZeng
04a5e17451 Merge pull request #550 from didi/dev
合并开发分支
2022-09-03 08:50:34 +08:00
zengqiao
47065c8042 文档名修改 2022-09-03 08:49:47 +08:00
EricZeng
488c778736 Merge pull request #548 from WYAOBO/master
新增task使用文档
2022-09-03 08:46:43 +08:00
EricZeng
d10a7bcc75 Merge pull request #549 from didi/dev
合并开发分支
2022-09-03 08:42:57 +08:00
zengqiao
afe44a2537 自动创建ES索引 & 主动填补指标历史曲线缺少的点 2022-09-03 08:34:32 +08:00
zengqiao
9eadafe850 健康巡检交由KS线程进行执行 2022-09-03 08:32:33 +08:00
zengqiao
dab3eefcc0 调整超时时间 2022-09-03 08:31:46 +08:00
zengqiao
2b9a6b28d8 消费组信息获取时不包含认证信息 2022-09-03 08:31:06 +08:00
zengqiao
465f98ca2b 日志错误信息中补充Topic名称信息 2022-09-03 08:28:38 +08:00
zengqiao
a0312be4fd Jmx连接的主机IP支持可选择 2022-09-03 08:26:50 +08:00
WYAOBO
4a5161372b Update KnowStreaming Task模块简介.md 2022-09-02 18:06:06 +08:00
WYAOBO
4c9921f752 Update KnowStreaming Task模块简介.md 2022-09-02 18:05:10 +08:00
WYAOBO
6dd72d40ee Update KnowStreaming Task模块简介.md 2022-09-02 18:04:14 +08:00
WYAOBO
db49c234bb Update KnowStreaming Task模块简介.md 2022-09-02 18:03:09 +08:00
WYAOBO
4a9df0c4d9 Update KnowStreaming Task模块简介.md 2022-09-02 18:02:21 +08:00
wangyaobo
461573c2ba task文档 2022-09-02 17:57:56 +08:00
wangyaobo
291992753f 文档更新 2022-09-02 17:28:18 +08:00
wangyaobo
fcefe7ac38 文档 2022-09-02 17:19:44 +08:00
zengqiao
7da712fcff 文案错误订正 2022-09-02 14:56:11 +08:00
zengqiao
2fd8687624 补充数据库错误说明 2022-09-02 14:55:02 +08:00
EricZeng
639b1f8336 Merge pull request #542 from didi/dev
前端优化
2022-09-02 12:46:04 +08:00
EricZeng
ab3b83e42a Merge pull request #539 from ruanliang-hualun/dev
备注与实现不一致,优先使用EXTERNAL_KEY
2022-09-02 12:42:30 +08:00
lucasun
4818629c40 Merge pull request #541 from GraceWalk/dev
前端代码打包优化 & bug 修复
2022-09-02 11:08:05 +08:00
ruanliang01
61784c860a 备注与实现不一致,优先使用EXTERNAL_KEY 2022-09-01 21:07:15 +08:00
GraceWalk
d5667254f2 fix: 样式 & 文案细节优化 2022-09-01 19:51:27 +08:00
GraceWalk
af2b93983f fix: 修正均衡卡片展示状态 2022-09-01 19:50:36 +08:00
GraceWalk
8281301cbd fix: 修复 Consumer 点击 Stop 不发停止检索问题修复 2022-09-01 19:50:04 +08:00
GraceWalk
0043ab8371 feat: 优化前端打包体积 2022-09-01 19:46:50 +08:00
GraceWalk
500eaace82 feat: 前端打包增加分包策略 2022-09-01 19:44:37 +08:00
GraceWalk
28e8540c78 fix: 修复创建/编辑角色报错的问题 2022-09-01 19:43:33 +08:00
EricZeng
69adf682e2 Merge pull request #537 from didi/dev
开发合入主分支
2022-09-01 17:06:32 +08:00
EricZeng
69cd1ff6e1 Merge pull request #536 from didi/master
合并主分支
2022-09-01 17:02:32 +08:00
EricZeng
415d67cc32 Merge pull request #535 from chaixiaoxue/dev
Fix page does not take effect
2022-09-01 16:51:32 +08:00
TinoC
46a2fec79b Fix page does not take effect 2022-09-01 16:35:30 +08:00
EricZeng
560b322fca Merge pull request #532 from Strangevy/dev_v3.0.0
Modify the default value of logi_security_oplog.operation_methods
2022-09-01 13:34:09 +08:00
Strangevy
effe17ac85 logi_security_oplog.operation_methods default '' 2022-09-01 10:28:00 +08:00
EricZeng
7699acfc1b Merge pull request #531 from didi/dev_v3.0.0
1、后端补充leader选举能力;2、图片链接调整;3、健康检查文案调整;4、版本列表增加排序;5、指标采集缓存时间调整;
2022-08-31 19:29:46 +08:00
zengqiao
6e058240b3 指标采集缓存时间调整 2022-08-31 17:15:49 +08:00
zengqiao
f005c6bc44 版本列表增加排序 2022-08-31 17:14:56 +08:00
zengqiao
7be462599f 健康检查文案调整 2022-08-31 17:13:18 +08:00
zengqiao
271ab432d9 图片链接调整 2022-08-31 17:12:34 +08:00
zengqiao
4114777a4e 补充leader选举能力 2022-08-31 17:11:12 +08:00
Peng
9189a54442 Update README.md 2022-08-30 21:56:43 +08:00
EricZeng
b95ee762e3 Merge pull request #529 from didi/dev_v3.0.0
解决raft集群controller信息不断记录问题
2022-08-30 19:42:06 +08:00
zengqiao
9e3c4dc06b 解决raft集群controller信息不断记录问题 2022-08-30 19:39:15 +08:00
EricZeng
1891a3ac86 Merge pull request #526 from yzydtc/dev_v3.0.0
[KnowStreaming-524] fix one markdown link 404 not found
2022-08-30 14:53:22 +08:00
zhiyuan.yang
9ecdcac06d [KnowStreaming-524] fix one markdown link 404 not found 2022-08-30 14:47:54 +08:00
EricZeng
790cb6a2e1 Merge pull request #521 from didi/dev_v3.0.0
1、bump version;2、ES客户端数可配置化;3、采样优化;4、文档修改;
2022-08-30 13:36:36 +08:00
EricZeng
4a98e5f025 Merge pull request #520 from didi/dev_v3.0.0
1、删除无效字段;2、日志优化;
2022-08-29 20:33:55 +08:00
zengqiao
507abc1d84 bump version 2022-08-29 20:32:41 +08:00
zengqiao
9b732fbbad ES客户端数可配置化 2022-08-29 20:32:01 +08:00
zengqiao
220f1c6fc3 采样优化 2022-08-29 20:31:34 +08:00
zengqiao
7a950c67b6 文档修改 2022-08-29 20:30:47 +08:00
zengqiao
78f625dc8c 日志优化 2022-08-29 16:45:48 +08:00
zengqiao
211d26a3ed 删除无效字段 2022-08-29 16:44:53 +08:00
EricZeng
dce2bc6326 Merge pull request #519 from didi/master
合并主分支
2022-08-29 16:24:02 +08:00
EricZeng
90e5d7f6f0 Merge pull request #516 from didi/v3.0.0-beta
V3.0.0 beta
2022-08-28 14:07:14 +08:00
Peng
71d4e0f9e6 Update README.md 2022-08-28 06:12:13 +08:00
Peng
580b4534e0 Update README.md 2022-08-28 06:12:03 +08:00
Peng
fc835e09c6 Update README.md 2022-08-28 06:10:24 +08:00
Peng
c6e782a637 Update README.md 2022-08-28 06:09:59 +08:00
Peng
1ddfbfc833 Update README.md 2022-08-28 06:09:17 +08:00
EricZeng
dbf637fe0f Merge pull request #515 from didi/master
merge master
2022-08-26 18:45:25 +08:00
EricZeng
110e129622 Merge pull request #514 from didi/dev
Dev
2022-08-26 18:44:36 +08:00
zengqiao
677e9d1b54 Helm调整 2022-08-26 18:43:50 +08:00
EricZeng
ad2adb905e Merge pull request #513 from GraceWalk/master
knowdesign 依赖版本锁定为 1.3.7
2022-08-26 17:44:28 +08:00
GraceWalk
5e9de7ac14 knowdesign 依赖版本锁定为 1.3.7 2022-08-26 17:42:09 +08:00
EricZeng
c63fb8380c Merge pull request #512 from didi/master
合并主分支
2022-08-26 17:23:37 +08:00
EricZeng
2d39acc224 Merge pull request #511 from didi/dev_v3.0.0
删除无关脚本
2022-08-26 11:30:56 +08:00
zengqiao
e68358e05f 删除无关脚本 2022-08-26 11:29:30 +08:00
EricZeng
a96f10edf0 Merge pull request #510 from didi/dev_v3.0.0
v3.0.0-beta版本
2022-08-26 11:24:36 +08:00
zengqiao
f03d94935b 均衡设置修复 2022-08-26 10:51:37 +08:00
zengqiao
9c1320cd95 SQL调整 2022-08-26 10:51:12 +08:00
zengqiao
4f2ae588a5 补充3.0.0-beta的信息 2022-08-26 10:50:54 +08:00
zengqiao
eff51034b7 调整链接及文案 2022-08-26 10:49:17 +08:00
zengqiao
18832dc448 前端调整 2022-08-25 20:32:49 +08:00
zengqiao
5262ae8907 采样调整 2022-08-25 20:30:15 +08:00
zengqiao
7f251679fa Job模块创建任务失败修复 2022-08-25 19:50:50 +08:00
zengqiao
5f5920b427 分页工具类优化 2022-08-25 19:50:08 +08:00
zengqiao
65a16d058a 增加返回Kafka内部Topic 2022-08-25 19:49:47 +08:00
zengqiao
a73484d23a 免登录接口调用说明修改 2022-08-25 19:48:58 +08:00
zengqiao
47887a20c6 前端优化 2022-08-24 21:29:57 +08:00
zengqiao
9465c6f198 删除无效说明 2022-08-24 21:28:24 +08:00
zengqiao
c09872c8c2 调整默认配置 2022-08-24 21:28:04 +08:00
zengqiao
b0501cc80d 修复db字段缺少默认值导致写入数据失败问题 2022-08-24 21:27:43 +08:00
zengqiao
f0792db6b3 兼容0.10.2版本offset信息的获取 2022-08-24 21:27:04 +08:00
zengqiao
e1514c901b 同步代码 2022-08-23 19:01:53 +08:00
zengqiao
e90c5003ae 官网路径调整 2022-08-23 18:30:05 +08:00
zengqiao
92a0d5d52c README调整 2022-08-22 18:40:23 +08:00
zengqiao
8912cb5323 README调整 2022-08-22 18:30:45 +08:00
zengqiao
d008c19149 文档更新 & 问题修复 2022-08-22 18:27:53 +08:00
EricZeng
e844b6444a Merge pull request #509 from Liubey/dev_v3.0.0
fix wrong link in readme
2022-08-19 10:14:50 +08:00
Liubey
02606cdce2 fix wrong link 2022-08-19 09:34:03 +08:00
Liubey
0081720f0e fix wrong link
Signed-off-by: Liubey <liubey1214@gmail.com>
2022-08-19 09:30:16 +08:00
Peng
cca1e92868 Update README.md 2022-08-18 18:03:00 +08:00
Peng
69b774a074 Update README.md 2022-08-18 17:51:41 +08:00
zengqiao
5656b03fb4 合并主分支 2022-08-18 17:42:18 +08:00
Peng
02d0dcbb7f Update README.md 2022-08-18 17:33:01 +08:00
zengqiao
7b2e06df12 change license 2022-08-18 17:32:07 +08:00
zengqiao
4259ae63d7 change license 2022-08-18 17:25:02 +08:00
zengqiao
d7b11803bc bump version to 3.0.0-beta 2022-08-18 17:21:43 +08:00
Peng
fed298a6d4 Update README.md 2022-08-18 17:10:00 +08:00
zengqiao
51832385b1 初始化3.0.0版本 2022-08-18 17:04:05 +08:00
Peng
462303fca0 Update README.md 2022-08-11 14:49:04 +08:00
Peng
4405703e42 Update README.md 2022-08-09 11:11:03 +08:00
Peng
23e398e121 Update README.md 2022-08-09 10:05:24 +08:00
Peng
b17bb89d04 Update README.md 2022-08-09 09:56:35 +08:00
Peng
5590cebf8f Update README.md 2022-08-09 09:54:44 +08:00
Peng
1fa043f09d Update README.md 2022-08-09 09:52:30 +08:00
Peng
3bd0af1451 Update README.md 2022-08-09 09:49:09 +08:00
Peng
1545962745 Update README.md
添加star趋势
2022-07-29 16:05:16 +08:00
EricZeng
d032571681 Merge pull request #503 from didi/dev
补充FutureUtil类
2022-07-06 16:21:24 +08:00
zengqiao
33fb0acc7e 补充FutureUtil类 2022-07-06 15:18:53 +08:00
EricZeng
1ec68a91e2 Merge pull request #499 from gzldc/master
关于logback的版本漏洞修复 #488
2022-07-01 08:57:11 +08:00
shishuai
a23c113a46 关于logback的版本漏洞修复 #488 2022-06-29 22:06:33 +08:00
Peng
371ae2c0a5 Update README.md 2022-06-28 10:19:18 +08:00
Peng
8f8f6ffa27 Update README.md 2022-06-28 10:18:03 +08:00
EricZeng
475fe0d91f Merge pull request #496 from didi/dev
删除application中无效的版本信息配置
2022-06-23 10:31:53 +08:00
zengqiao
3d74e60d03 删除application中无效的版本信息配置 2022-06-23 10:31:07 +08:00
EricZeng
83ac83bb28 Merge pull request #495 from didi/master
合并主分支
2022-06-23 10:29:22 +08:00
EricZeng
8478fb857c Merge pull request #494 from didi/dev
1、打包时自动生成版本信息及git提交信息;2、优化swagger对版本信息的获取;
2022-06-23 10:23:10 +08:00
zengqiao
7074bdaa9f 1、打包时自动生成版本信息及git提交信息;2、优化swagger对版本信息的获取 2022-06-23 10:17:36 +08:00
EricZeng
58164294cc Update README.md 2022-03-17 10:02:10 +08:00
EricZeng
7c0e9df156 Merge pull request #479 from didi/dev
集成测试&单元测试补充
2022-03-15 13:49:33 +08:00
EricZeng
bd62212ecb Merge pull request #472 from didi/dev_v2.5.0_addtest
LogiKM增加单元测试和集成测试
2022-03-07 17:29:20 +08:00
EricZeng
2292039b42 Merge pull request #474 from houxiufeng/dev_v2.5.0_addtest
modify AbstractSingleSignOnTest error
2022-03-07 17:28:39 +08:00
houxiufeng
73f8da8d5a modify AbstractSingleSignOnTest error 2022-03-07 17:10:15 +08:00
EricZeng
e51dbe0ca7 Merge pull request #473 from didi/dev
Dev
2022-03-07 14:49:52 +08:00
xuguang
482a375e31 Merge branch 'v2.5.0' of github.com:didi/LogiKM into dev_v2.5.0_addtest 2022-03-04 16:13:54 +08:00
xuguang
689c5ce455 增加单元测试和集成测试文档 & 问题修改 2022-03-04 16:04:36 +08:00
EricZeng
734a020ecc Merge pull request #470 from didi/dev
修复删除指标时,数据越界问题
2022-03-03 19:04:03 +08:00
zengqiao
44d537f78c 修复删除指标时,数据越界问题 2022-03-03 11:26:50 +08:00
zengqiao
b4c60eb910 增加过滤掉Broker的连接信息时,增加请求类型的判断 2022-02-28 12:07:50 +08:00
xuguang
e120b32375 剩余controller接口集成测试 2022-02-25 17:04:46 +08:00
xuguang
de54966d30 NormalJmxController注释修改 2022-02-24 16:20:37 +08:00
xuguang
39a6302c18 部分controller接口集成测试 2022-02-21 10:43:54 +08:00
xuguang
05ceeea4b0 bugfix: LogicalClusterDTO和RegionDTO校验参数冗余 & RdLogicalClusterController解释修正 2022-02-21 10:41:50 +08:00
EricZeng
9f8e3373a8 Merge pull request #465 from hailanxin/master
增加jmx连接失败的一个情况和解决方法
2022-02-17 21:08:21 +08:00
hailanxin
42521cbae4 Update connect_jmx_failed.md 2022-02-17 14:02:43 +08:00
hailanxin
b23c35197e 增加jmx连接失败的一个情况和解决方法 2022-02-17 14:02:13 +08:00
EricZeng
70f28d9ac4 Merge pull request #461 from zzzhangqi/rainbond
add rainbond installation
2022-02-16 10:18:15 +08:00
zhangqi
912d73d98a add rainbond installation 2022-02-15 18:34:49 +08:00
EricZeng
2a720fce6f Merge pull request #451 from zzzhangqi/master
Support docker source code construction
2022-02-15 18:05:46 +08:00
zhangqi
e4534c359f Support docker source code construction 2022-02-15 10:43:11 +08:00
zengqiao
b91bec15f2 bump version to v2.6.1 2022-01-26 15:33:08 +08:00
EricZeng
67ad5cacb7 Merge pull request #453 from didi/dev
1.设置版本为2.6.0; 2.补充v2.6.0的release notes
2022-01-24 13:32:13 +08:00
zengqiao
b4a739476a 补充v2.6.0的release notes 2022-01-24 13:30:48 +08:00
zengqiao
a7bf2085db 设置版本为2.6.0 2022-01-24 13:30:05 +08:00
zengqiao
c3802cf48b 去除@project.version@说明 2022-01-24 13:27:10 +08:00
EricZeng
54711c4491 Merge pull request #452 from didi/dev
允许配置文件中不进行配置
2022-01-24 13:22:48 +08:00
zengqiao
fcb52a69c0 允许配置文件中不进行配置 2022-01-24 13:16:30 +08:00
zengqiao
1b632f9754 调整文章顺序 2022-01-22 13:24:07 +08:00
EricZeng
73d7a0ecdc Merge pull request #450 from didi/dev
调整Task模块日志及Api请求统计日志的输出
2022-01-21 19:25:01 +08:00
EricZeng
08943593b3 Merge pull request #449 from kingdomrushing/dev_v2.6.0
增加对定时任务的说明
2022-01-21 15:27:17 +08:00
xuguang
c949a88f20 周期任务说明文档内容补充 2022-01-21 13:59:46 +08:00
xuguang
a49c11f655 统一定时任务cron格式 2022-01-21 13:14:31 +08:00
xuguang
a66aed4a88 增加对定时任务的注释和说明文档 2022-01-21 13:12:58 +08:00
xuguang
0045c953a0 Merge branch 'dev' of github.com:kingdomrushing/LogiKM into dev_v2.6.0 2022-01-21 10:59:38 +08:00
xuguang
fdce41b451 增加对定时任务的说明 2022-01-21 10:50:29 +08:00
xuguang
4d5e4d0f00 去掉单元测试敏感信息 2022-01-21 10:08:15 +08:00
didi
82c9b6481e 真实环境配置定义在配置文件中 2022-01-20 22:33:52 +08:00
EricZeng
053d4dcb18 Merge pull request #447 from didi/dev
补充KCM使用文档
2022-01-20 14:29:18 +08:00
zengqiao
e1b2c442aa 调整Task模块日志及Api请求统计日志的输出 2022-01-20 14:28:39 +08:00
zengqiao
0ed8ba8ca4 补充KCM使用文档 2022-01-20 11:47:55 +08:00
xuguang
f195847c68 Merge branch 'dev_v2.5.0_addtest' of github.com:didi/LogiKM into dev_v2.5.0_addtest 2022-01-20 10:24:10 +08:00
xuguang
5beb13b17e NormalAppControllerTest,OpAuthorityControllerTest 2022-01-20 10:22:20 +08:00
EricZeng
7d9ec05062 Merge pull request #445 from didi/dev
优化corn表达式解析失败后退出无任何日志提示问题
2022-01-20 10:22:16 +08:00
xuguang
fc604a9eaf 集成测试:物理集群的增删改查 2022-01-20 10:15:42 +08:00
zengqiao
4f3c1ad9b6 优化corn表达式解析失败后退出无任何日志提示问题 2022-01-19 20:23:18 +08:00
EricZeng
6d45ed586c Merge pull request #444 from didi/dev
1. 完善搜索用户时可以显示用户的其他元信息(完善chineseName和department); 2. 升级至v2.6.0说明
2022-01-19 15:25:11 +08:00
didi
1afb633b4f Merge branch 'dev_v2.5.0_addtest' of github.com:didi/LogiKM into dev_v2.5.0_addtest 2022-01-18 17:08:49 +08:00
didi
34d9f9174b 所有单测重新测试 2022-01-18 17:07:21 +08:00
zengqiao
3b0c208eff 补充v2.6.0升级说明及修复新增account提示mysql错误问题 2022-01-18 15:48:05 +08:00
EricZeng
05022f8db4 Merge pull request #442 from Huyueeer/devLookUserName
LDAP忽略大小写 & 认证携带必要元信息(姓名、部门、邮箱)
2022-01-18 15:26:32 +08:00
EricZeng
3336de457a Merge pull request #441 from didi/dev
bump springboot version to 2.1.18 and ignore springframework version …
2022-01-18 14:42:18 +08:00
EricZeng
10a27bc29c Merge pull request #443 from lucasun/master
V2.6.0 FE
2022-01-18 14:24:17 +08:00
Hu.Yue
542e5d3c2d Merge branch 'didi-dev' into devLookUserName 2022-01-18 14:09:05 +08:00
lucasun
7372617b14 Merge branch 'didi:master' into master 2022-01-18 14:07:44 +08:00
Hu.Yue
89735a130b Merge branch 'dev' of https://github.com/didi/LogiKM into didi-dev 2022-01-18 14:07:34 +08:00
孙超
859cf74bd6 2.6.0问题修复 2022-01-18 14:05:32 +08:00
zengqiao
e2744ab399 bump version to 2.6.0 2022-01-18 14:00:20 +08:00
Hu.Yue
16bd065098 Merge branch 'dev' of https://github.com/didi/LogiKM into didi-dev 2022-01-18 13:58:16 +08:00
zengqiao
71c52e6dd7 bump springboot version to 2.1.18 and ignore springframework version config 2022-01-17 21:18:06 +08:00
EricZeng
a7f8c3ced3 Merge pull request #440 from didi/dev
1.安装部署脚本LogiKM可配置; 2.增加网关接口及第三方接口可直接调用的开关;
2022-01-17 21:08:55 +08:00
zengqiao
f3f0432c65 1.安装部署脚本LogiKM可配置; 2.增加网关接口及第三方接口可直接调用的开关; 2022-01-17 20:41:53 +08:00
EricZeng
426ba2d150 Merge pull request #439 from didi/dev
1、修复swagger抛出的NumberFormatException问题; 2、Swagger提示版本和POM中版本通过配置保持一致; 3、梳理Task模块任务-BrokerMetrics任务梳理;
2022-01-17 20:39:10 +08:00
zengqiao
2790099efa 梳理Task模块任务-BrokerMetrics任务梳理 2022-01-17 15:28:36 +08:00
zengqiao
f6ba8bc95e Swagger提示版本和POM中版本通过配置保持一致 2022-01-17 13:17:07 +08:00
zengqiao
d6181522c0 修复swagger抛出的NumberFormatException问题 2022-01-17 11:42:30 +08:00
EricZeng
04cf071ca6 Merge pull request #437 from didi/dev
增加简单回退工具类,增加Jmx连接失败回退功能机制,优化Jmx连接失败日志
2022-01-15 13:56:44 +08:00
zengqiao
e4371b5d02 修正注释 2022-01-14 14:28:45 +08:00
zengqiao
52c52b2a0d 增加简单回退工具类,增加Jmx连接失败回退功能机制,优化Jmx连接失败日志 2022-01-14 14:24:04 +08:00
EricZeng
8f40f10575 Merge pull request #436 from didi/dev
1. 增加对BrokerMetadata中endpoints为internal|External方式的解析; 2. 增加线程池、客户端池可配置; 3. fix config incorrectly comment
2022-01-13 16:28:34 +08:00
zengqiao
fe0f6fcd0b fix config incorrectly comment 2022-01-13 16:02:33 +08:00
EricZeng
31b1ad8bb4 Merge pull request #435 from kingdomrushing/dev_v2.6.0
增加线程池、客户端池可配置 & 增加对BrokerMetadata中endpoints为internal|External方式的解析
2022-01-13 15:51:54 +08:00
xuguang
373680d854 线程池 & BrokerMetadata 问题修复 2022-01-13 15:39:39 +08:00
xuguang
9e3bc80495 线程池 & BrokerMetadata 问题修复 2022-01-13 15:35:11 +08:00
xuguang
52ccaeffd5 解决依赖冲突 2022-01-13 11:48:43 +08:00
kingdomrushing
18136c12fd Merge branch 'didi:master' into dev_v2.6.0 2022-01-13 11:41:20 +08:00
EricZeng
dec3f9e75e Merge pull request #429 from didi/dev
ldap config add default value
2022-01-13 11:40:46 +08:00
EricZeng
ccc0ee4d18 Merge pull request #432 from didi/master
merge master to dev
2022-01-13 11:40:17 +08:00
kingdomrushing
69e9708080 Merge branch 'dev' into dev_v2.6.0 2022-01-13 11:39:50 +08:00
lucasun
5944ba099a Merge pull request #431 from lucasun/hotfix/2.5.0_fe
修复图片名称大小写问题
2022-01-13 11:17:25 +08:00
孙超
ada2718b5e 修复图片名称大小写问题 2022-01-13 11:13:26 +08:00
lucasun
1f87bd63e7 Merge pull request #416 from potaaaaaato/master
update echarts from v4 to v5
2022-01-13 10:55:34 +08:00
xuguang
c0f3259cf6 增加线程池、客户端池可配置 2022-01-12 19:56:37 +08:00
EricZeng
e1d5749a40 Merge pull request #428 from didi/dev
Dockerfile mv application.yml to fix start failed problem
2022-01-12 18:14:20 +08:00
zengqiao
a8d7eb27d9 ldap config add default value 2022-01-12 18:03:16 +08:00
zengqiao
1eecdf3829 Dockerfile mv application.yml to fix start failed problem 2022-01-12 17:45:40 +08:00
EricZeng
be8b345889 Merge pull request #427 from didi/dev
del unused jmx_prometheus_javaagent-0.14.0.jar
2022-01-12 17:40:48 +08:00
zengqiao
074da389b3 del unused jmx_prometheus_javaagent-0.14.0.jar 2022-01-12 17:32:13 +08:00
xuguang
4df2dc09fe 增加对BrokerMetadata中endpoints为internal|External方式的解析 2022-01-12 16:15:46 +08:00
EricZeng
e8d42ba074 Merge pull request #425 from didi/dev
1. optimize reeassign task-name; 2. ignore read kafka-controller data when znode not exist; 3.add region created event and handle it to cal region capacity immediately;
2022-01-12 11:44:22 +08:00
zengqiao
c036483680 add region created event and handle it to cal region capacity immediately 2022-01-11 17:49:17 +08:00
zengqiao
2818584db6 ignore read kafka-controller data when znode not exist 2022-01-11 17:19:14 +08:00
zengqiao
37585f760d optimize reeassign task-name 2022-01-11 16:57:24 +08:00
EricZeng
f5477a03a1 Merge pull request #364 from Huyueeer/devConfig
平台配置EXPIRED_TOPIC_CONFIG新增正则过滤Topic
2022-01-11 16:33:21 +08:00
EricZeng
50388425b2 Merge pull request #424 from didi/dev
1.add lombok; 2.support change delete metrics rate;
2022-01-11 16:29:43 +08:00
zengqiao
725c59eab0 support change delete metrics rate 2022-01-11 16:03:22 +08:00
zengqiao
7bf1de29a4 add lombok 2022-01-11 14:04:48 +08:00
EricZeng
d90c3fc7dd Merge pull request #423 from didi/dev
1. bump swagger version; 2. fix NPE when flush logical cluster and physical cluster not in cache or not exist; 3. JmxConnectorWrap's log add cluster and broker info;
2022-01-11 13:16:46 +08:00
zengqiao
80785ce072 JmxConnectorWrap's log add cluster and broker info 2022-01-11 11:45:28 +08:00
zengqiao
44ea896de8 fix NPE when flush logical cluster and physical cluster not in cache or not exist 2022-01-11 11:37:23 +08:00
zengqiao
d30cb8a0f0 bump swagger version 2022-01-11 11:24:43 +08:00
EricZeng
6c7b333b34 Merge pull request #422 from didi/dev
网关增加配置及修改配置时,version不变化问题修复
2022-01-10 10:26:49 +08:00
石臻臻的杂货铺
6d34a00e77 Update README.md 2022-01-08 15:02:07 +08:00
xuguang
1f353e10ce application.yml修改 2022-01-07 15:58:12 +08:00
zengqiao
4e10f8d1c5 网关增加配置及修改配置时,version不变化问题修复 2022-01-07 15:35:14 +08:00
EricZeng
a22cd853fc Merge pull request #421 from didi/dev
修复批量往DB写入空指标数组时报SQL语法异常的问题
2022-01-07 14:16:07 +08:00
zengqiao
354e0d6a87 修复批量往DB写入空指标数组时报SQL语法异常的问题 2022-01-07 13:55:42 +08:00
EricZeng
dfabe28645 Merge pull request #420 from didi/master
合并主分支
2022-01-07 13:38:10 +08:00
EricZeng
fce230da48 Merge pull request #414 from didi/dev
设置Log4j 2的版本为2.16.0
2022-01-07 13:36:50 +08:00
didi
055ba9bda6 Merge branch 'dev_v2.5.0_addtest' of github.com:didi/LogiKM into dev_v2.5.0_addtest
 Conflicts:
	kafka-manager-core/src/test/java/com/xiaojukeji/kafka/manager/service/service/ExpertServiceTest.java
2022-01-07 11:50:15 +08:00
didi
ec19c3b4dd monitor、openapi、account模块下的单元测试 2022-01-07 11:43:31 +08:00
xuguang
37aa526404 单元测试:ClusterHostTaskServiceTest,ClusterRoleTaskServiceTest,ClusterTaskServiceTest,KafkaFileServiceTest,TopicCommandsTest,TopicReassignUtilsTest 2022-01-06 20:00:25 +08:00
xuguang
86c1faa40f bugfix: Result类问题修改 2022-01-06 11:33:17 +08:00
xuguang
8dcf15d0f9 kafka-manager-bpm包单元测试的编写 & bugfix 2022-01-04 17:29:28 +08:00
EricZeng
6835e1e680 Merge pull request #419 from ZHAOYINRUI/patch-6
更新视频教程地址
2022-01-04 11:07:52 +08:00
ZHAOYINRUI
d8f89b8f67 更新视频教程地址 2022-01-04 11:06:51 +08:00
eilenexuzhe
ec28eba781 feat: move webpack-dev-server to scripts 2021-12-27 17:12:51 +08:00
eilenexuzhe
5ef8fff5bc feat: update echarts v4 to v5 2021-12-27 17:12:02 +08:00
xuguang
4f317b76fa SpringTool.getUserName()方法中获取requestAttributes可能为null, 增加为null判断 2021-12-27 16:35:22 +08:00
didi
61672637dc Merge branch 'dev_v2.5.0_addtest' of github.com:didi/LogiKM into dev_v2.5.0_addtest 2021-12-27 14:56:48 +08:00
didi
ecf6e8f664 ConfigService,OperateRecordService,RegionService,ThrottleService,TopicExpiredService,TopicManagerService接口下的单元测试方法 2021-12-27 14:55:35 +08:00
xuguang
4115975320 kafka-manager-account, kafka-manager-bpm, kafka-manager-kcm, kafka-manager-monitor单元测试模块添加 2021-12-27 10:28:08 +08:00
didi
21904a8609 `TopicManagerServiceImpl的addAuthority中使用的是getId,应该是getAppId 2021-12-24 14:41:39 +08:00
Peng
10b0a3dabb Update README.md 2021-12-24 11:54:52 +08:00
xuguang
b2091e9aed 单元测试:AnalysisServiceTest && ConsumerServiceTest && JmxServiceTest &&
LogicalClusterServiceTest && ReassignServiceTest && TopicServiceTest
2021-12-23 18:17:47 +08:00
xuguang
f2cb5bd77c bugfix: TopicServiceImpl && JmxServiceImpl && ConsumerService && ConsumerServiceImpl 2021-12-23 18:15:40 +08:00
xuguang
19c61c52e6 bugfix: TopicService && TopicServiceImpl && ZookeeperServiceImpl 2021-12-22 16:04:06 +08:00
didi
b327359183 TopicManagerServiceImpl的modifyTopicByOp没有return ResultStatus.SUCCESS; 2021-12-21 18:06:23 +08:00
xuguang
9e9bb72e17 BrokerServiceTest && KafkaBillService && LogicalClusterServiceTest && AbstractAllocateQuotaStrategy && AbstractHealthScoreStrategy 单元测试 2021-12-20 10:26:43 +08:00
zengqiao
a23907e009 开启前端包 2021-12-17 15:45:45 +08:00
xuguang
ad131f5a2c bugfix: DidiHealthScoreStrategy.calTopicHealthScore计算topic健康分时,获取topic的brokerId异常 2021-12-17 14:41:04 +08:00
zengqiao
dbeae4ca68 设置log4j2版本为2.16.0以修复相关漏洞 2021-12-17 11:44:50 +08:00
zengqiao
0fb0e94848 Merge branch 'master' into dev 2021-12-16 22:29:45 +08:00
石臻臻的杂货铺
95d2a82d35 Merge pull request #412 from didi/v2.5.0
升级到2.5版本
2021-12-16 18:34:30 +08:00
xuguang
39cccd568e DiDiHealthScoreStrategy类中某些变量开头字母改成小写 2021-12-16 14:17:45 +08:00
didi
19b7f6ad8c RegionService下的updateRegion的重载方法含义错误;应该是根据regionId更新region,参数应该是regionId,不是clusterId 2021-12-14 18:32:14 +08:00
xuguang
41c000cf47 AuthorityServiceTest && SecurityServiceTest && TopicReportServiceTest && ClusterServiceTest && ZookeeperServiceTest单元测试 2021-12-14 18:30:12 +08:00
xuguang
1b8ea61e87 openTopicJmx方法的参数jmxSwitch需要判空 2021-12-13 18:16:23 +08:00
xuguang
4538593236 实现core包下TopicReportService接口单元测试 & TopicReportDao.xml中字段和关键字错误修改 2021-12-08 15:50:53 +08:00
xuguang
8086ef355b 实现core包下AppService,AuthorityService,QuotaService接口单元测试 & TopicQuotaData中bug修改 2021-12-07 14:08:09 +08:00
xuguang
60d038fe46 实现core包下AppService接口单元测试 2021-12-06 14:46:11 +08:00
huqidong
ff0f4463be Logi-KM testng 测试环境搭建 & springboot 集成 & mock 测试用例编写. 2021-12-03 18:04:20 +08:00
EricZeng
820571d993 Merge pull request #408 from didi/master
merge master
2021-12-02 19:39:41 +08:00
huyueeer
fffc0c3add 完善搜索用户时可以显示用户的其他元信息(完善chineseName和department) 2021-08-12 15:28:23 +08:00
huyueeer
022f9eb551 更新EXPIRED_TOPIC_CONFIG文档描述 2021-08-06 16:00:16 +08:00
huyueeer
6e7b82cfcb 平台配置EXPIRED_TOPIC_CONFIG新增正则过滤Topic 2021-08-06 14:52:22 +08:00
huyueeer
b5fb24b360 本地认证或LDAP认证支持携带‘姓名’、‘部门’、‘邮箱’等用户元信息 2021-08-06 11:40:24 +08:00
huyueeer
b77345222c LDAP认证忽略大小写,修正判断顺序,相同LDAP用户反复REPLACE 2021-08-05 11:17:38 +08:00
huyueeer
793e81406e LDAP认证忽略大小写,修正从LDAP服务器返回值设置Username 2021-08-04 16:23:57 +08:00
huyueeer
cef1ec95d2 LDAP验证忽略账户大小写 2021-08-04 14:14:53 +08:00
1210 changed files with 917 additions and 104927 deletions

51
.github/ISSUE_TEMPLATE/bug_report.md vendored Normal file
View File

@@ -0,0 +1,51 @@
---
name: 报告Bug
about: 报告KnowStreaming的相关Bug
title: ''
labels: bug
assignees: ''
---
- [ ] 我已经在 [issues](https://github.com/didi/KnowStreaming/issues) 搜索过相关问题了,并没有重复的。
你是否希望来认领这个Bug。
「 Y / N 」
### 环境信息
* KnowStreaming version : <font size=4 color =red> xxx </font>
* Operating System version : <font size=4 color =red> xxx </font>
* Java version : <font size=4 color =red> xxx </font>
### 重现该问题的步骤
1. xxx
2. xxx
3. xxx
### 预期结果
<!-- 写下应该出现的预期结果?-->
### 实际结果
<!-- 实际发生了什么? -->
---
如果有异常请附上异常Trace:
```
Just put your stack trace here!
```

8
.github/ISSUE_TEMPLATE/config.yml vendored Normal file
View File

@@ -0,0 +1,8 @@
blank_issues_enabled: true
contact_links:
- name: 讨论问题
url: https://github.com/didi/KnowStreaming/discussions/new
about: 发起问题、讨论 等等
- name: KnowStreaming官网
url: https://knowstreaming.com/
about: KnowStreaming website

View File

@@ -0,0 +1,26 @@
---
name: 优化建议
about: 相关功能优化建议
title: ''
labels: Optimization Suggestions
assignees: ''
---
- [ ] 我已经在 [issues](https://github.com/didi/KnowStreaming/issues) 搜索过相关问题了,并没有重复的。
你是否希望来认领这个优化建议。
「 Y / N 」
### 环境信息
* KnowStreaming version : <font size=4 color =red> xxx </font>
* Operating System version : <font size=4 color =red> xxx </font>
* Java version : <font size=4 color =red> xxx </font>
### 需要优化的功能点
### 建议如何优化

View File

@@ -0,0 +1,20 @@
---
name: 提议新功能/需求
about: 给KnowStreaming提一个功能需求
title: ''
labels: feature
assignees: ''
---
- [ ] 我在 [issues](https://github.com/didi/KnowStreaming/issues) 中并未搜索到与此相关的功能需求。
- [ ] 我在 [release note](https://github.com/didi/KnowStreaming/releases) 已经发布的版本中并没有搜到相关功能.
你是否希望来认领这个Feature。
「 Y / N 」
## 这里描述需求
<!--请尽可能的描述清楚您的需求 -->

12
.github/ISSUE_TEMPLATE/question.md vendored Normal file
View File

@@ -0,0 +1,12 @@
---
name: 提个问题
about: 问KnowStreaming相关问题
title: ''
labels: question
assignees: ''
---
- [ ] 我已经在 [issues](https://github.com/didi/KnowStreaming/issues) 搜索过相关问题了,并没有重复的。
## 在这里提出你的问题

23
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View File

@@ -0,0 +1,23 @@
请不要在没有先创建Issue的情况下创建Pull Request。
## 变更的目的是什么
XXXXX
## 简短的更新日志
XX
## 验证这一变化
XXXX
请遵循此清单,以帮助我们快速轻松地整合您的贡献:
* [ ] 一个 PRPull Request的简写只解决一个问题禁止一个 PR 解决多个问题;
* [ ] 确保 PR 有对应的 Issue通常在您开始处理之前创建除非是书写错误之类的琐碎更改不需要 Issue
* [ ] 格式化 PR 及 Commit-Log 的标题及内容,例如 #861 。PSCommit-Log 需要在 Git Commit 代码时进行填写,在 GitHub 上修改不了;
* [ ] 编写足够详细的 PR 描述,以了解 PR 的作用、方式和原因;
* [ ] 编写必要的单元测试来验证您的逻辑更正。如果提交了新功能或重大更改,请记住在 test 模块中添加 integration-test
* [ ] 确保编译通过,集成测试通过;

229
.gitignore vendored
View File

@@ -1,113 +1,116 @@
### Intellij ### ### Intellij ###
# Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and Webstorm # Covers JetBrains IDEs: IntelliJ, RubyMine, PhpStorm, AppCode, PyCharm, CLion, Android Studio and Webstorm
*.iml *.iml
## Directory-based project format: ## Directory-based project format:
.idea/ .idea/
# if you remove the above rule, at least ignore the following: # if you remove the above rule, at least ignore the following:
# User-specific stuff: # User-specific stuff:
# .idea/workspace.xml # .idea/workspace.xml
# .idea/tasks.xml # .idea/tasks.xml
# .idea/dictionaries # .idea/dictionaries
# .idea/shelf # .idea/shelf
# Sensitive or high-churn files: # Sensitive or high-churn files:
.idea/dataSources.ids .idea/dataSources.ids
.idea/dataSources.xml .idea/dataSources.xml
.idea/sqlDataSources.xml .idea/sqlDataSources.xml
.idea/dynamic.xml .idea/dynamic.xml
.idea/uiDesigner.xml .idea/uiDesigner.xml
# Mongo Explorer plugin: # Mongo Explorer plugin:
.idea/mongoSettings.xml .idea/mongoSettings.xml
## File-based project format: ## File-based project format:
*.ipr *.ipr
*.iws *.iws
## Plugin-specific files: ## Plugin-specific files:
# IntelliJ # IntelliJ
/out/ /out/
# mpeltonen/sbt-idea plugin # mpeltonen/sbt-idea plugin
.idea_modules/ .idea_modules/
# JIRA plugin # JIRA plugin
atlassian-ide-plugin.xml atlassian-ide-plugin.xml
# Crashlytics plugin (for Android Studio and IntelliJ) # Crashlytics plugin (for Android Studio and IntelliJ)
com_crashlytics_export_strings.xml com_crashlytics_export_strings.xml
crashlytics.properties crashlytics.properties
crashlytics-build.properties crashlytics-build.properties
fabric.properties fabric.properties
### Java ### ### Java ###
*.class *.class
# Mobile Tools for Java (J2ME) # Mobile Tools for Java (J2ME)
.mtj.tmp/ .mtj.tmp/
# Package Files # # Package Files #
*.jar *.jar
*.war *.war
*.ear *.ear
*.tar.gz *.tar.gz
# virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml # virtual machine crash logs, see http://www.java.com/en/download/help/error_hotspot.xml
hs_err_pid* hs_err_pid*
### OSX ### ### OSX ###
.DS_Store .DS_Store
.AppleDouble .AppleDouble
.LSOverride .LSOverride
# Icon must end with two \r # Icon must end with two \r
Icon Icon
# Thumbnails # Thumbnails
._* ._*
# Files that might appear in the root of a volume # Files that might appear in the root of a volume
.DocumentRevisions-V100 .DocumentRevisions-V100
.fseventsd .fseventsd
.Spotlight-V100 .Spotlight-V100
.TemporaryItems .TemporaryItems
.Trashes .Trashes
.VolumeIcon.icns .VolumeIcon.icns
# Directories potentially created on remote AFP share # Directories potentially created on remote AFP share
.AppleDB .AppleDB
.AppleDesktop .AppleDesktop
Network Trash Folder Network Trash Folder
Temporary Items Temporary Items
.apdisk .apdisk
/target /target
target/ target/
*.log *.log
*.log.* *.log.*
*.bak *.bak
*.vscode *.vscode
*/.vscode/* */.vscode/*
*/.vscode */.vscode
*/velocity.log* */velocity.log*
*/*.log */*.log
*/*.log.* */*.log.*
node_modules/ node_modules/
node_modules/* node_modules/*
workspace.xml workspace.xml
/output/* /output/*
.gitversion .gitversion
node_modules/* out/*
out/* dist/
dist/ dist/*
dist/* km-rest/src/main/resources/templates/
kafka-manager-web/src/main/resources/templates/ *dependency-reduced-pom*
.DS_Store #filter flattened xml
*/.flattened-pom.xml
.flattened-pom.xml
*/*/.flattened-pom.xml

View File

@@ -1,28 +0,0 @@
# Contribution Guideline
Thanks for considering to contribute this project. All issues and pull requests are highly appreciated.
## Pull Requests
Before sending pull request to this project, please read and follow guidelines below.
1. Branch: We only accept pull request on `dev` branch.
2. Coding style: Follow the coding style used in kafka-manager.
3. Commit message: Use English and be aware of your spell.
4. Test: Make sure to test your code.
Add device mode, API version, related log, screenshots and other related information in your pull request if possible.
NOTE: We assume all your contribution can be licensed under the [Apache License 2.0](LICENSE).
## Issues
We love clearly described issues. :)
Following information can help us to resolve the issue faster.
* Device mode and hardware information.
* API version.
* Logs.
* Screenshots.
* Steps to reproduce the issue.

BIN
KS-PRD-3.0-beta1.docx Normal file

Binary file not shown.

BIN
KS-PRD-3.0-beta2.docx Normal file

Binary file not shown.

BIN
KS-PRD-3.1-ZK.docx Normal file

Binary file not shown.

BIN
KS-PRD-3.2-Connect.docx Normal file

Binary file not shown.

BIN
KS-PRD-3.3-MM2.docx Normal file

Binary file not shown.

1094
LICENSE

File diff suppressed because it is too large Load Diff

102
README.md
View File

@@ -1,102 +0,0 @@
---
![logikm_logo](https://user-images.githubusercontent.com/71620349/125024570-9e07a100-e0b3-11eb-8ebc-22e73e056771.png)
**一站式`Apache Kafka`集群指标监控与运维管控平台**
阅读本README文档您可以了解到滴滴Logi-KafkaManager的用户群体、产品定位等信息并通过体验地址快速体验Kafka集群指标监控与运维管控的全流程。
## 1 产品简介
滴滴Logi-KafkaManager脱胎于滴滴内部多年的Kafka运营实践经验是面向Kafka用户、Kafka运维人员打造的共享多租户Kafka云平台。专注于Kafka运维管控、监控告警、资源治理等核心场景经历过大规模集群、海量大数据的考验。内部满意度高达90%的同时,还与多家知名企业达成商业化合作。
### 1.1 快速体验地址
- 体验地址 http://117.51.150.133:8080 账号密码 admin/admin
### 1.2 体验地图
相比较于同类产品的用户视角单一大多为管理员视角滴滴Logi-KafkaManager建立了基于分角色、多场景视角的体验地图。分别是**用户体验地图、运维体验地图、运营体验地图**
#### 1.2.1 用户体验地图
- 平台租户申请&nbsp;&nbsp;申请应用App作为Kafka中的用户名并用 AppID+password作为身份验证
- 集群资源申请&nbsp;&nbsp;:按需申请、按需使用。可使用平台提供的共享集群,也可为应用申请独立的集群
- Topic&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;可根据应用App创建Topic或者申请其他topic的读写权限
- Topic&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Topic数据采样、调整配额、申请分区等操作
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;基于Topic生产消费各环节耗时统计监控不同分位数性能指标
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;:支持将消费偏移重置至指定时间或指定位置
#### 1.2.2 运维体验地图
- 多版本集群管控&nbsp;&nbsp;:支持从`0.10.2``2.x`版本
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;集群Topic、Broker等多维度历史与实时关键指标查看建立健康分体系
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;划分部分Broker作为Region使用Region定义资源划分单位并按照业务、保障能力区分逻辑集群
- Broker&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;:包括优先副本选举等操作
- Topic&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;:包括创建、查询、扩容、修改属性、迁移、下线等
#### 1.2.3 运营体验地图
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;沉淀资源治理方法。针对Topic分区热点、分区不足等高频常见问题沉淀资源治理方法实现资源治理专家化
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;工单体系。Topic创建、调整配额、申请分区等操作由专业运维人员审批规范资源使用保障平台平稳运行
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;成本控制。Topic资源、集群资源按需申请、按需使用。根据流量核算费用帮助企业建设大数据成本核算体系
### 1.3 核心优势
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;:监控多项核心指标,统计不同分位数据,提供种类丰富的指标监控报表,帮助用户、运维人员快速高效定位问题
- 便&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;按照Region定义集群资源划分单位将逻辑集群根据保障等级划分。在方便资源隔离、提高扩展能力的同时实现对服务端的强管控
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;基于滴滴内部多年运营实践沉淀资源治理方法建立健康分体系。针对Topic分区热点、分区不足等高频常见问题实现资源治理专家化
-&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;:与滴滴夜莺监控告警系统打通,集成监控告警、集群部署、集群升级等能力。形成运维生态,凝练专家服务,使运维更高效
### 1.4 滴滴Logi-KafkaManager架构图
![kafka-manager-arch](https://img-ys011.didistatic.com/static/dicloudpub/do1_xgDHNDLj2ChKxctSuf72)
## 2 相关文档
### 2.1 产品文档
- [滴滴Logi-KafkaManager 安装手册](docs/install_guide/install_guide_cn.md)
- [滴滴Logi-KafkaManager 接入集群](docs/user_guide/add_cluster/add_cluster.md)
- [滴滴Logi-KafkaManager 用户使用手册](docs/user_guide/user_guide_cn.md)
- [滴滴Logi-KafkaManager FAQ](docs/user_guide/faq.md)
### 2.2 社区文章
- [滴滴云官网产品介绍](https://www.didiyun.com/production/logi-KafkaManager.html)
- [7年沉淀之作--滴滴Logi日志服务套件](https://mp.weixin.qq.com/s/-KQp-Qo3WKEOc9wIR2iFnw)
- [滴滴Logi-KafkaManager 一站式Kafka监控与管控平台](https://mp.weixin.qq.com/s/9qSZIkqCnU6u9nLMvOOjIQ)
- [滴滴Logi-KafkaManager 开源之路](https://xie.infoq.cn/article/0223091a99e697412073c0d64)
- [滴滴Logi-KafkaManager 系列视频教程](https://mp.weixin.qq.com/s/9X7gH0tptHPtfjPPSdGO8g)
- [kafka实践十五滴滴开源Kafka管控平台 Logi-KafkaManager研究--A叶子叶来](https://blog.csdn.net/yezonggang/article/details/113106244)
- [kafka的灵魂伴侣Logi-KafkaManager系列文章专栏 --石臻](https://blog.csdn.net/u010634066/category_10977588.html)
## 3 滴滴Logi开源用户交流群
![image](https://user-images.githubusercontent.com/5287750/111266722-e531d800-8665-11eb-9242-3484da5a3099.png)
微信加群添加mike_zhangliang的微信号备注Logi加群或关注公众号 云原生可观测性 回复 "Logi加群"
## 4 知识星球
![image](https://user-images.githubusercontent.com/51046167/140718512-5ab1b336-5c48-46c0-90bd-44b5c7e168c8.png)
✅知识星球首个【Kafka中文社区】内测期免费加入https://z.didi.cn/5gSF9
有问必答~
互动有礼~
1600+群友一起共建国内最专业的【Kafka中文社区】
PS:提问请尽量把问题一次性描述清楚,并告知环境信息情况哦~!如使用版本、操作步骤、报错/警告信息等,方便嘉宾们快速解答~
## 5 项目成员
### 5.1 内部核心人员
`iceyuhui``liuyaguang``limengmonty``zhangliangmike``nullhuangyiming``zengqiao``eilenexuzhe``huangjiaweihjw``zhaoyinrui``marzkonglingxu``joysunchao`
### 5.2 外部贡献者
`fangjunyu``zhoutaiyang`
## 6 协议
`LogiKM`基于`Apache-2.0`协议进行分发和使用,更多信息参见[协议文件](./LICENSE)

View File

@@ -1,141 +0,0 @@
---
![kafka-manager-logo](./docs/assets/images/common/logo_name.png)
**一站式`Apache Kafka`集群指标监控与运维管控平台**
---
## v2.4.1+
版本上线时间2021-05-21
### 能力提升
- 增加直接增加权限和配额的接口(v2.4.1)
- 增加接口调用可绕过登录的功能(v2.4.1)
### 体验优化
- tomcat 版本提升至8.5.66(v2.4.2)
- op接口优化拆分util接口为topic、leader两类接口(v2.4.1)
- 简化Gateway配置的Key长度(v2.4.1)
### bug修复
- 修复页面展示版本错误问题(v2.4.2)
## v2.4.0
版本上线时间2021-05-18
### 能力提升
- 增加App与Topic自动化审批开关
- Broker元信息中增加Rack信息
- 升级MySQL 驱动支持MySQL 8+
- 增加操作记录查询界面
### 体验优化
- FAQ告警组说明优化
- 用户手册共享及 独享集群概念优化
- 用户管理界面,前端限制用户删除自己
### bug修复
- 修复op-util类中创建Topic失败的接口
- 周期同步Topic到DB的任务修复将Topic列表查询从缓存调整为直接查DB
- 应用下线审批失败的功能修复将权限为0(无权限)的数据进行过滤
- 修复登录及权限绕过的漏洞
- 修复研发角色展示接入集群、暂停监控等按钮的问题
## v2.3.0
版本上线时间2021-02-08
### 能力提升
- 新增支持docker化部署
- 可指定Broker作为候选controller
- 可新增并管理网关配置
- 可获取消费组状态
- 增加集群的JMX认证
### 体验优化
- 优化编辑用户角色、修改密码的流程
- 新增consumerID的搜索功能
- 优化“Topic连接信息”、“消费组重置消费偏移”、“修改Topic保存时间”的文案提示
- 在相应位置增加《资源申请文档》链接
### bug修复
- 修复Broker监控图表时间轴展示错误的问题
- 修复创建夜莺监控告警规则时,使用的告警周期的单位不正确的问题
## v2.2.0
版本上线时间2021-01-25
### 能力提升
- 优化工单批量操作流程
- 增加获取Topic75分位/99分位的实时耗时数据
- 增加定时任务可将无主未落DB的Topic定期写入DB
### 体验优化
- 在相应位置增加《集群接入文档》链接
- 优化物理集群、逻辑集群含义
- 在Topic详情页、Topic扩分区操作弹窗增加展示Topic所属Region的信息
- 优化Topic审批时Topic数据保存时间的配置流程
- 优化Topic/应用申请、审批时的错误提示文案
- 优化Topic数据采样的操作项文案
- 优化运维人员删除Topic时的提示文案
- 优化运维人员删除Region的删除逻辑与提示文案
- 优化运维人员删除逻辑集群的提示文案
- 优化上传集群配置文件时的文件类型限制条件
### bug修复
- 修复填写应用名称时校验特殊字符出错的问题
- 修复普通用户越权访问应用详情的问题
- 修复由于Kafka版本升级导致的数据压缩格式无法获取的问题
- 修复删除逻辑集群或Topic之后界面依旧展示的问题
- 修复进行Leader rebalance操作时执行结果重复提示的问题
## v2.1.0
版本上线时间2020-12-19
### 体验优化
- 优化页面加载时的背景样式
- 优化普通用户申请Topic权限的流程
- 优化Topic申请配额、申请分区的权限限制
- 优化取消Topic权限的文案提示
- 优化申请配额表单的表单项名称
- 优化重置消费偏移的操作流程
- 优化创建Topic迁移任务的表单内容
- 优化Topic扩分区操作的弹窗界面样式
- 优化集群Broker监控可视化图表样式
- 优化创建逻辑集群的表单内容
- 优化集群安全协议的提示文案
### bug修复
- 修复偶发性重置消费偏移失败的问题

View File

@@ -1,28 +0,0 @@
FROM openjdk:16-jdk-alpine3.13
LABEL author="fengxsong"
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories && apk add --no-cache tini
ENV VERSION 2.4.2
WORKDIR /opt/
ENV AGENT_HOME /opt/agent/
COPY docker-depends/config.yaml $AGENT_HOME
COPY docker-depends/jmx_prometheus_javaagent-0.15.0.jar $AGENT_HOME
ENV JAVA_AGENT="-javaagent:$AGENT_HOME/jmx_prometheus_javaagent-0.15.0.jar=9999:$AGENT_HOME/config.yaml"
ENV JAVA_HEAP_OPTS="-Xms1024M -Xmx1024M -Xmn100M "
ENV JAVA_OPTS="-verbose:gc \
-XX:MaxMetaspaceSize=256M -XX:+DisableExplicitGC -XX:+UseStringDeduplication \
-XX:+UseG1GC -XX:+HeapDumpOnOutOfMemoryError -XX:-UseContainerSupport"
RUN wget https://github.com/didi/Logi-KafkaManager/releases/download/v${VERSION}/kafka-manager-${VERSION}.tar.gz && \
tar xvf kafka-manager-${VERSION}.tar.gz && \
mv kafka-manager-${VERSION}/kafka-manager.jar /opt/app.jar && \
rm -rf kafka-manager-${VERSION}*
EXPOSE 8080 9999
ENTRYPOINT ["tini", "--"]
CMD [ "sh", "-c", "java -jar $JAVA_AGENT $JAVA_HEAP_OPTS $JAVA_OPTS app.jar --spring.config.location=application.yml"]

View File

@@ -1,5 +0,0 @@
---
startDelaySeconds: 0
ssl: false
lowercaseOutputName: false
lowercaseOutputLabelNames: false

View File

@@ -1,23 +0,0 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/

View File

@@ -1,6 +0,0 @@
dependencies:
- name: mysql
repository: https://charts.bitnami.com/bitnami
version: 8.6.3
digest: sha256:d250c463c1d78ba30a24a338a06a551503c7a736621d974fe4999d2db7f6143e
generated: "2021-06-24T11:34:54.625217+08:00"

View File

@@ -1,29 +0,0 @@
apiVersion: v2
name: didi-km
description: Logi-KafkaManager
# A chart can be either an 'application' or a 'library' chart.
#
# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.1.0
# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application. Versions are not expected to
# follow Semantic Versioning. They should reflect the version the application is using.
# It is recommended to use it with quotes.
appVersion: "2.4.2"
dependencies:
- condition: mysql.enabled
name: mysql
repository: https://charts.bitnami.com/bitnami
version: 8.x.x

View File

@@ -1,22 +0,0 @@
1. Get the application URL by running these commands:
{{- if .Values.ingress.enabled }}
{{- range $host := .Values.ingress.hosts }}
{{- range .paths }}
http{{ if $.Values.ingress.tls }}s{{ end }}://{{ $host.host }}{{ .path }}
{{- end }}
{{- end }}
{{- else if contains "NodePort" .Values.service.type }}
export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "didi-km.fullname" . }})
export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
echo http://$NODE_IP:$NODE_PORT
{{- else if contains "LoadBalancer" .Values.service.type }}
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
You can watch the status of by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "didi-km.fullname" . }}'
export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include "didi-km.fullname" . }} --template "{{"{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}"}}")
echo http://$SERVICE_IP:{{ .Values.service.port }}
{{- else if contains "ClusterIP" .Values.service.type }}
export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "didi-km.name" . }},app.kubernetes.io/instance={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
export CONTAINER_PORT=$(kubectl get pod --namespace {{ .Release.Namespace }} $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
echo "Visit http://127.0.0.1:8080 to use your application"
kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 8080:$CONTAINER_PORT
{{- end }}

View File

@@ -1,62 +0,0 @@
{{/*
Expand the name of the chart.
*/}}
{{- define "didi-km.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Create a default fully qualified app name.
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
If release name contains chart name it will be used as a full name.
*/}}
{{- define "didi-km.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "didi-km.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Common labels
*/}}
{{- define "didi-km.labels" -}}
helm.sh/chart: {{ include "didi-km.chart" . }}
{{ include "didi-km.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}
{{/*
Selector labels
*/}}
{{- define "didi-km.selectorLabels" -}}
app.kubernetes.io/name: {{ include "didi-km.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
{{/*
Create the name of the service account to use
*/}}
{{- define "didi-km.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "didi-km.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}

View File

@@ -1,101 +0,0 @@
{{- define "datasource.mysql" -}}
{{- if .Values.mysql.enabled }}
{{- printf "%s-mysql" (include "didi-km.fullname" .) -}}
{{- else -}}
{{- printf "%s" .Values.externalDatabase.host -}}
{{- end -}}
{{- end -}}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "didi-km.fullname" . }}-configs
labels:
{{- include "didi-km.labels" . | nindent 4 }}
data:
application.yml: |
server:
port: 8080
tomcat:
accept-count: 1000
max-connections: 10000
max-threads: 800
min-spare-threads: 100
spring:
application:
name: kafkamanager
datasource:
kafka-manager:
jdbc-url: jdbc:mysql://{{ include "datasource.mysql" . }}:3306/{{ .Values.mysql.auth.database }}?characterEncoding=UTF-8&serverTimezone=GMT%2B8&useSSL=false
username: {{ .Values.mysql.auth.username }}
password: {{ .Values.mysql.auth.password }}
driver-class-name: com.mysql.jdbc.Driver
main:
allow-bean-definition-overriding: true
profiles:
active: dev
servlet:
multipart:
max-file-size: 100MB
max-request-size: 100MB
logging:
config: classpath:logback-spring.xml
custom:
idc: cn
jmx:
max-conn: 20
store-metrics-task:
community:
broker-metrics-enabled: true
topic-metrics-enabled: true
didi:
app-topic-metrics-enabled: false
topic-request-time-metrics-enabled: false
topic-throttled-metrics: false
save-days: 7
# 任务相关的开关
task:
op:
sync-topic-enabled: false # 未落盘的Topic定期同步到DB中
account:
# ldap settings
ldap:
enabled: false
authUserRegistration: false
kcm:
enabled: false
storage:
base-url: http://127.0.0.1
n9e:
base-url: http://127.0.0.1:8004
user-token: 12345678
timeout: 300
account: root
script-file: kcm_script.sh
monitor:
enabled: false
n9e:
nid: 2
user-token: 1234567890
mon:
base-url: http://127.0.0.1:8032
sink:
base-url: http://127.0.0.1:8006
rdb:
base-url: http://127.0.0.1:80
notify:
kafka:
cluster-id: 95
topic-name: didi-kafka-notify
order:
detail-url: http://127.0.0.1

View File

@@ -1,64 +0,0 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "didi-km.fullname" . }}
labels:
{{- include "didi-km.labels" . | nindent 4 }}
spec:
{{- if not .Values.autoscaling.enabled }}
replicas: {{ .Values.replicaCount }}
{{- end }}
selector:
matchLabels:
{{- include "didi-km.selectorLabels" . | nindent 6 }}
template:
metadata:
{{- with .Values.podAnnotations }}
annotations:
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "didi-km.selectorLabels" . | nindent 8 }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ include "didi-km.serviceAccountName" . }}
securityContext:
{{- toYaml .Values.podSecurityContext | nindent 8 }}
containers:
- name: {{ .Chart.Name }}
securityContext:
{{- toYaml .Values.securityContext | nindent 12 }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
ports:
- name: http
containerPort: 8080
protocol: TCP
- name: jmx-metrics
containerPort: 9999
protocol: TCP
resources:
{{- toYaml .Values.resources | nindent 12 }}
volumeMounts:
- name: configs
mountPath: /tmp/application.yml
subPath: application.yml
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
volumes:
- name: configs
configMap:
name: {{ include "didi-km.fullname" . }}-configs

View File

@@ -1,28 +0,0 @@
{{- if .Values.autoscaling.enabled }}
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
name: {{ include "didi-km.fullname" . }}
labels:
{{- include "didi-km.labels" . | nindent 4 }}
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: {{ include "didi-km.fullname" . }}
minReplicas: {{ .Values.autoscaling.minReplicas }}
maxReplicas: {{ .Values.autoscaling.maxReplicas }}
metrics:
{{- if .Values.autoscaling.targetCPUUtilizationPercentage }}
- type: Resource
resource:
name: cpu
targetAverageUtilization: {{ .Values.autoscaling.targetCPUUtilizationPercentage }}
{{- end }}
{{- if .Values.autoscaling.targetMemoryUtilizationPercentage }}
- type: Resource
resource:
name: memory
targetAverageUtilization: {{ .Values.autoscaling.targetMemoryUtilizationPercentage }}
{{- end }}
{{- end }}

View File

@@ -1,41 +0,0 @@
{{- if .Values.ingress.enabled -}}
{{- $fullName := include "didi-km.fullname" . -}}
{{- $svcPort := .Values.service.port -}}
{{- if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
apiVersion: networking.k8s.io/v1beta1
{{- else -}}
apiVersion: extensions/v1beta1
{{- end }}
kind: Ingress
metadata:
name: {{ $fullName }}
labels:
{{- include "didi-km.labels" . | nindent 4 }}
{{- with .Values.ingress.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
{{- if .Values.ingress.tls }}
tls:
{{- range .Values.ingress.tls }}
- hosts:
{{- range .hosts }}
- {{ . | quote }}
{{- end }}
secretName: {{ .secretName }}
{{- end }}
{{- end }}
rules:
{{- range .Values.ingress.hosts }}
- host: {{ .host | quote }}
http:
paths:
{{- range .paths }}
- path: {{ .path }}
backend:
serviceName: {{ $fullName }}
servicePort: {{ $svcPort }}
{{- end }}
{{- end }}
{{- end }}

View File

@@ -1,15 +0,0 @@
apiVersion: v1
kind: Service
metadata:
name: {{ include "didi-km.fullname" . }}
labels:
{{- include "didi-km.labels" . | nindent 4 }}
spec:
type: {{ .Values.service.type }}
ports:
- port: {{ .Values.service.port }}
targetPort: http
protocol: TCP
name: http
selector:
{{- include "didi-km.selectorLabels" . | nindent 4 }}

View File

@@ -1,12 +0,0 @@
{{- if .Values.serviceAccount.create -}}
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ include "didi-km.serviceAccountName" . }}
labels:
{{- include "didi-km.labels" . | nindent 4 }}
{{- with .Values.serviceAccount.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
{{- end }}

View File

@@ -1,15 +0,0 @@
apiVersion: v1
kind: Pod
metadata:
name: "{{ include "didi-km.fullname" . }}-test-connection"
labels:
{{- include "didi-km.labels" . | nindent 4 }}
annotations:
"helm.sh/hook": test
spec:
containers:
- name: wget
image: busybox
command: ['wget']
args: ['{{ include "didi-km.fullname" . }}:{{ .Values.service.port }}']
restartPolicy: Never

View File

@@ -1,93 +0,0 @@
# Default values for didi-km.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
replicaCount: 1
image:
repository: docker.io/fengxsong/logi-kafka-manager
pullPolicy: IfNotPresent
# Overrides the image tag whose default is the chart appVersion.
tag: "v2.4.2"
imagePullSecrets: []
nameOverride: ""
# fullnameOverride must set same as release name
fullnameOverride: "km"
serviceAccount:
# Specifies whether a service account should be created
create: true
# Annotations to add to the service account
annotations: {}
# The name of the service account to use.
# If not set and create is true, a name is generated using the fullname template
name: ""
podAnnotations: {}
podSecurityContext: {}
# fsGroup: 2000
securityContext: {}
# capabilities:
# drop:
# - ALL
# readOnlyRootFilesystem: true
# runAsNonRoot: true
# runAsUser: 1000
service:
type: ClusterIP
port: 8080
ingress:
enabled: false
annotations: {}
# kubernetes.io/ingress.class: nginx
# kubernetes.io/tls-acme: "true"
hosts:
- host: chart-example.local
paths: []
tls: []
# - secretName: chart-example-tls
# hosts:
# - chart-example.local
resources:
# We usually recommend not to specify default resources and to leave this as a conscious
# choice for the user. This also increases chances charts run on environments with little
# resources, such as Minikube. If you do want to specify resources, uncomment the following
# lines, adjust them as necessary, and remove the curly braces after 'resources:'.
limits:
cpu: 500m
memory: 2048Mi
requests:
cpu: 100m
memory: 200Mi
autoscaling:
enabled: false
minReplicas: 1
maxReplicas: 100
targetCPUUtilizationPercentage: 80
# targetMemoryUtilizationPercentage: 80
nodeSelector: {}
tolerations: []
affinity: {}
# more configurations are set with configmap in file template/configmap.yaml
externalDatabase:
host: ""
mysql:
# if enabled is set to false, then you should manually specified externalDatabase.host
enabled: true
architecture: standalone
auth:
rootPassword: "s3cretR00t"
database: "logi_kafka_manager"
username: "logi_kafka_manager"
password: "n0tp@55w0rd"

View File

@@ -1,16 +0,0 @@
#!/bin/bash
cd `dirname $0`/../target
target_dir=`pwd`
pid=`ps ax | grep -i 'kafka-manager' | grep ${target_dir} | grep java | grep -v grep | awk '{print $1}'`
if [ -z "$pid" ] ; then
echo "No kafka-manager running."
exit -1;
fi
echo "The kafka-manager (${pid}) is running..."
kill ${pid}
echo "Send shutdown request to kafka-manager (${pid}) OK"

View File

@@ -1,81 +0,0 @@
error_exit ()
{
echo "ERROR: $1 !!"
exit 1
}
[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=$HOME/jdk/java
[ ! -e "$JAVA_HOME/bin/java" ] && JAVA_HOME=/usr/java
[ ! -e "$JAVA_HOME/bin/java" ] && unset JAVA_HOME
if [ -z "$JAVA_HOME" ]; then
if $darwin; then
if [ -x '/usr/libexec/java_home' ] ; then
export JAVA_HOME=`/usr/libexec/java_home`
elif [ -d "/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home" ]; then
export JAVA_HOME="/System/Library/Frameworks/JavaVM.framework/Versions/CurrentJDK/Home"
fi
else
JAVA_PATH=`dirname $(readlink -f $(which javac))`
if [ "x$JAVA_PATH" != "x" ]; then
export JAVA_HOME=`dirname $JAVA_PATH 2>/dev/null`
fi
fi
if [ -z "$JAVA_HOME" ]; then
error_exit "Please set the JAVA_HOME variable in your environment, We need java(x64)! jdk8 or later is better!"
fi
fi
export WEB_SERVER="kafka-manager"
export JAVA_HOME
export JAVA="$JAVA_HOME/bin/java"
export BASE_DIR=`cd $(dirname $0)/..; pwd`
export CUSTOM_SEARCH_LOCATIONS=file:${BASE_DIR}/conf/
#===========================================================================================
# JVM Configuration
#===========================================================================================
JAVA_OPT="${JAVA_OPT} -server -Xms2g -Xmx2g -Xmn1g -XX:MetaspaceSize=128m -XX:MaxMetaspaceSize=320m"
JAVA_OPT="${JAVA_OPT} -XX:-OmitStackTraceInFastThrow -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=${BASE_DIR}/logs/java_heapdump.hprof"
## jdk版本高的情况 有些 参数废弃了
JAVA_MAJOR_VERSION=$($JAVA -version 2>&1 | sed -E -n 's/.* version "([0-9]*).*$/\1/p')
if [[ "$JAVA_MAJOR_VERSION" -ge "9" ]] ; then
JAVA_OPT="${JAVA_OPT} -Xlog:gc*:file=${BASE_DIR}/logs/km_gc.log:time,tags:filecount=10,filesize=102400"
else
JAVA_OPT="${JAVA_OPT} -Djava.ext.dirs=${JAVA_HOME}/jre/lib/ext:${JAVA_HOME}/lib/ext"
JAVA_OPT="${JAVA_OPT} -Xloggc:${BASE_DIR}/logs/km_gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=10 -XX:GCLogFileSize=100M"
fi
JAVA_OPT="${JAVA_OPT} -jar ${BASE_DIR}/target/${WEB_SERVER}.jar"
JAVA_OPT="${JAVA_OPT} --spring.config.additional-location=${CUSTOM_SEARCH_LOCATIONS}"
JAVA_OPT="${JAVA_OPT} --logging.config=${BASE_DIR}/conf/logback-spring.xml"
JAVA_OPT="${JAVA_OPT} --server.max-http-header-size=524288"
if [ ! -d "${BASE_DIR}/logs" ]; then
mkdir ${BASE_DIR}/logs
fi
echo "$JAVA ${JAVA_OPT}"
# check the start.out log output file
if [ ! -f "${BASE_DIR}/logs/start.out" ]; then
touch "${BASE_DIR}/logs/start.out"
fi
# start
echo -e "---- 启动脚本 ------\n $JAVA ${JAVA_OPT}" > ${BASE_DIR}/logs/start.out 2>&1 &
nohup $JAVA ${JAVA_OPT} >> ${BASE_DIR}/logs/start.out 2>&1 &
echo "${WEB_SERVER} is startingyou can check the ${BASE_DIR}/logs/start.out"

View File

@@ -1,28 +0,0 @@
## kafka-manager的配置文件该文件中的配置会覆盖默认配置
## 下面的配置信息基本就是jar中的 application.yml默认配置了;
## 可以只修改自己变更的配置,其他的删除就行了; 比如只配置一下mysql
server:
port: 8080
tomcat:
accept-count: 1000
max-connections: 10000
max-threads: 800
min-spare-threads: 100
spring:
application:
name: kafkamanager
profiles:
active: dev
datasource:
kafka-manager:
jdbc-url: jdbc:mysql://localhost:3306/logi_kafka_manager?characterEncoding=UTF-8&useSSL=false&serverTimezone=GMT%2B8
username: root
password: 123456
driver-class-name: com.mysql.cj.jdbc.Driver
main:
allow-bean-definition-overriding: true

View File

@@ -1,116 +0,0 @@
## kafka-manager的配置文件该文件中的配置会覆盖默认配置
## 下面的配置信息基本就是jar中的 application.yml默认配置了;
## 可以只修改自己变更的配置,其他的删除就行了; 比如只配置一下mysql
server:
port: 8080
tomcat:
accept-count: 1000
max-connections: 10000
max-threads: 800
min-spare-threads: 100
spring:
application:
name: kafkamanager
profiles:
active: dev
datasource:
kafka-manager:
jdbc-url: jdbc:mysql://localhost:3306/logi_kafka_manager?characterEncoding=UTF-8&useSSL=false&serverTimezone=GMT%2B8
username: root
password: 123456
driver-class-name: com.mysql.cj.jdbc.Driver
main:
allow-bean-definition-overriding: true
servlet:
multipart:
max-file-size: 100MB
max-request-size: 100MB
logging:
config: classpath:logback-spring.xml
custom:
idc: cn # 部署的数据中心, 忽略该配置, 后续会进行删除
jmx:
max-conn: 10 # 2.3版本配置不在这个地方生效
store-metrics-task:
community:
broker-metrics-enabled: true # 社区部分broker metrics信息收集开关, 关闭之后metrics信息将不会进行收集及写DB
topic-metrics-enabled: true # 社区部分topic的metrics信息收集开关, 关闭之后metrics信息将不会进行收集及写DB
didi:
app-topic-metrics-enabled: false # 滴滴埋入的指标, 社区AK不存在该指标因此默认关闭
topic-request-time-metrics-enabled: false # 滴滴埋入的指标, 社区AK不存在该指标因此默认关闭
topic-throttled-metrics: false # 滴滴埋入的指标, 社区AK不存在该指标因此默认关闭
save-days: 7 #指标在DB中保持的天数-1表示永久保存7表示保存近7天的数据
# 任务相关的开关
task:
op:
sync-topic-enabled: false # 未落盘的Topic定期同步到DB中
order-auto-exec: # 工单自动化审批线程的开关
topic-enabled: false # Topic工单自动化审批开关, false:关闭自动化审批, true:开启
app-enabled: false # App工单自动化审批开关, false:关闭自动化审批, true:开启
# ldap相关的配置
account:
ldap:
enabled: false
url: ldap://127.0.0.1:389/
basedn: dc=tsign,dc=cn
factory: com.sun.jndi.ldap.LdapCtxFactory
filter: sAMAccountName
security:
authentication: simple
principal: cn=admin,dc=tsign,dc=cn
credentials: admin
auth-user-registration: true
auth-user-registration-role: normal
# 集群升级部署相关的功能需要配合夜莺及S3进行使用
kcm:
enabled: false
s3:
endpoint: s3.didiyunapi.com
access-key: 1234567890
secret-key: 0987654321
bucket: logi-kafka
n9e:
base-url: http://127.0.0.1:8004
user-token: 12345678
timeout: 300
account: root
script-file: kcm_script.sh
# 监控告警相关的功能,需要配合夜莺进行使用
# enabled: 表示是否开启监控告警的功能, true: 开启, false: 不开启
# n9e.nid: 夜莺的节点ID
# n9e.user-token: 用户的密钥,在夜莺的个人设置中
# n9e.mon.base-url: 监控地址
# n9e.sink.base-url: 数据上报地址
# n9e.rdb.base-url: 用户资源中心地址
monitor:
enabled: false
n9e:
nid: 2
user-token: 1234567890
mon:
base-url: http://127.0.0.1:8000 # 夜莺v4版本默认端口统一调整为了8000
sink:
base-url: http://127.0.0.1:8000 # 夜莺v4版本默认端口统一调整为了8000
rdb:
base-url: http://127.0.0.1:8000 # 夜莺v4版本默认端口统一调整为了8000
notify: # 通知的功能
kafka: # 默认通知发送到kafka的指定Topic中
cluster-id: 95 # Topic的集群ID
topic-name: didi-kafka-notify # Topic名称
order: # 部署的KM的地址
detail-url: http://127.0.0.1

View File

@@ -1,591 +0,0 @@
-- create database
CREATE DATABASE logi_kafka_manager;
USE logi_kafka_manager;
--
-- Table structure for table `account`
--
-- DROP TABLE IF EXISTS `account`;
CREATE TABLE `account` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
`username` varchar(128) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '' COMMENT '用户名',
`password` varchar(128) NOT NULL DEFAULT '' COMMENT '密码',
`role` tinyint(8) NOT NULL DEFAULT '0' COMMENT '角色类型, 0:普通用户 1:研发 2:运维',
`status` int(16) NOT NULL DEFAULT '0' COMMENT '0标识使用中-1标识已废弃',
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`gmt_modify` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_username` (`username`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='账号表';
INSERT INTO account(username, password, role) VALUES ('admin', '21232f297a57a5a743894a0e4a801fc3', 2);
--
-- Table structure for table `app`
--
-- DROP TABLE IF EXISTS `app`;
CREATE TABLE `app` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增id',
`app_id` varchar(128) NOT NULL DEFAULT '' COMMENT '应用id',
`name` varchar(192) NOT NULL DEFAULT '' COMMENT '应用名称',
`password` varchar(256) NOT NULL DEFAULT '' COMMENT '应用密码',
`type` int(11) NOT NULL DEFAULT '0' COMMENT '类型, 0:普通用户, 1:超级用户',
`applicant` varchar(64) NOT NULL DEFAULT '' COMMENT '申请人',
`principals` text COMMENT '应用负责人',
`description` text COMMENT '应用描述',
`create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`modify_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_name` (`name`),
UNIQUE KEY `uniq_app_id` (`app_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='应用信息';
--
-- Table structure for table `authority`
--
-- DROP TABLE IF EXISTS `authority`;
CREATE TABLE `authority` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增id',
`app_id` varchar(128) NOT NULL DEFAULT '' COMMENT '应用id',
`cluster_id` bigint(20) NOT NULL DEFAULT '0' COMMENT '集群id',
`topic_name` varchar(192) NOT NULL DEFAULT '' COMMENT 'topic名称',
`access` int(11) NOT NULL DEFAULT '0' COMMENT '0:无权限, 1:读, 2:写, 3:读写',
`create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`modify_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_app_id_cluster_id_topic_name` (`app_id`,`cluster_id`,`topic_name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='权限信息(kafka-manager)';
--
-- Table structure for table `broker`
--
-- DROP TABLE IF EXISTS `broker`;
CREATE TABLE `broker` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
`cluster_id` bigint(20) NOT NULL DEFAULT '-1' COMMENT '集群id',
`broker_id` int(16) NOT NULL DEFAULT '-1' COMMENT 'brokerid',
`host` varchar(128) NOT NULL DEFAULT '' COMMENT 'broker主机名',
`port` int(16) NOT NULL DEFAULT '-1' COMMENT 'broker端口',
`timestamp` bigint(20) NOT NULL DEFAULT '-1' COMMENT '启动时间',
`max_avg_bytes_in` bigint(20) NOT NULL DEFAULT '-1' COMMENT '峰值的均值流量',
`version` varchar(128) NOT NULL DEFAULT '' COMMENT 'broker版本',
`status` int(16) NOT NULL DEFAULT '0' COMMENT '状态: 0有效-1无效',
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`gmt_modify` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_cluster_id_broker_id` (`cluster_id`,`broker_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='broker信息表';
--
-- Table structure for table `broker_metrics`
--
-- DROP TABLE IF EXISTS `broker_metrics`;
CREATE TABLE `broker_metrics` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增id',
`cluster_id` bigint(20) NOT NULL DEFAULT '-1' COMMENT '集群id',
`broker_id` int(16) NOT NULL DEFAULT '-1' COMMENT 'brokerid',
`metrics` text COMMENT '指标',
`messages_in` double(53,2) NOT NULL DEFAULT '0.00' COMMENT '每秒消息数流入',
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
PRIMARY KEY (`id`),
KEY `idx_cluster_id_broker_id_gmt_create` (`cluster_id`,`broker_id`,`gmt_create`),
KEY `idx_gmt_create` (`gmt_create`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='broker-metric信息表';
--
-- Table structure for table `cluster`
--
-- DROP TABLE IF EXISTS `cluster`;
CREATE TABLE `cluster` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT '集群id',
`cluster_name` varchar(128) NOT NULL DEFAULT '' COMMENT '集群名称',
`zookeeper` varchar(512) NOT NULL DEFAULT '' COMMENT 'zk地址',
`bootstrap_servers` varchar(512) NOT NULL DEFAULT '' COMMENT 'server地址',
`kafka_version` varchar(32) NOT NULL DEFAULT '' COMMENT 'kafka版本',
`security_properties` text COMMENT 'Kafka安全认证参数',
`jmx_properties` text COMMENT 'JMX配置',
`status` tinyint(4) NOT NULL DEFAULT '1' COMMENT ' 监控标记, 0表示未监控, 1表示监控中',
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`gmt_modify` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_cluster_name` (`cluster_name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='cluster信息表';
--
-- Table structure for table `cluster_metrics`
--
-- DROP TABLE IF EXISTS `cluster_metrics`;
CREATE TABLE `cluster_metrics` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增id',
`cluster_id` bigint(20) NOT NULL DEFAULT '0' COMMENT '集群id',
`metrics` text COMMENT '指标',
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
PRIMARY KEY (`id`),
KEY `idx_cluster_id_gmt_create` (`cluster_id`,`gmt_create`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='clustermetrics信息';
--
-- Table structure for table `cluster_tasks`
--
-- DROP TABLE IF EXISTS `cluster_tasks`;
CREATE TABLE `cluster_tasks` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增id',
`uuid` varchar(128) NOT NULL DEFAULT '' COMMENT '任务UUID',
`cluster_id` bigint(128) NOT NULL DEFAULT '-1' COMMENT '集群id',
`task_type` varchar(128) NOT NULL DEFAULT '' COMMENT '任务类型',
`kafka_package` text COMMENT 'kafka包',
`kafka_package_md5` varchar(128) NOT NULL DEFAULT '' COMMENT 'kafka包的md5',
`server_properties` text COMMENT 'kafkaserver配置',
`server_properties_md5` varchar(128) NOT NULL DEFAULT '' COMMENT '配置文件的md5',
`agent_task_id` bigint(128) NOT NULL DEFAULT '-1' COMMENT '任务id',
`agent_rollback_task_id` bigint(128) NOT NULL DEFAULT '-1' COMMENT '回滚任务id',
`host_list` text COMMENT '升级的主机',
`pause_host_list` text COMMENT '暂停点',
`rollback_host_list` text COMMENT '回滚机器列表',
`rollback_pause_host_list` text COMMENT '回滚暂停机器列表',
`operator` varchar(64) NOT NULL DEFAULT '' COMMENT '操作人',
`task_status` int(11) NOT NULL DEFAULT '0' COMMENT '任务状态',
`create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`modify_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='集群任务(集群升级部署)';
--
-- Table structure for table `config`
--
-- DROP TABLE IF EXISTS `config`;
CREATE TABLE `config` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
`config_key` varchar(128) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '' COMMENT '配置key',
`config_value` text COMMENT '配置value',
`config_description` text COMMENT '备注说明',
`status` int(16) NOT NULL DEFAULT '0' COMMENT '0标识使用中-1标识已废弃',
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`gmt_modify` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_config_key` (`config_key`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='配置表';
--
-- Table structure for table `controller`
--
-- DROP TABLE IF EXISTS `controller`;
CREATE TABLE `controller` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增id',
`cluster_id` bigint(20) NOT NULL DEFAULT '-1' COMMENT '集群id',
`broker_id` int(16) NOT NULL DEFAULT '-1' COMMENT 'brokerid',
`host` varchar(256) NOT NULL DEFAULT '' COMMENT '主机名',
`timestamp` bigint(20) NOT NULL DEFAULT '-1' COMMENT 'controller变更时间',
`version` int(16) NOT NULL DEFAULT '-1' COMMENT 'controller格式版本',
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_cluster_id_broker_id_timestamp` (`cluster_id`,`broker_id`,`timestamp`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='controller记录表';
--
-- Table structure for table `gateway_config`
--
-- DROP TABLE IF EXISTS `gateway_config`;
CREATE TABLE `gateway_config` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
`type` varchar(128) NOT NULL DEFAULT '' COMMENT '配置类型',
`name` varchar(128) NOT NULL DEFAULT '' COMMENT '配置名称',
`value` text COMMENT '配置值',
`version` bigint(20) unsigned NOT NULL DEFAULT '1' COMMENT '版本信息',
`description` text COMMENT '描述信息',
`create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`modify_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_type_name` (`type`,`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='gateway配置';
INSERT INTO gateway_config(type, name, value, `version`, `description`) values('SD_QUEUE_SIZE', 'SD_QUEUE_SIZE', 100000000, 1, '任意集群队列大小');
INSERT INTO gateway_config(type, name, value, `version`, `description`) values('SD_APP_RATE', 'SD_APP_RATE', 100000000, 1, '任意一个App限速');
INSERT INTO gateway_config(type, name, value, `version`, `description`) values('SD_IP_RATE', 'SD_IP_RATE', 100000000, 1, '任意一个IP限速');
INSERT INTO gateway_config(type, name, value, `version`, `description`) values('SD_SP_RATE', 'app_01234567', 100000000, 1, '指定App限速');
INSERT INTO gateway_config(type, name, value, `version`, `description`) values('SD_SP_RATE', '192.168.0.1', 100000000, 1, '指定IP限速');
--
-- Table structure for table `heartbeat`
--
-- DROP TABLE IF EXISTS `heartbeat`;
CREATE TABLE `heartbeat` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
`ip` varchar(128) NOT NULL DEFAULT '' COMMENT '主机ip',
`hostname` varchar(256) NOT NULL DEFAULT '' COMMENT '主机名',
`create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`modify_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_ip` (`ip`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='心跳信息';
--
-- Table structure for table `kafka_acl`
--
-- DROP TABLE IF EXISTS `kafka_acl`;
CREATE TABLE `kafka_acl` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增id',
`app_id` varchar(128) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '' COMMENT '用户id',
`cluster_id` bigint(20) NOT NULL DEFAULT '0' COMMENT '集群id',
`topic_name` varchar(192) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '' COMMENT 'topic名称',
`access` int(11) NOT NULL DEFAULT '0' COMMENT '0:无权限, 1:读, 2:写, 3:读写',
`operation` int(11) NOT NULL DEFAULT '0' COMMENT '0:创建, 1:更新 2:删除, 以最新的一条数据为准',
`create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='权限信息(kafka-broker)';
--
-- Table structure for table `kafka_bill`
--
-- DROP TABLE IF EXISTS `kafka_bill`;
CREATE TABLE `kafka_bill` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
`cluster_id` bigint(20) NOT NULL DEFAULT '0' COMMENT '集群id',
`topic_name` varchar(192) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '' COMMENT 'topic名称',
`principal` varchar(64) NOT NULL DEFAULT '' COMMENT '负责人',
`quota` double(53,2) NOT NULL DEFAULT '0.00' COMMENT '配额, 单位mb/s',
`cost` double(53,2) NOT NULL DEFAULT '0.00' COMMENT '成本, 单位元',
`cost_type` int(16) NOT NULL DEFAULT '0' COMMENT '成本类型, 0:共享集群, 1:独享集群, 2:独立集群',
`gmt_day` varchar(64) NOT NULL DEFAULT '' COMMENT '计价的日期, 例如2019-02-02的计价结果',
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_cluster_id_topic_name_gmt_day` (`cluster_id`,`topic_name`,`gmt_day`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='kafka账单';
--
-- Table structure for table `kafka_file`
--
-- DROP TABLE IF EXISTS `kafka_file`;
CREATE TABLE `kafka_file` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增id',
`cluster_id` bigint(20) NOT NULL DEFAULT '-1' COMMENT '集群id',
`storage_name` varchar(128) NOT NULL DEFAULT '' COMMENT '存储位置',
`file_name` varchar(128) NOT NULL DEFAULT '' COMMENT '文件名',
`file_md5` varchar(256) NOT NULL DEFAULT '' COMMENT '文件md5',
`file_type` int(16) NOT NULL DEFAULT '-1' COMMENT '0:kafka压缩包, 1:kafkaserver配置',
`description` text COMMENT '备注信息',
`operator` varchar(64) NOT NULL DEFAULT '' COMMENT '创建用户',
`status` int(16) NOT NULL DEFAULT '0' COMMENT '状态, 0:正常, -1:删除',
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`gmt_modify` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_cluster_id_file_name_storage_name` (`cluster_id`,`file_name`,`storage_name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='文件管理';
--
-- Table structure for table `kafka_user`
--
-- DROP TABLE IF EXISTS `kafka_user`;
CREATE TABLE `kafka_user` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增id',
`app_id` varchar(128) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '' COMMENT '应用id',
`password` varchar(256) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '' COMMENT '密码',
`user_type` int(11) NOT NULL DEFAULT '0' COMMENT '0:普通用户, 1:超级用户',
`operation` int(11) NOT NULL DEFAULT '0' COMMENT '0:创建, 1:更新 2:删除, 以最新一条的记录为准',
`create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='kafka用户表';
INSERT INTO app(app_id, name, password, type, applicant, principals, description) VALUES ('dkm_admin', 'KM管理员', 'km_kMl4N8as1Kp0CCY', 1, 'admin', 'admin', 'KM管理员应用-谨慎对外提供');
INSERT INTO kafka_user(app_id, password, user_type, operation) VALUES ('dkm_admin', 'km_kMl4N8as1Kp0CCY', 1, 0);
--
-- Table structure for table `logical_cluster`
--
CREATE TABLE `logical_cluster` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
`name` varchar(192) NOT NULL DEFAULT '' COMMENT '逻辑集群名称',
`identification` varchar(192) NOT NULL DEFAULT '' COMMENT '逻辑集群标识',
`mode` int(16) NOT NULL DEFAULT '0' COMMENT '逻辑集群类型, 0:共享集群, 1:独享集群, 2:独立集群',
`app_id` varchar(64) NOT NULL DEFAULT '' COMMENT '所属应用',
`cluster_id` bigint(20) NOT NULL DEFAULT '-1' COMMENT '集群id',
`region_list` varchar(256) NOT NULL DEFAULT '' COMMENT 'regionid列表',
`description` text COMMENT '备注说明',
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`gmt_modify` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_name` (`name`),
UNIQUE KEY `uniq_identification` (`identification`)
) ENGINE=InnoDB AUTO_INCREMENT=7 DEFAULT CHARSET=utf8 COMMENT='逻辑集群信息表';
--
-- Table structure for table `monitor_rule`
--
-- DROP TABLE IF EXISTS `monitor_rule`;
CREATE TABLE `monitor_rule` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增id',
`name` varchar(192) NOT NULL DEFAULT '' COMMENT '告警名称',
`strategy_id` bigint(20) NOT NULL DEFAULT '-1' COMMENT '监控id',
`app_id` varchar(64) NOT NULL DEFAULT '' COMMENT 'appid',
`operator` varchar(64) NOT NULL DEFAULT '' COMMENT '操作人',
`create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`modify_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='监控规则';
--
-- Table structure for table `operate_record`
--
-- DROP TABLE IF EXISTS `operate_record`;
CREATE TABLE `operate_record` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
`module_id` int(16) NOT NULL DEFAULT '-1' COMMENT '模块类型, 0:topic, 1:应用, 2:配额, 3:权限, 4:集群, -1:未知',
`operate_id` int(16) NOT NULL DEFAULT '-1' COMMENT '操作类型, 0:新增, 1:删除, 2:修改',
`resource` varchar(256) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '' COMMENT 'topic名称、app名称',
`content` text COMMENT '操作内容',
`operator` varchar(64) NOT NULL DEFAULT '' COMMENT '操作人',
`create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`modify_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
PRIMARY KEY (`id`),
KEY `idx_module_id_operate_id_operator` (`module_id`,`operate_id`,`operator`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='操作记录';
--
-- Table structure for table `reassign_task`
--
-- DROP TABLE IF EXISTS `reassign_task`;
CREATE TABLE `reassign_task` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增id',
`task_id` bigint(20) NOT NULL DEFAULT '-1' COMMENT '任务ID',
`name` varchar(256) NOT NULL DEFAULT '' COMMENT '任务名称',
`cluster_id` bigint(20) NOT NULL DEFAULT '0' COMMENT '集群id',
`topic_name` varchar(192) NOT NULL DEFAULT '' COMMENT 'Topic名称',
`partitions` text COMMENT '分区',
`reassignment_json` text COMMENT '任务参数',
`real_throttle` bigint(20) NOT NULL DEFAULT '0' COMMENT '限流值',
`max_throttle` bigint(20) NOT NULL DEFAULT '0' COMMENT '限流上限',
`min_throttle` bigint(20) NOT NULL DEFAULT '0' COMMENT '限流下限',
`begin_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '开始时间',
`operator` varchar(64) NOT NULL DEFAULT '' COMMENT '操作人',
`description` varchar(256) NOT NULL DEFAULT '' COMMENT '备注说明',
`status` int(16) NOT NULL DEFAULT '0' COMMENT '任务状态',
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '任务创建时间',
`gmt_modify` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '任务修改时间',
`original_retention_time` bigint(20) NOT NULL DEFAULT '86400000' COMMENT 'Topic存储时间',
`reassign_retention_time` bigint(20) NOT NULL DEFAULT '86400000' COMMENT '迁移时的存储时间',
`src_brokers` text COMMENT '源Broker',
`dest_brokers` text COMMENT '目标Broker',
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='topic迁移信息';
--
-- Table structure for table `region`
--
-- DROP TABLE IF EXISTS `region`;
CREATE TABLE `region` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
`name` varchar(192) NOT NULL DEFAULT '' COMMENT 'region名称',
`cluster_id` bigint(20) NOT NULL DEFAULT '-1' COMMENT '集群id',
`broker_list` varchar(256) NOT NULL DEFAULT '' COMMENT 'broker列表',
`capacity` bigint(20) NOT NULL DEFAULT '0' COMMENT '容量(B/s)',
`real_used` bigint(20) NOT NULL DEFAULT '0' COMMENT '实际使用量(B/s)',
`estimate_used` bigint(20) NOT NULL DEFAULT '0' COMMENT '预估使用量(B/s)',
`description` text COMMENT '备注说明',
`status` int(16) NOT NULL DEFAULT '0' COMMENT '状态0正常1已满',
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`gmt_modify` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='region信息表';
--
-- Table structure for table `topic`
--
-- DROP TABLE IF EXISTS `topic`;
CREATE TABLE `topic` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
`cluster_id` bigint(20) NOT NULL DEFAULT '-1' COMMENT '集群id',
`topic_name` varchar(192) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '' COMMENT 'topic名称',
`app_id` varchar(64) NOT NULL DEFAULT '' COMMENT 'topic所属appid',
`peak_bytes_in` bigint(20) NOT NULL DEFAULT '0' COMMENT '峰值流量',
`description` text COMMENT '备注信息',
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`gmt_modify` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_cluster_id_topic_name` (`cluster_id`,`topic_name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='topic信息表';
--
-- Table structure for table `topic_app_metrics`
--
-- DROP TABLE IF EXISTS `topic_app_metrics`;
CREATE TABLE `topic_app_metrics` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
`cluster_id` bigint(20) NOT NULL DEFAULT '0' COMMENT '集群id',
`topic_name` varchar(192) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '' COMMENT 'topic名称',
`app_id` varchar(64) NOT NULL DEFAULT '' COMMENT 'appid',
`metrics` text COMMENT '指标',
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
PRIMARY KEY (`id`),
KEY `idx_cluster_id_topic_name_app_id_gmt_create` (`cluster_id`,`topic_name`,`app_id`,`gmt_create`),
KEY `idx_gmt_create` (`gmt_create`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='topic app metrics';
--
-- Table structure for table `topic_connections`
--
-- DROP TABLE IF EXISTS `topic_connections`;
CREATE TABLE `topic_connections` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
`app_id` varchar(64) NOT NULL DEFAULT '' COMMENT '应用id',
`cluster_id` bigint(20) NOT NULL DEFAULT '0' COMMENT '集群id',
`topic_name` varchar(192) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '' COMMENT 'topic名称',
`type` varchar(16) NOT NULL DEFAULT '' COMMENT 'producer or consumer',
`ip` varchar(32) NOT NULL DEFAULT '' COMMENT 'ip地址',
`client_version` varchar(8) NOT NULL DEFAULT '' COMMENT '客户端版本',
`create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_app_id_cluster_id_topic_name_type_ip_client_version` (`app_id`,`cluster_id`,`topic_name`,`type`,`ip`,`client_version`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='topic连接信息表';
--
-- Table structure for table `topic_expired`
--
-- DROP TABLE IF EXISTS `topic_expired`;
CREATE TABLE `topic_expired` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
`cluster_id` bigint(20) NOT NULL DEFAULT '-1' COMMENT '集群id',
`topic_name` varchar(192) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '' COMMENT 'topic名称',
`produce_connection_num` bigint(20) NOT NULL DEFAULT '0' COMMENT '发送连接数',
`fetch_connection_num` bigint(20) NOT NULL DEFAULT '0' COMMENT '消费连接数',
`expired_day` bigint(20) NOT NULL DEFAULT '0' COMMENT '过期天数',
`gmt_retain` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '保留截止时间',
`status` int(16) NOT NULL DEFAULT '0' COMMENT '-1:可下线, 0:过期待通知, 1+:已通知待反馈',
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`gmt_modify` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_cluster_id_topic_name` (`cluster_id`,`topic_name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='topic过期信息表';
--
-- Table structure for table `topic_metrics`
--
-- DROP TABLE IF EXISTS `topic_metrics`;
CREATE TABLE `topic_metrics` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增id',
`cluster_id` bigint(20) NOT NULL DEFAULT '-1' COMMENT '集群id',
`topic_name` varchar(192) NOT NULL DEFAULT '' COMMENT 'topic名称',
`metrics` text COMMENT '指标数据JSON',
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
PRIMARY KEY (`id`),
KEY `idx_cluster_id_topic_name_gmt_create` (`cluster_id`,`topic_name`,`gmt_create`),
KEY `idx_gmt_create` (`gmt_create`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='topicmetrics表';
--
-- Table structure for table `topic_report`
--
-- DROP TABLE IF EXISTS `topic_report`;
CREATE TABLE `topic_report` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
`cluster_id` bigint(20) NOT NULL DEFAULT '0' COMMENT '集群id',
`topic_name` varchar(192) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '' COMMENT 'topic名称',
`start_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '开始上报时间',
`end_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '结束上报时间',
`create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`modify_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_cluster_id_topic_name` (`cluster_id`,`topic_name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='开启jmx采集的topic';
--
-- Table structure for table `topic_request_time_metrics`
--
-- DROP TABLE IF EXISTS `topic_request_time_metrics`;
CREATE TABLE `topic_request_time_metrics` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
`cluster_id` bigint(20) NOT NULL DEFAULT '-1' COMMENT '集群id',
`topic_name` varchar(192) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '' COMMENT 'topic名称',
`metrics` text COMMENT '指标',
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
PRIMARY KEY (`id`),
KEY `idx_cluster_id_topic_name_gmt_create` (`cluster_id`,`topic_name`,`gmt_create`),
KEY `idx_gmt_create` (`gmt_create`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='topic请求耗时信息';
--
-- Table structure for table `topic_statistics`
--
-- DROP TABLE IF EXISTS `topic_statistics`;
CREATE TABLE `topic_statistics` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT '自增id',
`cluster_id` bigint(20) NOT NULL DEFAULT '-1' COMMENT '集群id',
`topic_name` varchar(192) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '' COMMENT 'topic名称',
`offset_sum` bigint(20) NOT NULL DEFAULT '-1' COMMENT 'offset和',
`max_avg_bytes_in` double(53,2) NOT NULL DEFAULT '-1.00' COMMENT '峰值的均值流量',
`gmt_day` varchar(64) NOT NULL DEFAULT '' COMMENT '日期2020-03-30的形式',
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`max_avg_messages_in` double(53,2) NOT NULL DEFAULT '-1.00' COMMENT '峰值的均值消息条数',
PRIMARY KEY (`id`),
UNIQUE KEY `uniq_cluster_id_topic_name_gmt_day` (`cluster_id`,`topic_name`,`gmt_day`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='topic统计信息表';
--
-- Table structure for table `topic_throttled_metrics`
--
-- DROP TABLE IF EXISTS `topic_throttled_metrics`;
CREATE TABLE `topic_throttled_metrics` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
`cluster_id` bigint(20) NOT NULL DEFAULT '-1' COMMENT '集群id',
`topic_name` varchar(192) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '' COMMENT 'topic name',
`app_id` varchar(64) NOT NULL DEFAULT '' COMMENT 'app',
`produce_throttled` tinyint(8) NOT NULL DEFAULT '0' COMMENT '是否是生产耗时',
`fetch_throttled` tinyint(8) NOT NULL DEFAULT '0' COMMENT '是否是消费耗时',
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
PRIMARY KEY (`id`),
KEY `idx_cluster_id_topic_name_app_id` (`cluster_id`,`topic_name`,`app_id`),
KEY `idx_gmt_create` (`gmt_create`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='topic限流信息';
--
-- Table structure for table `work_order`
--
-- DROP TABLE IF EXISTS `work_order`;
CREATE TABLE `work_order` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
`type` int(16) NOT NULL DEFAULT '-1' COMMENT '工单类型',
`title` varchar(512) NOT NULL DEFAULT '' COMMENT '工单标题',
`applicant` varchar(64) NOT NULL DEFAULT '' COMMENT '申请人',
`description` text COMMENT '备注信息',
`approver` varchar(64) NOT NULL DEFAULT '' COMMENT '审批人',
`gmt_handle` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '审批时间',
`opinion` varchar(256) NOT NULL DEFAULT '' COMMENT '审批信息',
`extensions` text COMMENT '扩展信息',
`status` int(16) NOT NULL DEFAULT '0' COMMENT '工单状态, 0:待审批, 1:通过, 2:拒绝, 3:取消',
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`gmt_modify` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='工单表';

View File

@@ -1,215 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true" scanPeriod="10 seconds">
<contextName>logback</contextName>
<property name="log.path" value="./logs" />
<!-- 彩色日志 -->
<!-- 彩色日志依赖的渲染类 -->
<conversionRule conversionWord="clr" converterClass="org.springframework.boot.logging.logback.ColorConverter" />
<conversionRule conversionWord="wex" converterClass="org.springframework.boot.logging.logback.WhitespaceThrowableProxyConverter" />
<conversionRule conversionWord="wEx" converterClass="org.springframework.boot.logging.logback.ExtendedWhitespaceThrowableProxyConverter" />
<!-- 彩色日志格式 -->
<property name="CONSOLE_LOG_PATTERN" value="${CONSOLE_LOG_PATTERN:-%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(${LOG_LEVEL_PATTERN:-%5p}) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %m%n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}}"/>
<!--输出到控制台-->
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>info</level>
</filter>
<encoder>
<Pattern>${CONSOLE_LOG_PATTERN}</Pattern>
<charset>UTF-8</charset>
</encoder>
</appender>
<!--输出到文件-->
<!-- 时间滚动输出 level为 DEBUG 日志 -->
<appender name="DEBUG_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${log.path}/log_debug.log</file>
<!--日志文件输出格式-->
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg%n</pattern>
<charset>UTF-8</charset> <!-- 设置字符集 -->
</encoder>
<!-- 日志记录器的滚动策略,按日期,按大小记录 -->
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- 日志归档 -->
<fileNamePattern>${log.path}/log_debug_%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<!--日志文件保留天数-->
<maxHistory>7</maxHistory>
</rollingPolicy>
<!-- 此日志文件只记录debug级别的 -->
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>debug</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>DENY</onMismatch>
</filter>
</appender>
<!-- 时间滚动输出 level为 INFO 日志 -->
<appender name="INFO_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<!-- 正在记录的日志文件的路径及文件名 -->
<file>${log.path}/log_info.log</file>
<!--日志文件输出格式-->
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg%n</pattern>
<charset>UTF-8</charset>
</encoder>
<!-- 日志记录器的滚动策略,按日期,按大小记录 -->
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- 每天日志归档路径以及格式 -->
<fileNamePattern>${log.path}/log_info_%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<!--日志文件保留天数-->
<maxHistory>7</maxHistory>
</rollingPolicy>
<!-- 此日志文件只记录info级别的 -->
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>info</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>DENY</onMismatch>
</filter>
</appender>
<!-- 时间滚动输出 level为 WARN 日志 -->
<appender name="WARN_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<!-- 正在记录的日志文件的路径及文件名 -->
<file>${log.path}/log_warn.log</file>
<!--日志文件输出格式-->
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg%n</pattern>
<charset>UTF-8</charset> <!-- 此处设置字符集 -->
</encoder>
<!-- 日志记录器的滚动策略,按日期,按大小记录 -->
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${log.path}/log_warn_%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<!--日志文件保留天数-->
<maxHistory>7</maxHistory>
</rollingPolicy>
<!-- 此日志文件只记录warn级别的 -->
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>warn</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>DENY</onMismatch>
</filter>
</appender>
<!-- 时间滚动输出 level为 ERROR 日志 -->
<appender name="ERROR_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<!-- 正在记录的日志文件的路径及文件名 -->
<file>${log.path}/log_error.log</file>
<!--日志文件输出格式-->
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg%n</pattern>
<charset>UTF-8</charset> <!-- 此处设置字符集 -->
</encoder>
<!-- 日志记录器的滚动策略,按日期,按大小记录 -->
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${log.path}/log_error_%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<!--日志文件保留天数-->
<maxHistory>7</maxHistory>
</rollingPolicy>
<!-- 此日志文件只记录ERROR级别的 -->
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>ERROR</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>DENY</onMismatch>
</filter>
</appender>
<!-- Metrics信息收集日志 -->
<appender name="COLLECTOR_METRICS_LOGGER" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${log.path}/metrics/collector_metrics.log</file>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg%n</pattern>
<charset>UTF-8</charset>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${log.path}/metrics/collector_metrics_%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<maxHistory>3</maxHistory>
</rollingPolicy>
</appender>
<!-- Metrics信息收集日志 -->
<appender name="API_METRICS_LOGGER" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${log.path}/metrics/api_metrics.log</file>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg%n</pattern>
<charset>UTF-8</charset>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${log.path}/metrics/api_metrics_%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<maxHistory>3</maxHistory>
</rollingPolicy>
</appender>
<!-- Metrics信息收集日志 -->
<appender name="SCHEDULED_TASK_LOGGER" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${log.path}/metrics/scheduled_tasks.log</file>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{50} - %msg%n</pattern>
<charset>UTF-8</charset>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${log.path}/metrics/scheduled_tasks_%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<maxHistory>5</maxHistory>
</rollingPolicy>
</appender>
<logger name="COLLECTOR_METRICS_LOGGER" level="DEBUG" additivity="false">
<appender-ref ref="COLLECTOR_METRICS_LOGGER"/>
</logger>
<logger name="API_METRICS_LOGGER" level="DEBUG" additivity="false">
<appender-ref ref="API_METRICS_LOGGER"/>
</logger>
<logger name="SCHEDULED_TASK_LOGGER" level="DEBUG" additivity="false">
<appender-ref ref="SCHEDULED_TASK_LOGGER"/>
</logger>
<logger name="org.apache.ibatis" level="INFO" additivity="false" />
<logger name="org.mybatis.spring" level="INFO" additivity="false" />
<logger name="com.github.miemiedev.mybatis.paginator" level="INFO" additivity="false" />
<root level="info">
<appender-ref ref="CONSOLE" />
<appender-ref ref="DEBUG_FILE" />
<appender-ref ref="INFO_FILE" />
<appender-ref ref="WARN_FILE" />
<appender-ref ref="ERROR_FILE" />
<!--<appender-ref ref="METRICS_LOG" />-->
</root>
<!--生产环境:输出到文件-->
<!--<springProfile name="pro">-->
<!--<root level="info">-->
<!--<appender-ref ref="CONSOLE" />-->
<!--<appender-ref ref="DEBUG_FILE" />-->
<!--<appender-ref ref="INFO_FILE" />-->
<!--<appender-ref ref="ERROR_FILE" />-->
<!--<appender-ref ref="WARN_FILE" />-->
<!--</root>-->
<!--</springProfile>-->
</configuration>

View File

@@ -1,64 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>kafka-manager</artifactId>
<groupId>com.xiaojukeji.kafka</groupId>
<version>${kafka-manager.revision}</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>distribution</artifactId>
<name>distribution</name>
<packaging>pom</packaging>
<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>kafka-manager-web</artifactId>
<version>${kafka-manager.revision}</version>
</dependency>
</dependencies>
<profiles>
<profile>
<id>release-kafka-manager</id>
<dependencies>
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>kafka-manager-web</artifactId>
<version>${kafka-manager.revision}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<configuration>
<descriptors>
<descriptor>release-km.xml</descriptor>
</descriptors>
<tarLongFileMode>posix</tarLongFileMode>
</configuration>
<executions>
<execution>
<id>make-assembly</id>
<phase>install</phase>
<goals>
<goal>single</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
<finalName>kafka-manager</finalName>
</build>
</profile>
</profiles>
</project>

View File

@@ -1,22 +0,0 @@
## 说明
### 1.创建mysql数据库文件
> conf/create_mysql_table.sql
### 2. 修改配置文件
> conf/application.yml.example
> 请将application.yml.example 复制一份改名为application.yml
> 并放在同级目录下(conf/); 并修改成自己的配置
> 这里的优先级比jar包内配置文件的默认值高;
>
### 3.启动/关闭kafka-manager
> sh bin/startup.sh 启动
>
> sh shutdown.sh 关闭
>
### 4.升级jar包
> 如果是升级, 可以看看文件 `upgrade_config.md` 的配置变更历史;
>

View File

@@ -1,51 +0,0 @@
<?xml version="1.0" encoding="UTF-8"?>
<assembly>
<id>${project.version}</id>
<includeBaseDirectory>true</includeBaseDirectory>
<formats>
<format>dir</format>
<format>tar.gz</format>
<format>zip</format>
</formats>
<fileSets>
<fileSet>
<includes>
<include>conf/**</include>
</includes>
</fileSet>
<fileSet>
<includes>
<include>bin/*</include>
</includes>
<fileMode>0755</fileMode>
</fileSet>
</fileSets>
<files>
<file>
<source>readme.md</source>
<destName>readme.md</destName>
</file>
<file>
<source>upgrade_config.md</source>
<destName>upgrade_config.md</destName>
</file>
<file>
<!--打好的jar包名称和放置目录-->
<source>../kafka-manager-web/target/kafka-manager.jar</source>
<outputDirectory>target/</outputDirectory>
</file>
</files>
<moduleSets>
<moduleSet>
<useAllReactorProjects>true</useAllReactorProjects>
<includes>
<include>com.xiaojukeji.kafka:kafka-manager-web</include>
</includes>
</moduleSet>
</moduleSets>
</assembly>

View File

@@ -1,42 +0,0 @@
## 版本升级配置变更
> 本文件 从 V2.2.0 开始记录; 如果配置有变更则会填写到下文中; 如果没有,则表示无变更;
> 当您从一个很低的版本升级时候,应该依次执行中间有过变更的sql脚本
**一站式`Apache Kafka`集群指标监控与运维管控平台**
---
### 1.升级至`V2.2.0`版本
#### 1.mysql变更
`2.2.0`版本在`cluster`表及`logical_cluster`各增加了一个字段因此需要执行下面的sql进行字段的增加。
```sql
# cluster表中增加jmx_properties字段, 这个字段会用于存储jmx相关的认证以及配置信息
ALTER TABLE `cluster` ADD COLUMN `jmx_properties` TEXT NULL COMMENT 'JMX配置' AFTER `security_properties`;
# logical_cluster中增加identification字段, 同时数据和原先name数据相同, 最后增加一个唯一键.
# 此后, name字段还是表示集群名称, identification字段表示的是集群标识, 只能是字母数字及下划线组成,
# 数据上报到监控系统时, 集群这个标识采用的字段就是identification字段, 之前使用的是name字段.
ALTER TABLE `logical_cluster` ADD COLUMN `identification` VARCHAR(192) NOT NULL DEFAULT '' COMMENT '逻辑集群标识' AFTER `name`;
UPDATE `logical_cluster` SET `identification`=`name` WHERE id>=0;
ALTER TABLE `logical_cluster` ADD INDEX `uniq_identification` (`identification` ASC);
```
### 升级至`2.3.0`版本
#### 1.mysql变更
`2.3.0`版本在`gateway_config`表增加了一个描述说明的字段因此需要执行下面的sql进行字段的增加。
```sql
ALTER TABLE `gateway_config`
ADD COLUMN `description` TEXT NULL COMMENT '描述信息' AFTER `version`;
```

Binary file not shown.

Before

Width:  |  Height:  |  Size: 73 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 7.4 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 382 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 270 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.5 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 589 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 652 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 511 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 672 KiB

View File

@@ -1,101 +0,0 @@
---
![kafka-manager-logo](../assets/images/common/logo_name.png)
**一站式`Apache Kafka`集群指标监控与运维管控平台**
---
## JMX-连接失败问题解决
集群正常接入Logi-KafkaManager之后即可以看到集群的Broker列表此时如果查看不了Topic的实时流量或者是Broker的实时流量信息时那么大概率就是JMX连接的问题了。
下面我们按照步骤来一步一步的检查。
### 1、问题&说明
**类型一JMX配置未开启**
未开启时,直接到`2、解决方法`查看如何开启即可。
![check_jmx_opened](./assets/connect_jmx_failed/check_jmx_opened.jpg)
**类型二:配置错误**
`JMX`端口已经开启的情况下,有的时候开启的配置不正确,此时也会导致出现连接失败的问题。这里大概列举几种原因:
- `JMX`配置错误:见`2、解决方法`
- 存在防火墙或者网络限制:网络通的另外一台机器`telnet`试一下看是否可以连接上。
- 需要进行用户名及密码的认证:见`3、解决方法 —— 认证的JMX`
错误日志例子:
```
# 错误一: 错误提示的是真实的IP这样的话基本就是JMX配置的有问题了。
2021-01-27 10:06:20.730 ERROR 50901 --- [ics-Thread-1-62] c.x.k.m.c.utils.jmx.JmxConnectorWrap : JMX connect exception, host:192.168.0.1 port:9999.
java.rmi.ConnectException: Connection refused to host: 192.168.0.1; nested exception is:
# 错误二错误提示的是127.0.0.1这个IP这个是机器的hostname配置的可能有问题。
2021-01-27 10:06:20.730 ERROR 50901 --- [ics-Thread-1-62] c.x.k.m.c.utils.jmx.JmxConnectorWrap : JMX connect exception, host:127.0.0.1 port:9999.
java.rmi.ConnectException: Connection refused to host: 127.0.0.1;; nested exception is:
```
### 2、解决方法
这里仅介绍一下比较通用的解决方式,如若有更好的方式,欢迎大家指导告知一下。
修改`kafka-server-start.sh`文件:
```
# 在这个下面增加JMX端口的配置
if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
export JMX_PORT=9999 # 增加这个配置, 这里的数值并不一定是要9999
fi
```
&nbsp;
修改`kafka-run-class.sh`文件
```
# JMX settings
if [ -z "$KAFKA_JMX_OPTS" ]; then
KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=${当前机器的IP}"
fi
# JMX port to use
if [ $JMX_PORT ]; then
KAFKA_JMX_OPTS="$KAFKA_JMX_OPTS -Dcom.sun.management.jmxremote.port=$JMX_PORT -Dcom.sun.management.jmxremote.rmi.port=$JMX_PORT"
fi
```
### 3、解决方法 —— 认证的JMX
如果您是直接看的这个部分,建议先看一下上一节:`2、解决方法`以确保`JMX`的配置没有问题了。
在JMX的配置等都没有问题的情况下如果是因为认证的原因导致连接不了的此时可以使用下面介绍的方法进行解决。
**当前这块后端刚刚开发完成,可能还不够完善,有问题随时沟通。**
`Logi-KafkaManager 2.2.0+`之后的版本后端已经支持`JMX`认证方式的连接,但是还没有界面,此时我们可以往`cluster`表的`jmx_properties`字段写入`JMX`的认证信息。
这个数据是`json`格式的字符串,例子如下所示:
```json
{
"maxConn": 10, # KM对单台Broker的最大JMX连接数
"username": "xxxxx", # 用户名
"password": "xxxx", # 密码
"openSSL": true, # 开启SSL, true表示开启ssl, false表示关闭
}
```
&nbsp;
SQL的例子
```sql
UPDATE cluster SET jmx_properties='{ "maxConn": 10, "username": "xxxxx", "password": "xxxx", "openSSL": false }' where id={xxx};
```

View File

@@ -1,168 +0,0 @@
---
![kafka-manager-logo](../assets/images/common/logo_name.png)
**一站式`Apache Kafka`集群指标监控与运维管控平台**
---
# 动态配置管理
## 0、目录
- 1、Topic定时同步任务
- 2、专家服务——Topic分区热点
- 3、专家服务——Topic分区不足
- 4、专家服务——Topic资源治理
- 5、账单配置
## 1、Topic定时同步任务
### 1.1、配置的用途
`Logi-KafkaManager`在设计上,所有的资源都是挂在应用(app)下面。 如果接入的Kafka集群已经存在Topic了那么会导致这些Topic不属于任何的应用从而导致很多管理上的不便。
因此需要有一个方式将这些无主的Topic挂到某个应用下面。
这里提供了一个配置会定时自动将集群无主的Topic挂到某个应用下面下面。
### 1.2、相关实现
就是一个定时任务,该任务会定期做同步的工作。具体代码的位置在`com.xiaojukeji.kafka.manager.task.dispatch.op`包下面的`SyncTopic2DB`类。
### 1.3、配置说明
**步骤一:开启该功能**
在application.yml文件中增加如下配置已经有该配置的话直接把false修改为true即可
```yml
# 任务相关的开关
task:
op:
sync-topic-enabled: true # 无主的Topic定期同步到DB中
```
**步骤二:配置管理中指定挂在那个应用下面**
配置的位置:
![sync_topic_to_db](./assets/dynamic_config_manager/sync_topic_to_db.jpg)
配置键:`SYNC_TOPIC_2_DB_CONFIG_KEY`
配置值(JSON数组)
- clusterId需要进行定时同步的集群ID
- defaultAppId该集群无主的Topic将挂在哪个应用下面
- addAuthority是否需要加上权限, 默认是false。因为考虑到这个挂载只是临时的我们不希望用户使用这个App同时后续可能移交给真正的所属的应用因此默认是不加上权限。
**注意这里的集群ID或者是应用ID不存在的话会导致配置不生效。该任务对已经在DB中的Topic不会进行修改**
```json
[
{
"clusterId": 1234567,
"defaultAppId": "ANONYMOUS",
"addAuthority": false
},
{
"clusterId": 7654321,
"defaultAppId": "ANONYMOUS",
"addAuthority": false
}
]
```
---
## 2、专家服务——Topic分区热点
`Region`所圈定的Broker范围内某个Topic的Leader数在这些圈定的Broker上分布不均衡时我们认为该Topic是存在热点的Topic。
备注单纯的查看Leader数的分布确实存在一定的局限性这块欢迎贡献更多的热点定义于代码。
Topic分区热点相关的动态配置(页面在运维管控->平台管理->配置管理)
配置Key
```
REGION_HOT_TOPIC_CONFIG
```
配置Value
```json
{
"maxDisPartitionNum": 2, # Region内Broker间的leader数差距超过2时则认为是存在热点的Topic
"minTopicBytesInUnitB": 1048576, # 流量低于该值的Topic不做统计
"ignoreClusterIdList": [ # 忽略的集群
50
]
}
```
---
## 3、专家服务——Topic分区不足
总流量除以分区数超过指定值时则我们认为存在Topic分区不足。
Topic分区不足相关的动态配置(页面在运维管控->平台管理->配置管理)
配置Key
```
TOPIC_INSUFFICIENT_PARTITION_CONFIG
```
配置Value
```json
{
"maxBytesInPerPartitionUnitB": 3145728, # 单分区流量超过该值, 则认为分区不去
"minTopicBytesInUnitB": 1048576, # 流量低于该值的Topic不做统计
"ignoreClusterIdList": [ # 忽略的集群
50
]
}
```
## 4、专家服务——Topic资源治理
首先我们认为在一定的时间长度内Topic的分区offset没有任何变化的Topic即没有数据写入的Topic为过期的Topic。
Topic分区不足相关的动态配置(页面在运维管控->平台管理->配置管理)
配置Key
```
EXPIRED_TOPIC_CONFIG
```
配置Value
```json
{
"minExpiredDay": 30, #过期时间大于此值才显示
"ignoreClusterIdList": [ # 忽略的集群
50
]
}
```
## 5、账单配置
Logi-KafkaManager除了作为Kafka运维管控平台之外实际上还会有一些资源定价相关的功能。
当前定价方式当月Topic的maxAvgDay天的峰值的均值流量作为Topic的使用额度。使用的额度 * 单价 * 溢价(预留buffer) 就等于当月的费用。
详细的计算逻辑见com.xiaojukeji.kafka.manager.task.dispatch.biz.CalKafkaTopicBill; 和 com.xiaojukeji.kafka.manager.task.dispatch.biz.CalTopicStatistics;
这块在计算Topic的费用的配置如下所示
配置Key
```
KAFKA_TOPIC_BILL_CONFIG
```
配置Value
```json
{
"maxAvgDay": 10, # 使用额度的计算规则
"quotaRatio": 1.5, # 溢价率
"priseUnitMB": 100 # 单价即单MB/s流量多少钱
}
```

View File

@@ -1,10 +0,0 @@
---
![kafka-manager-logo](../assets/images/common/logo_name.png)
**一站式`Apache Kafka`集群指标监控与运维管控平台**
---
# Kafka-Gateway 配置说明

View File

@@ -1,42 +0,0 @@
---
![kafka-manager-logo](../assets/images/common/logo_name.png)
**一站式`Apache Kafka`集群指标监控与运维管控平台**
---
# 监控系统集成——夜莺
- `Kafka-Manager`通过将 监控的数据 以及 监控的规则 都提交给夜莺,然后依赖夜莺的监控系统从而实现监控告警功能。
- 监控数据上报 & 告警规则的创建等能力已经具备。但类似查看告警历史,告警触发时的监控数据等正在集成中(暂时可以到夜莺系统进行查看),欢迎有兴趣的同学进行共建 或 贡献代码。
## 1、配置说明
```yml
# 配置文件中关于监控部分的配置
monitor:
enabled: false
n9e:
nid: 2
user-token: 123456
# 夜莺 mon监控服务 地址
mon:
base-url: http://127.0.0.1:8006
# 夜莺 transfer上传服务 地址
sink:
base-url: http://127.0.0.1:8008
# 夜莺 rdb资源服务 地址
rdb:
base-url: http://127.0.0.1:80
# enabled: 表示是否开启监控告警的功能, true: 开启, false: 不开启
# n9e.nid: 夜莺的节点ID
# n9e.user-token: 用户的密钥,在夜莺的个人设置中
# n9e.mon.base-url: 监控地址
# n9e.sink.base-url: 数据上报地址
# n9e.rdb.base-url: 用户资源中心地址
```

View File

@@ -1,54 +0,0 @@
---
![kafka-manager-logo](../assets/images/common/logo_name.png)
**一站式`Apache Kafka`集群指标监控与运维管控平台**
---
# 监控系统集成
- 监控系统默认与 [夜莺] (https://github.com/didi/nightingale) 进行集成;
- 对接自有的监控系统需要进行简单的二次开发,即实现部分监控告警模块的相关接口即可;
- 集成会有两块内容,一个是指标数据上报的集成,还有一个是监控告警规则的集成;
## 1、指标数据上报集成
仅完成这一步的集成之后,即可将监控数据上报到监控系统中,此时已能够在自己的监控系统进行监控告警规则的配置了。
**步骤一:实现指标上报的接口**
- 按照自己内部监控系统的数据格式要求,将数据进行组装成符合自己内部监控系统要求的数据进行上报,具体的可以参考夜莺集成的实现代码。
- 至于会上报哪些指标,可以查看有哪些地方调用了该接口。
![sink_metrics](./assets/monitor_system_integrate_with_self/sink_metrics.jpg)
**步骤二:相关配置修改**
![change_config](./assets/monitor_system_integrate_with_self/change_config.jpg)
**步骤三:开启上报任务**
![open_sink_schedule](./assets/monitor_system_integrate_with_self/open_sink_schedule.jpg)
## 2、监控告警规则集成
完成**1、指标数据上报集成**之后,即可在自己的监控系统进行监控告警规则的配置了。完成该步骤的集成之后,可以在`Logi-KafkaManager`中进行监控告警规则的增删改查等等。
大体上和**1、指标数据上报集成**一致,
**步骤一:实现相关接口**
![integrate_ms](./assets/monitor_system_integrate_with_self/integrate_ms.jpg)
实现完成步骤一之后,接下来的步骤和**1、指标数据上报集成**中的步骤二、步骤三一致,都需要进行相关配置的修改即可。
## 3、总结
简单介绍了一下监控告警的集成,嫌麻烦的同学可以仅做 **1、指标数据上报集成** 这一节的内容即可满足一定场景下的需求。
**集成过程中有任何觉得文档没有说清楚的地方或者建议欢迎入群交流也欢迎贡献代码觉得好也辛苦给个star。**

View File

@@ -1,27 +0,0 @@
---
![kafka-manager-logo](../../assets/images/common/logo_name.png)
**一站式`Apache Kafka`集群指标监控与运维管控平台**
---
# 升级至`2.2.0`版本
`2.2.0`版本在`cluster`表及`logical_cluster`各增加了一个字段因此需要执行下面的sql进行字段的增加。
```sql
# cluster表中增加jmx_properties字段, 这个字段会用于存储jmx相关的认证以及配置信息
ALTER TABLE `cluster` ADD COLUMN `jmx_properties` TEXT NULL COMMENT 'JMX配置' AFTER `security_properties`;
# logical_cluster中增加identification字段, 同时数据和原先name数据相同, 最后增加一个唯一键.
# 此后, name字段还是表示集群名称, identification字段表示的是集群标识, 只能是字母数字及下划线组成,
# 数据上报到监控系统时, 集群这个标识采用的字段就是identification字段, 之前使用的是name字段.
ALTER TABLE `logical_cluster` ADD COLUMN `identification` VARCHAR(192) NOT NULL DEFAULT '' COMMENT '逻辑集群标识' AFTER `name`;
UPDATE `logical_cluster` SET `identification`=`name` WHERE id>=0;
ALTER TABLE `logical_cluster` ADD INDEX `uniq_identification` (`identification` ASC);
```

View File

@@ -1,17 +0,0 @@
---
![kafka-manager-logo](../../assets/images/common/logo_name.png)
**一站式`Apache Kafka`集群指标监控与运维管控平台**
---
# 升级至`2.3.0`版本
`2.3.0`版本在`gateway_config`表增加了一个描述说明的字段因此需要执行下面的sql进行字段的增加。
```sql
ALTER TABLE `gateway_config`
ADD COLUMN `description` TEXT NULL COMMENT '描述信息' AFTER `version`;
```

View File

@@ -1,41 +0,0 @@
---
![kafka-manager-logo](../assets/images/common/logo_name.png)
**一站式`Apache Kafka`集群指标监控与运维管控平台**
---
# 使用`MySQL 8`
感谢 [herry-hu](https://github.com/herry-hu) 提供的方案。
当前因为无法同时兼容`MySQL 8``MySQL 5.7`,因此代码中默认的版本还是`MySQL 5.7`
当前如需使用`MySQL 8`,则需按照下述流程进行简单修改代码。
- Step1. 修改application.yml中的MySQL驱动类
```shell
# 将driver-class-name后面的驱动类修改为:
# driver-class-name: com.mysql.jdbc.Driver
driver-class-name: com.mysql.cj.jdbc.Driver
```
- Step2. 修改MySQL依赖包
```shell
# 将根目录下面的pom.xml文件依赖的`MySQL`依赖包版本调整为
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
# <version>5.1.41</version>
<version>8.0.20</version>
</dependency>
```

View File

@@ -1,53 +0,0 @@
---
![kafka-manager-logo](../assets/images/common/logo_name.png)
**一站式`Apache Kafka`集群指标监控与运维管控平台**
---
# 如何增加上报监控系统指标?
## 0、前言
LogiKM是 **一站式`Apache Kafka`集群指标监控与运维管控平台** 当前会将消费LagTopic流量等指标上报到监控系统中从而方便用户在监控系统中对这些指标配置监控告警规则进而达到监控自身客户端是否正常的目的。
那么如果我们想增加一个新的监控指标应该如何做呢比如我们想监控Broker的流量监控Broker的存活信息监控集群Controller个数等等。
在具体介绍之前我们大家都知道Kafka监控相关的信息基本都存储于Broker、Jmx以及ZK中。当前LogiKM也已经具备从这三个地方获取数据的基本能力因此基于LogiKM我们再获取其他指标总体上还是非常方便的。
这里我们就以已经获取到的Topic流量信息为例看LogiKM如何实现Topic指标的获取并上报的。
---
## 1、确定指标位置
基于对Kafka的了解我们知道Topic流量信息这个指标是存储于Jmx中的因此我们需要从Jmx中获取。大家如果对于自己所需要获取的指标存储在何处不太清楚的可以加入我们维护的Kafka中文社区(README中有二维码)中今天沟通交流。
---
## 2、指标获取
Topic流量指标的获取详细见图中说明。
![Topic流量指标采集说明](./assets/increase_the_indicators_reported_to_monitor_system/collect_topic_metrics.jpg)
---
## 3、指标上报
上一步我们已经采集到Topic流量指标了下一步就是将该指标上报到监控系统这块只需要按照监控系统要求的格式将数据上报即可。
LogiKM中有一个monitor模块具体的如下图所示
![指标上报](./assets/increase_the_indicators_reported_to_monitor_system/sink_metrcis.png)
## 4、补充说明
监控系统对接的相关内容见:
[监控系统集成](./monitor_system_integrate_with_self.md)
[监控系统集成例子——集成夜莺](./monitor_system_integrate_with_n9e.md)

View File

@@ -1,107 +0,0 @@
---
![kafka-manager-logo](../assets/images/common/logo_name.png)
**一站式`Apache Kafka`集群指标监控与运维管控平台**
---
# 配置说明
```yaml
server:
port: 8080 # 服务端口
tomcat:
accept-count: 1000
max-connections: 10000
max-threads: 800
min-spare-threads: 100
spring:
application:
name: kafkamanager
datasource:
kafka-manager: # 数据库连接配置
jdbc-url: jdbc:mysql://127.0.0.1:3306/kafka_manager?characterEncoding=UTF-8&serverTimezone=GMT%2B8 #数据库的地址
username: admin # 用户名
password: admin # 密码
driver-class-name: com.mysql.jdbc.Driver
main:
allow-bean-definition-overriding: true
profiles:
active: dev # 启用的配置
servlet:
multipart:
max-file-size: 100MB
max-request-size: 100MB
logging:
config: classpath:logback-spring.xml
custom:
idc: cn # 部署的数据中心, 忽略该配置, 后续会进行删除
jmx:
max-conn: 10 # 和单台 broker 的最大JMX连接数
store-metrics-task:
community:
broker-metrics-enabled: true # 社区部分broker metrics信息收集开关, 关闭之后metrics信息将不会进行收集及写DB
topic-metrics-enabled: true # 社区部分topic的metrics信息收集开关, 关闭之后metrics信息将不会进行收集及写DB
didi:
app-topic-metrics-enabled: false # 滴滴埋入的指标, 社区AK不存在该指标因此默认关闭
topic-request-time-metrics-enabled: false # 滴滴埋入的指标, 社区AK不存在该指标因此默认关闭
topic-throttled-metrics: false # 滴滴埋入的指标, 社区AK不存在该指标因此默认关闭
save-days: 7 #指标在DB中保持的天数-1表示永久保存7表示保存近7天的数据
# 任务相关的开关
task:
op:
sync-topic-enabled: false # 未落盘的Topic定期同步到DB中
order-auto-exec: # 工单自动化审批线程的开关
topic-enabled: false # Topic工单自动化审批开关, false:关闭自动化审批, true:开启
app-enabled: false # App工单自动化审批开关, false:关闭自动化审批, true:开启
account: # ldap相关的配置, 社区版本暂时支持不够完善,可以先忽略,欢迎贡献代码对这块做优化
ldap:
kcm: # 集群升级部署相关的功能需要配合夜莺及S3进行使用这块我们后续专门补充一个文档细化一下牵扯到kcm_script.sh脚本的修改
enabled: false # 默认关闭
storage:
base-url: http://127.0.0.1 # 存储地址
n9e:
base-url: http://127.0.0.1:8004 # 夜莺任务中心的地址
user-token: 12345678 # 夜莺用户的token
timeout: 300 # 集群任务的超时时间,单位秒
account: root # 集群任务使用的账号
script-file: kcm_script.sh # 集群任务的脚本
monitor: # 监控告警相关的功能,需要配合夜莺进行使用
enabled: false # 默认关闭true就是开启
n9e:
nid: 2
user-token: 1234567890
mon:
# 夜莺 mon监控服务 地址
base-url: http://127.0.0.1:8032
sink:
# 夜莺 transfer上传服务 地址
base-url: http://127.0.0.1:8006
rdb:
# 夜莺 rdb资源服务 地址
base-url: http://127.0.0.1:80
# enabled: 表示是否开启监控告警的功能, true: 开启, false: 不开启
# n9e.nid: 夜莺的节点ID
# n9e.user-token: 用户的密钥,在夜莺的个人设置中
# n9e.mon.base-url: 监控地址
# n9e.sink.base-url: 数据上报地址
# n9e.rdb.base-url: 用户资源中心地址
notify: # 通知的功能
kafka: # 默认通知发送到kafka的指定Topic中
cluster-id: 95 # Topic的集群ID
topic-name: didi-kafka-notify # Topic名称
order: # 部署的KM的地址
detail-url: http://127.0.0.1
```

View File

@@ -1,93 +0,0 @@
---
![kafka-manager-logo](../assets/images/common/logo_name.png)
**一站式`Apache Kafka`集群指标监控与运维管控平台**
---
# 安装手册
## 1、环境依赖
如果是以Release包进行安装的则仅安装`Java``MySQL`即可。如果是要先进行源码包进行打包,然后再使用,则需要安装`Maven``Node`环境。
- `Java 8+`(运行环境需要)
- `MySQL 5.7`(数据存储)
- `Maven 3.5+`(后端打包依赖)
- `Node 10+`(前端打包依赖)
---
## 2、获取安装包
**1、Release直接下载**
这里如果觉得麻烦然后也不想进行二次开发则可以直接下载Release包下载地址[Github Release包下载地址](https://github.com/didi/Logi-KafkaManager/releases)
如果觉得Github的下载地址太慢了也可以进入`Logi-KafkaManager`的用户群获取群地址在README中。
**2、源代码进行打包**
下载好代码之后,进入`Logi-KafkaManager`的主目录,执行`mvn -Prelease-kafka-manager -Dmaven.test.skip=true clean install -U `命令即可,
执行完成之后会在`distribution/target`目录下面生成一个`kafka-manager-*.tar.gz`
和一个`kafka-manager-*.zip` 文件,随便任意一个压缩包都可以;
当然此时同级目录有一个已经解压好的文件夹;
---
## 3. 解压安装包
解压完成后; 在文件目录中可以看到有`kafka-manager/conf/create_mysql_table.sql` 有个mysql初始化文件
先初始化DB
## 4、MySQL-DB初始化
执行[create_mysql_table.sql](../../distribution/conf/create_mysql_table.sql)中的SQL命令从而创建所需的MySQL库及表默认创建的库名是`logi_kafka_manager`
```
# 示例:
mysql -uXXXX -pXXX -h XXX.XXX.XXX.XXX -PXXXX < ./create_mysql_table.sql
```
---
## 5.修该配置
请将`conf/application.yml.example` 文件复制一份出来命名为`application.yml` 放在同级目录:conf/application.yml ;
并且修改配置; 当然不修改的话 就会用默认的配置;
至少 mysql配置成自己的吧
## 6、启动/关闭
解压包中有启动和关闭脚本
`kafka-manager/bin/shutdown.sh`
`kafka-manager/bin/startup.sh`
执行 sh startup.sh 启动
执行 sh shutdown.sh 关闭
### 6、使用
本地启动的话,访问`http://localhost:8080`,输入帐号及密码(默认`admin/admin`)进行登录。更多参考:[kafka-manager 用户使用手册](../user_guide/user_guide_cn.md)
### 7. 升级
如果是升级版本,请查看文件 [kafka-manager 升级手册](../../distribution/upgrade_config.md)
在您下载的启动包(V2.5及其后)中也有记录,在 kafka-manager/upgrade_config.md 中
### 8. 在IDE中启动
> 如果想参与开发或者想在IDE中启动的话
> 先执行 `mvn -Dmaven.test.skip=true clean install -U `
>
> 然后这个时候可以选择去 [pom.xml](../../pom.xml) 中将`kafka-manager-console`模块注释掉;
> 注释是因为每次install的时候都会把前端文件`kafka-manager-console`重新打包进`kafka-manager-web`
>
> 完事之后,只需要直接用IDE启动运行`kafka-manager-web`模块中的
> com.xiaojukeji.kafka.manager.web.MainApplication main方法就行了

View File

@@ -1,94 +0,0 @@
---
![kafka-manager-logo](../assets/images/common/logo_name.png)
**一站式`Apache Kafka`集群指标监控与运维管控平台**
---
## nginx配置-安装手册
# 一、独立部署
请参考参考:[kafka-manager 安装手册](install_guide_cn.md)
# 二、nginx配置
## 1、独立部署配置
```
#nginx 根目录访问配置如下
location / {
proxy_pass http://ip:port;
}
```
## 2、前后端分离&配置多个静态资源
以下配置解决`nginx代理多个静态资源`,实现项目前后端分离,版本更新迭代。
### 1、源码下载
根据所需版本下载对应代码,下载地址:[Github 下载地址](https://github.com/didi/Logi-KafkaManager)
### 2、修改webpack.config.js 配置文件
修改`kafka-manager-console`模块 `webpack.config.js`
以下所有<font color='red'>xxxx</font>为nginx代理路径和打包静态文件加载前缀,<font color='red'>xxxx</font>可根据需求自行更改。
```
cd kafka-manager-console
vi webpack.config.js
# publicPath默认打包方式根目录下修改为nginx代理访问路径。
let publicPath = '/xxxx';
```
### 3、打包
```
npm cache clean --force && npm install
```
ps如果打包过程中报错运行`npm install clipboard@2.0.6`,相反请忽略!
### 4、部署
#### 1、前段静态文件部署
静态资源 `../kafka-manager-web/src/main/resources/templates`
上传到指定目录,目前以`root目录`做demo
#### 2、上传jar包并启动请参考[kafka-manager 安装手册](install_guide_cn.md)
#### 3、修改nginx 配置
```
location /xxxx {
# 静态文件存放位置
alias /root/templates;
try_files $uri $uri/ /xxxx/index.html;
index index.html;
}
location /api {
proxy_pass http://ip:port;
}
#后代端口建议使用/api如果冲突可以使用以下配置
#location /api/v2 {
# proxy_pass http://ip:port;
#}
#location /api/v1 {
# proxy_pass http://ip:port;
#}
```

View File

@@ -1,49 +0,0 @@
---
![kafka-manager-logo](../../assets/images/common/logo_name.png)
**一站式`Apache Kafka`集群指标监控与运维管控平台**
---
# 集群接入
## 主要概念讲解
面对大规模集群、业务场景复杂的情况引入Region、逻辑集群的概念
- Region划分部分Broker作为一个 Region用Region定义资源划分的单位提高扩展性和隔离性。如果部分Topic异常也不会影响大面积的Broker
- 逻辑集群逻辑集群由部分Region组成便于对大规模集群按照业务划分、保障能力进行管理
![op_cluster_arch](assets/op_cluster_arch.png)
集群的接入总共需要三个步骤,分别是:
1. 接入物理集群:填写机器地址、安全协议等配置信息,接入真实的物理集群
2. 创建Region将部分Broker划分为一个Region
3. 创建逻辑集群逻辑集群由部分Region组成可根据业务划分、保障等级来创建相应的逻辑集群
![op_cluster_flow](assets/op_cluster_flow.png)
**备注接入集群需要2、3两步是因为普通用户的视角下看到的都是逻辑集群如果没有2、3两步那么普通用户看不到任何信息。**
## 1、接入物理集群
![op_add_cluster](assets/op_add_cluster.jpg)
如上图所示,填写集群信息,然后点击确定,即可完成集群的接入。因为考虑到分布式部署,添加集群之后,需要稍等**`1分钟`**才可以在界面上看到集群的详细信息。
## 2、创建Region
![op_add_region](assets/op_add_region.jpg)
如上图所示填写Region信息然后点击确定即可完成Region的创建。
备注Region即为Broker的集合可以按照业务需要将Broker归类从而创建相应的Region。
## 3、创建逻辑集群
![op_add_logical_cluster](assets/op_add_logical_cluster.jpg)
如上图所示,填写逻辑集群信息,然后点击确定,即可完成逻辑集群的创建。

Binary file not shown.

Before

Width:  |  Height:  |  Size: 261 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 240 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 195 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 124 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 105 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 94 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 181 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 65 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 166 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 78 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 48 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 55 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 297 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 189 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 173 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 197 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 244 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 118 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 150 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 177 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 276 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 257 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 153 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 189 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 187 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 92 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 116 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 166 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 158 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 124 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 209 KiB

Some files were not shown because too many files have changed in this diff Show More