mirror of
https://github.com/didi/KnowStreaming.git
synced 2025-12-24 20:22:12 +08:00
Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
f8ca797f16 |
100
README.md
100
README.md
@@ -5,94 +5,60 @@
|
|||||||
|
|
||||||
**一站式`Apache Kafka`集群指标监控与运维管控平台**
|
**一站式`Apache Kafka`集群指标监控与运维管控平台**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 主要功能特性
|
||||||
|
|
||||||
|
|
||||||
阅读本README文档，您可以了解到滴滴Logi-KafkaManager的用户群体、产品定位等信息，并通过体验地址，快速体验Kafka集群指标监控与运维管控的全流程。<br>若滴滴Logi-KafkaManager已在贵司的生产环境进行使用，并想要获得官方更好地支持和指导，可以通过[`OCE认证`](http://obsuite.didiyun.com/open/openAuth)，加入官方交流平台。
|
### 集群监控维度
|
||||||
|
|
||||||
|
- 多版本集群管控，支持从`0.10.2`到`2.x`版本；
|
||||||
|
- 集群Topic、Broker等多维度历史与实时关键指标查看；
|
||||||
|
|
||||||
|
|
||||||
## 1 产品简介
|
### 集群管控维度
|
||||||
滴滴Logi-KafkaManager脱胎于滴滴内部多年的Kafka运营实践经验，是面向Kafka用户、Kafka运维人员打造的共享多租户Kafka云平台。专注于Kafka运维管控、监控告警、资源治理等核心场景，经历过大规模集群、海量大数据的考验。内部满意度高达90%的同时，还与多家知名企业达成商业化合作。
|
|
||||||
|
|
||||||
### 1.1 快速体验地址
|
- 集群运维，包括逻辑Region方式管理集群
|
||||||
- 体验地址 http://117.51.146.109:8080 账号密码 admin/admin
|
- Broker运维，包括优先副本选举
|
||||||
|
- Topic运维，包括创建、查询、扩容、修改属性、数据采样及迁移等；
|
||||||
### 1.2 体验地图
|
- 消费组运维，包括指定时间或指定偏移两种方式进行重置消费偏移
|
||||||
相比较于同类产品的用户视角单一（大多为管理员视角），滴滴Logi-KafkaManager建立了基于分角色、多场景视角的体验地图。分别是：**用户体验地图、运维体验地图、运营体验地图**
|
|
||||||
|
|
||||||
#### 1.2.1 用户体验地图
|
|
||||||
- 平台租户申请 ：申请应用（App）作为Kafka中的用户名，并用 AppID+password作为身份验证
|
|
||||||
- 集群资源申请 ：按需申请、按需使用。可使用平台提供的共享集群，也可为应用申请独立的集群
|
|
||||||
- Topic 申 请 ：可根据应用（App）创建Topic，或者申请其他topic的读写权限
|
|
||||||
- Topic 运 维 ：Topic数据采样、调整配额、申请分区等操作
|
|
||||||
- 指 标 监 控 ：基于Topic生产消费各环节耗时统计，监控不同分位数性能指标
|
|
||||||
- 消 费 组 运 维 ：支持将消费偏移重置至指定时间或指定位置
|
|
||||||
|
|
||||||
#### 1.2.2 运维体验地图
|
|
||||||
- 多版本集群管控 ：支持从`0.10.2`到`2.x`版本
|
|
||||||
- 集 群 监 控 ：集群Topic、Broker等多维度历史与实时关键指标查看，建立健康分体系
|
|
||||||
- 集 群 运 维 ：划分部分Broker作为Region，使用Region定义资源划分单位，并按照业务、保障能力区分逻辑集群
|
|
||||||
- Broker 运 维 ：包括优先副本选举等操作
|
|
||||||
- Topic 运 维 ：包括创建、查询、扩容、修改属性、迁移、下线等
|
|
||||||
|
|
||||||
|
|
||||||
#### 1.2.3 运营体验地图
|
### 用户使用维度
|
||||||
- 资 源 治 理 ：沉淀资源治理方法。针对Topic分区热点、分区不足等高频常见问题，沉淀资源治理方法，实现资源治理专家化
|
|
||||||
- 资 源 审 批 ：工单体系。Topic创建、调整配额、申请分区等操作，由专业运维人员审批，规范资源使用，保障平台平稳运行
|
|
||||||
- 账 单 体 系 ：成本控制。Topic资源、集群资源按需申请、按需使用。根据流量核算费用，帮助企业建设大数据成本核算体系
|
|
||||||
|
|
||||||
### 1.3 核心优势
|
- Kafka用户、Kafka研发、Kafka运维 视角区分
|
||||||
- 高 效 的 问 题 定 位 ：监控多项核心指标，统计不同分位数据，提供种类丰富的指标监控报表，帮助用户、运维人员快速高效定位问题
|
- Kafka用户、Kafka研发、Kafka运维 权限区分
|
||||||
- 便 捷 的 集 群 运 维 ：按照Region定义集群资源划分单位，将逻辑集群根据保障等级划分。在方便资源隔离、提高扩展能力的同时，实现对服务端的强管控
|
|
||||||
- 专 业 的 资 源 治 理 ：基于滴滴内部多年运营实践，沉淀资源治理方法，建立健康分体系。针对Topic分区热点、分区不足等高频常见问题，实现资源治理专家化
|
|
||||||
- 友 好 的 运 维 生 态 ：与滴滴夜莺监控告警系统打通，集成监控告警、集群部署、集群升级等能力。形成运维生态，凝练专家服务，使运维更高效
|
|
||||||
|
|
||||||
### 1.4 滴滴Logi-KafkaManager架构图
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## 2 相关文档
|
## kafka-manager架构图
|
||||||
|
|
||||||
### 2.1 产品文档
|
|
||||||
- [滴滴Logi-KafkaManager 安装手册](docs/install_guide/install_guide_cn.md)
|
|
||||||
- [滴滴Logi-KafkaManager 接入集群](docs/user_guide/add_cluster/add_cluster.md)
|
|
||||||
- [滴滴Logi-KafkaManager 用户使用手册](docs/user_guide/user_guide_cn.md)
|
|
||||||
- [滴滴Logi-KafkaManager FAQ](docs/user_guide/faq.md)
|
|
||||||
|
|
||||||
### 2.2 社区文章
|
|
||||||
- [滴滴云官网产品介绍](https://www.didiyun.com/production/logi-KafkaManager.html)
|
|
||||||
- [7年沉淀之作--滴滴Logi日志服务套件](https://mp.weixin.qq.com/s/-KQp-Qo3WKEOc9wIR2iFnw)
|
|
||||||
- [滴滴Logi-KafkaManager 一站式Kafka监控与管控平台](https://mp.weixin.qq.com/s/9qSZIkqCnU6u9nLMvOOjIQ)
|
|
||||||
- [滴滴Logi-KafkaManager 开源之路](https://xie.infoq.cn/article/0223091a99e697412073c0d64)
|
|
||||||
- [滴滴Logi-KafkaManager 系列视频教程](https://mp.weixin.qq.com/s/9X7gH0tptHPtfjPPSdGO8g)
|
|
||||||
- [kafka实践（十五）：滴滴开源Kafka管控平台 Logi-KafkaManager研究--A叶子叶来](https://blog.csdn.net/yezonggang/article/details/113106244)
|
|
||||||
|
|
||||||
## 3 滴滴Logi开源用户交流群
|
|
||||||
|
|
||||||
|
|
||||||
|
## 相关文档
|
||||||
微信加群：关注公众号 Obsuite(官方公众号) 回复 "Logi加群"
|
|
||||||
|
|
||||||
|
- [kafka-manager 安装手册](docs/install_guide/install_guide_cn.md)
|
||||||
钉钉群ID：32821440
|
- [kafka-manager 接入集群](docs/user_guide/add_cluster/add_cluster.md)
|
||||||
|
- [kafka-manager 用户使用手册](docs/user_guide/user_guide_cn.md)
|
||||||
|
- [kafka-manager FAQ](docs/user_guide/faq.md)
|
||||||
|
|
||||||
|
## 钉钉交流群
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## 4 OCE认证
|
## 项目成员
|
||||||
OCE是一个认证机制和交流平台，为滴滴Logi-KafkaManager生产用户量身打造，我们会为OCE企业提供更好的技术支持，比如专属的技术沙龙、企业一对一的交流机会、专属的答疑群等，如果贵司Logi-KafkaManager上了生产，[快来加入吧](http://obsuite.didiyun.com/open/openAuth)
|
|
||||||
|
### 内部核心人员
|
||||||
|
|
||||||
|
`iceyuhui`、`liuyaguang`、`limengmonty`、`zhangliangmike`、`nullhuangyiming`、`zengqiao`、`eilenexuzhe`、`huangjiaweihjw`
|
||||||
|
|
||||||
|
|
||||||
## 5 项目成员
|
### 外部贡献者
|
||||||
|
|
||||||
### 5.1 内部核心人员
|
|
||||||
|
|
||||||
`iceyuhui`、`liuyaguang`、`limengmonty`、`zhangliangmike`、`nullhuangyiming`、`zengqiao`、`eilenexuzhe`、`huangjiaweihjw`、`zhaoyinrui`、`marzkonglingxu`、`joysunchao`
|
|
||||||
|
|
||||||
|
|
||||||
### 5.2 外部贡献者
|
|
||||||
|
|
||||||
`fangjunyu`、`zhoutaiyang`
|
`fangjunyu`、`zhoutaiyang`
|
||||||
|
|
||||||
|
|
||||||
## 6 协议
|
## 协议
|
||||||
|
|
||||||
`kafka-manager`基于`Apache-2.0`协议进行分发和使用，更多信息参见[协议文件](./LICENSE)
|
`kafka-manager`基于`Apache-2.0`协议进行分发和使用，更多信息参见[协议文件](./LICENSE)
|
||||||
|
|||||||
@@ -1,97 +0,0 @@
|
|||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
**一站式`Apache Kafka`集群指标监控与运维管控平台**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## v2.3.0
|
|
||||||
|
|
||||||
版本上线时间：2021-02-08
|
|
||||||
|
|
||||||
|
|
||||||
### 能力提升
|
|
||||||
|
|
||||||
- 新增支持docker化部署
|
|
||||||
- 可指定Broker作为候选controller
|
|
||||||
- 可新增并管理网关配置
|
|
||||||
- 可获取消费组状态
|
|
||||||
- 增加集群的JMX认证
|
|
||||||
|
|
||||||
### 体验优化
|
|
||||||
|
|
||||||
- 优化编辑用户角色、修改密码的流程
|
|
||||||
- 新增consumerID的搜索功能
|
|
||||||
- 优化“Topic连接信息”、“消费组重置消费偏移”、“修改Topic保存时间”的文案提示
|
|
||||||
- 在相应位置增加《资源申请文档》链接
|
|
||||||
|
|
||||||
### bug修复
|
|
||||||
|
|
||||||
- 修复Broker监控图表时间轴展示错误的问题
|
|
||||||
- 修复创建夜莺监控告警规则时，使用的告警周期的单位不正确的问题
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## v2.2.0
|
|
||||||
|
|
||||||
版本上线时间：2021-01-25
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### 能力提升
|
|
||||||
|
|
||||||
- 优化工单批量操作流程
|
|
||||||
- 增加获取Topic75分位/99分位的实时耗时数据
|
|
||||||
- 增加定时任务，可将无主未落DB的Topic定期写入DB
|
|
||||||
|
|
||||||
### 体验优化
|
|
||||||
|
|
||||||
- 在相应位置增加《集群接入文档》链接
|
|
||||||
- 优化物理集群、逻辑集群含义
|
|
||||||
- 在Topic详情页、Topic扩分区操作弹窗增加展示Topic所属Region的信息
|
|
||||||
- 优化Topic审批时，Topic数据保存时间的配置流程
|
|
||||||
- 优化Topic/应用申请、审批时的错误提示文案
|
|
||||||
- 优化Topic数据采样的操作项文案
|
|
||||||
- 优化运维人员删除Topic时的提示文案
|
|
||||||
- 优化运维人员删除Region的删除逻辑与提示文案
|
|
||||||
- 优化运维人员删除逻辑集群的提示文案
|
|
||||||
- 优化上传集群配置文件时的文件类型限制条件
|
|
||||||
|
|
||||||
### bug修复
|
|
||||||
|
|
||||||
- 修复填写应用名称时校验特殊字符出错的问题
|
|
||||||
- 修复普通用户越权访问应用详情的问题
|
|
||||||
- 修复由于Kafka版本升级，导致的数据压缩格式无法获取的问题
|
|
||||||
- 修复删除逻辑集群或Topic之后，界面依旧展示的问题
|
|
||||||
- 修复进行Leader rebalance操作时执行结果重复提示的问题
|
|
||||||
|
|
||||||
|
|
||||||
## v2.1.0
|
|
||||||
|
|
||||||
版本上线时间：2020-12-19
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
### 体验优化
|
|
||||||
|
|
||||||
- 优化页面加载时的背景样式
|
|
||||||
- 优化普通用户申请Topic权限的流程
|
|
||||||
- 优化Topic申请配额、申请分区的权限限制
|
|
||||||
- 优化取消Topic权限的文案提示
|
|
||||||
- 优化申请配额表单的表单项名称
|
|
||||||
- 优化重置消费偏移的操作流程
|
|
||||||
- 优化创建Topic迁移任务的表单内容
|
|
||||||
- 优化Topic扩分区操作的弹窗界面样式
|
|
||||||
- 优化集群Broker监控可视化图表样式
|
|
||||||
- 优化创建逻辑集群的表单内容
|
|
||||||
- 优化集群安全协议的提示文案
|
|
||||||
|
|
||||||
### bug修复
|
|
||||||
|
|
||||||
- 修复偶发性重置消费偏移失败的问题
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
65
build.sh
65
build.sh
@@ -3,53 +3,72 @@ workspace=$(cd $(dirname $0) && pwd -P)
|
|||||||
cd $workspace
|
cd $workspace
|
||||||
|
|
||||||
## constant
|
## constant
|
||||||
OUTPUT_DIR=./output
|
app_name=kafka-manager
|
||||||
KM_VERSION=2.4.0
|
output_dir=output
|
||||||
APP_NAME=kafka-manager
|
|
||||||
APP_DIR=${APP_NAME}-${KM_VERSION}
|
|
||||||
|
|
||||||
MYSQL_TABLE_SQL_FILE=./docs/install_guide/create_mysql_table.sql
|
gitversion=.gitversion
|
||||||
CONFIG_FILE=./kafka-manager-web/src/main/resources/application.yml
|
control=./control.sh
|
||||||
|
create_mysql_table=./docs/install_guide/create_mysql_table.sql
|
||||||
|
app_config_file=./kafka-manager-web/src/main/resources/application.yml
|
||||||
|
|
||||||
## function
|
## function
|
||||||
function build() {
|
function build() {
|
||||||
|
# 进行编译
|
||||||
|
# # cmd 设置使用的JDK, 按需选择, 默认已安装了JDK 8
|
||||||
|
# JVERSION=`java -version 2>&1 | awk 'NR==1{gsub(/"/,"");print $3}'`
|
||||||
|
# major=`echo $JVERSION | awk -F. '{print $1}'`
|
||||||
|
# mijor=`echo $JVERSION | awk -F. '{print $2}'`
|
||||||
|
# if [ $major -le 1 ] && [ $mijor -lt 8 ]; then
|
||||||
|
# export JAVA_HOME=/usr/local/jdk1.8.0_65 #(使用jdk8请设置)
|
||||||
|
# export PATH=$JAVA_HOME/bin:$PATH
|
||||||
|
# fi
|
||||||
|
|
||||||
# 编译命令
|
# 编译命令
|
||||||
mvn -U clean package -Dmaven.test.skip=true
|
mvn -U clean package -Dmaven.test.skip=true
|
||||||
|
|
||||||
local sc=$?
|
local sc=$?
|
||||||
if [ $sc -ne 0 ];then
|
if [ $sc -ne 0 ];then
|
||||||
## 编译失败, 退出码为 非0
|
## 编译失败, 退出码为 非0
|
||||||
echo "$APP_NAME build error"
|
echo "$app_name build error"
|
||||||
exit $sc
|
exit $sc
|
||||||
else
|
else
|
||||||
echo "$APP_NAME build ok"
|
echo -n "$app_name build ok, vsn="`gitversion`
|
||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
function make_output() {
|
function make_output() {
|
||||||
# 新建output目录
|
# 新建output目录
|
||||||
rm -rf ${OUTPUT_DIR} &>/dev/null
|
rm -rf $output_dir &>/dev/null
|
||||||
mkdir -p ${OUTPUT_DIR}/${APP_DIR} &>/dev/null
|
mkdir -p $output_dir &>/dev/null
|
||||||
|
|
||||||
# 填充output目录, output内的内容
|
# 填充output目录, output内的内容 即为 线上部署内容
|
||||||
(
|
(
|
||||||
cp -rf ${MYSQL_TABLE_SQL_FILE} ${OUTPUT_DIR}/${APP_DIR} && # 拷贝 sql 初始化脚本 至output目录
|
# cp -rf $control $output_dir && # 拷贝 control.sh 脚本 至output目录
|
||||||
cp -rf ${CONFIG_FILE} ${OUTPUT_DIR}/${APP_DIR} && # 拷贝 application.yml 至output目录
|
cp -rf $create_mysql_table $output_dir && # 拷贝 sql 初始化脚本 至output目录
|
||||||
|
cp -rf $app_config_file $output_dir && # 拷贝 application.yml 至output目录
|
||||||
|
|
||||||
# 拷贝程序包到output路径
|
# 拷贝程序包到output路径
|
||||||
cp kafka-manager-web/target/kafka-manager-web-${KM_VERSION}-SNAPSHOT.jar ${OUTPUT_DIR}/${APP_DIR}/${APP_NAME}.jar
|
cp kafka-manager-web/target/${app_name}-*-SNAPSHOT.jar ${output_dir}/${app_name}.jar
|
||||||
echo -e "make output ok."
|
echo -e "make output ok."
|
||||||
) || { echo -e "make output error"; exit 2; } # 填充output目录失败后, 退出码为 非0
|
) || { echo -e "make output error"; exit 2; } # 填充output目录失败后, 退出码为 非0
|
||||||
}
|
}
|
||||||
|
|
||||||
function make_package() {
|
function make_package() {
|
||||||
# 压缩output目录
|
# 压缩output目录
|
||||||
(
|
(
|
||||||
cd ${OUTPUT_DIR} && tar cvzf ${APP_DIR}.tar.gz ${APP_DIR}
|
tar cvzf ${app_name}.tar.gz ${output_dir}
|
||||||
echo -e "make package ok."
|
echo -e "make package ok."
|
||||||
) || { echo -e "make package error"; exit 2; } # 压缩output目录失败后, 退出码为 非0
|
) || { echo -e "make package error"; exit 2; } # 压缩output目录失败后, 退出码为 非0
|
||||||
}
|
}
|
||||||
|
|
||||||
|
## internals
|
||||||
|
function gitversion() {
|
||||||
|
git log -1 --pretty=%h > $gitversion
|
||||||
|
local gv=`cat $gitversion`
|
||||||
|
echo "$gv"
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
##########################################
|
##########################################
|
||||||
## main
|
## main
|
||||||
## 其中,
|
## 其中,
|
||||||
@@ -69,4 +88,4 @@ make_package
|
|||||||
|
|
||||||
# 编译成功
|
# 编译成功
|
||||||
echo -e "build done"
|
echo -e "build done"
|
||||||
exit 0
|
exit 0
|
||||||
|
|||||||
@@ -1,43 +0,0 @@
|
|||||||
FROM openjdk:16-jdk-alpine3.13
|
|
||||||
|
|
||||||
LABEL author="yangvipguang"
|
|
||||||
|
|
||||||
ENV VERSION 2.3.1
|
|
||||||
|
|
||||||
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.aliyun.com/g' /etc/apk/repositories
|
|
||||||
RUN apk add --no-cache --virtual .build-deps \
|
|
||||||
font-adobe-100dpi \
|
|
||||||
ttf-dejavu \
|
|
||||||
fontconfig \
|
|
||||||
curl \
|
|
||||||
apr \
|
|
||||||
apr-util \
|
|
||||||
apr-dev \
|
|
||||||
tomcat-native \
|
|
||||||
&& apk del .build-deps
|
|
||||||
|
|
||||||
RUN apk add --no-cache tini
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
ENV AGENT_HOME /opt/agent/
|
|
||||||
|
|
||||||
WORKDIR /tmp
|
|
||||||
|
|
||||||
COPY $JAR_PATH/kafka-manager.jar app.jar
|
|
||||||
# COPY application.yml application.yml ##默认使用helm 挂载，防止敏感配置泄露
|
|
||||||
|
|
||||||
COPY docker-depends/config.yaml $AGENT_HOME
|
|
||||||
COPY docker-depends/jmx_prometheus_javaagent-0.15.0.jar $AGENT_HOME
|
|
||||||
|
|
||||||
ENV JAVA_AGENT="-javaagent:$AGENT_HOME/jmx_prometheus_javaagent-0.15.0.jar=9999:$AGENT_HOME/config.yaml"
|
|
||||||
ENV JAVA_HEAP_OPTS="-Xms1024M -Xmx1024M -Xmn100M "
|
|
||||||
ENV JAVA_OPTS="-verbose:gc \
|
|
||||||
-XX:MaxMetaspaceSize=256M -XX:+DisableExplicitGC -XX:+UseStringDeduplication \
|
|
||||||
-XX:+UseG1GC -XX:+HeapDumpOnOutOfMemoryError -XX:-UseContainerSupport"
|
|
||||||
EXPOSE 8080 9999
|
|
||||||
|
|
||||||
ENTRYPOINT ["tini", "--"]
|
|
||||||
|
|
||||||
CMD ["sh","-c","java -jar $JAVA_AGENT $JAVA_HEAP_OPTS $JAVA_OPTS app.jar --spring.config.location=application.yml"]
|
|
||||||
@@ -1,5 +0,0 @@
|
|||||||
---
|
|
||||||
startDelaySeconds: 0
|
|
||||||
ssl: false
|
|
||||||
lowercaseOutputName: false
|
|
||||||
lowercaseOutputLabelNames: false
|
|
||||||
Binary file not shown.
Binary file not shown.
@@ -1,23 +0,0 @@
|
|||||||
# Patterns to ignore when building packages.
|
|
||||||
# This supports shell glob matching, relative path matching, and
|
|
||||||
# negation (prefixed with !). Only one pattern per line.
|
|
||||||
.DS_Store
|
|
||||||
# Common VCS dirs
|
|
||||||
.git/
|
|
||||||
.gitignore
|
|
||||||
.bzr/
|
|
||||||
.bzrignore
|
|
||||||
.hg/
|
|
||||||
.hgignore
|
|
||||||
.svn/
|
|
||||||
# Common backup files
|
|
||||||
*.swp
|
|
||||||
*.bak
|
|
||||||
*.tmp
|
|
||||||
*.orig
|
|
||||||
*~
|
|
||||||
# Various IDEs
|
|
||||||
.project
|
|
||||||
.idea/
|
|
||||||
*.tmproj
|
|
||||||
.vscode/
|
|
||||||
@@ -1,24 +0,0 @@
|
|||||||
apiVersion: v2
|
|
||||||
name: didi-km
|
|
||||||
description: A Helm chart for Kubernetes
|
|
||||||
|
|
||||||
# A chart can be either an 'application' or a 'library' chart.
|
|
||||||
#
|
|
||||||
# Application charts are a collection of templates that can be packaged into versioned archives
|
|
||||||
# to be deployed.
|
|
||||||
#
|
|
||||||
# Library charts provide useful utilities or functions for the chart developer. They're included as
|
|
||||||
# a dependency of application charts to inject those utilities and functions into the rendering
|
|
||||||
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
|
|
||||||
type: application
|
|
||||||
|
|
||||||
# This is the chart version. This version number should be incremented each time you make changes
|
|
||||||
# to the chart and its templates, including the app version.
|
|
||||||
# Versions are expected to follow Semantic Versioning (https://semver.org/)
|
|
||||||
version: 0.1.0
|
|
||||||
|
|
||||||
# This is the version number of the application being deployed. This version number should be
|
|
||||||
# incremented each time you make changes to the application. Versions are not expected to
|
|
||||||
# follow Semantic Versioning. They should reflect the version the application is using.
|
|
||||||
# It is recommended to use it with quotes.
|
|
||||||
appVersion: "1.16.0"
|
|
||||||
@@ -1,22 +0,0 @@
|
|||||||
1. Get the application URL by running these commands:
|
|
||||||
{{- if .Values.ingress.enabled }}
|
|
||||||
{{- range $host := .Values.ingress.hosts }}
|
|
||||||
{{- range .paths }}
|
|
||||||
http{{ if $.Values.ingress.tls }}s{{ end }}://{{ $host.host }}{{ .path }}
|
|
||||||
{{- end }}
|
|
||||||
{{- end }}
|
|
||||||
{{- else if contains "NodePort" .Values.service.type }}
|
|
||||||
export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ include "didi-km.fullname" . }})
|
|
||||||
export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
|
|
||||||
echo http://$NODE_IP:$NODE_PORT
|
|
||||||
{{- else if contains "LoadBalancer" .Values.service.type }}
|
|
||||||
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
|
|
||||||
You can watch the status of by running 'kubectl get --namespace {{ .Release.Namespace }} svc -w {{ include "didi-km.fullname" . }}'
|
|
||||||
export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ include "didi-km.fullname" . }} --template "{{"{{ range (index .status.loadBalancer.ingress 0) }}{{.}}{{ end }}"}}")
|
|
||||||
echo http://$SERVICE_IP:{{ .Values.service.port }}
|
|
||||||
{{- else if contains "ClusterIP" .Values.service.type }}
|
|
||||||
export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "didi-km.name" . }},app.kubernetes.io/instance={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
|
|
||||||
export CONTAINER_PORT=$(kubectl get pod --namespace {{ .Release.Namespace }} $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}")
|
|
||||||
echo "Visit http://127.0.0.1:8080 to use your application"
|
|
||||||
kubectl --namespace {{ .Release.Namespace }} port-forward $POD_NAME 8080:$CONTAINER_PORT
|
|
||||||
{{- end }}
|
|
||||||
@@ -1,62 +0,0 @@
|
|||||||
{{/*
|
|
||||||
Expand the name of the chart.
|
|
||||||
*/}}
|
|
||||||
{{- define "didi-km.name" -}}
|
|
||||||
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
|
|
||||||
{{- end }}
|
|
||||||
|
|
||||||
{{/*
|
|
||||||
Create a default fully qualified app name.
|
|
||||||
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
|
|
||||||
If release name contains chart name it will be used as a full name.
|
|
||||||
*/}}
|
|
||||||
{{- define "didi-km.fullname" -}}
|
|
||||||
{{- if .Values.fullnameOverride }}
|
|
||||||
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
|
|
||||||
{{- else }}
|
|
||||||
{{- $name := default .Chart.Name .Values.nameOverride }}
|
|
||||||
{{- if contains $name .Release.Name }}
|
|
||||||
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
|
|
||||||
{{- else }}
|
|
||||||
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
|
|
||||||
{{- end }}
|
|
||||||
{{- end }}
|
|
||||||
{{- end }}
|
|
||||||
|
|
||||||
{{/*
|
|
||||||
Create chart name and version as used by the chart label.
|
|
||||||
*/}}
|
|
||||||
{{- define "didi-km.chart" -}}
|
|
||||||
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
|
|
||||||
{{- end }}
|
|
||||||
|
|
||||||
{{/*
|
|
||||||
Common labels
|
|
||||||
*/}}
|
|
||||||
{{- define "didi-km.labels" -}}
|
|
||||||
helm.sh/chart: {{ include "didi-km.chart" . }}
|
|
||||||
{{ include "didi-km.selectorLabels" . }}
|
|
||||||
{{- if .Chart.AppVersion }}
|
|
||||||
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
|
|
||||||
{{- end }}
|
|
||||||
app.kubernetes.io/managed-by: {{ .Release.Service }}
|
|
||||||
{{- end }}
|
|
||||||
|
|
||||||
{{/*
|
|
||||||
Selector labels
|
|
||||||
*/}}
|
|
||||||
{{- define "didi-km.selectorLabels" -}}
|
|
||||||
app.kubernetes.io/name: {{ include "didi-km.name" . }}
|
|
||||||
app.kubernetes.io/instance: {{ .Release.Name }}
|
|
||||||
{{- end }}
|
|
||||||
|
|
||||||
{{/*
|
|
||||||
Create the name of the service account to use
|
|
||||||
*/}}
|
|
||||||
{{- define "didi-km.serviceAccountName" -}}
|
|
||||||
{{- if .Values.serviceAccount.create }}
|
|
||||||
{{- default (include "didi-km.fullname" .) .Values.serviceAccount.name }}
|
|
||||||
{{- else }}
|
|
||||||
{{- default "default" .Values.serviceAccount.name }}
|
|
||||||
{{- end }}
|
|
||||||
{{- end }}
|
|
||||||
@@ -1,88 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: ConfigMap
|
|
||||||
metadata:
|
|
||||||
name: km-cm
|
|
||||||
data:
|
|
||||||
application.yml: |
|
|
||||||
server:
|
|
||||||
port: 8080
|
|
||||||
tomcat:
|
|
||||||
accept-count: 1000
|
|
||||||
max-connections: 10000
|
|
||||||
max-threads: 800
|
|
||||||
min-spare-threads: 100
|
|
||||||
|
|
||||||
spring:
|
|
||||||
application:
|
|
||||||
name: kafkamanager
|
|
||||||
datasource:
|
|
||||||
kafka-manager:
|
|
||||||
jdbc-url: jdbc:mysql://xxxxx:3306/kafka-manager?characterEncoding=UTF-8&serverTimezone=GMT%2B8&useSSL=false
|
|
||||||
username: admin
|
|
||||||
password: admin
|
|
||||||
driver-class-name: com.mysql.jdbc.Driver
|
|
||||||
main:
|
|
||||||
allow-bean-definition-overriding: true
|
|
||||||
|
|
||||||
profiles:
|
|
||||||
active: dev
|
|
||||||
servlet:
|
|
||||||
multipart:
|
|
||||||
max-file-size: 100MB
|
|
||||||
max-request-size: 100MB
|
|
||||||
|
|
||||||
logging:
|
|
||||||
config: classpath:logback-spring.xml
|
|
||||||
|
|
||||||
custom:
|
|
||||||
idc: cn
|
|
||||||
jmx:
|
|
||||||
max-conn: 20
|
|
||||||
store-metrics-task:
|
|
||||||
community:
|
|
||||||
broker-metrics-enabled: true
|
|
||||||
topic-metrics-enabled: true
|
|
||||||
didi:
|
|
||||||
app-topic-metrics-enabled: false
|
|
||||||
topic-request-time-metrics-enabled: false
|
|
||||||
topic-throttled-metrics: false
|
|
||||||
save-days: 7
|
|
||||||
|
|
||||||
# 任务相关的开关
|
|
||||||
task:
|
|
||||||
op:
|
|
||||||
sync-topic-enabled: false # 未落盘的Topic定期同步到DB中
|
|
||||||
|
|
||||||
account:
|
|
||||||
ldap:
|
|
||||||
|
|
||||||
kcm:
|
|
||||||
enabled: false
|
|
||||||
storage:
|
|
||||||
base-url: http://127.0.0.1
|
|
||||||
n9e:
|
|
||||||
base-url: http://127.0.0.1:8004
|
|
||||||
user-token: 12345678
|
|
||||||
timeout: 300
|
|
||||||
account: root
|
|
||||||
script-file: kcm_script.sh
|
|
||||||
|
|
||||||
monitor:
|
|
||||||
enabled: false
|
|
||||||
n9e:
|
|
||||||
nid: 2
|
|
||||||
user-token: 1234567890
|
|
||||||
mon:
|
|
||||||
base-url: http://127.0.0.1:8032
|
|
||||||
sink:
|
|
||||||
base-url: http://127.0.0.1:8006
|
|
||||||
rdb:
|
|
||||||
base-url: http://127.0.0.1:80
|
|
||||||
|
|
||||||
notify:
|
|
||||||
kafka:
|
|
||||||
cluster-id: 95
|
|
||||||
topic-name: didi-kafka-notify
|
|
||||||
order:
|
|
||||||
detail-url: http://127.0.0.1
|
|
||||||
|
|
||||||
@@ -1,56 +0,0 @@
|
|||||||
apiVersion: apps/v1
|
|
||||||
kind: Deployment
|
|
||||||
metadata:
|
|
||||||
name: {{ include "didi-km.fullname" . }}
|
|
||||||
labels:
|
|
||||||
{{- include "didi-km.labels" . | nindent 4 }}
|
|
||||||
spec:
|
|
||||||
{{- if not .Values.autoscaling.enabled }}
|
|
||||||
replicas: {{ .Values.replicaCount }}
|
|
||||||
{{- end }}
|
|
||||||
selector:
|
|
||||||
matchLabels:
|
|
||||||
{{- include "didi-km.selectorLabels" . | nindent 6 }}
|
|
||||||
template:
|
|
||||||
metadata:
|
|
||||||
{{- with .Values.podAnnotations }}
|
|
||||||
annotations:
|
|
||||||
{{- toYaml . | nindent 8 }}
|
|
||||||
{{- end }}
|
|
||||||
labels:
|
|
||||||
{{- include "didi-km.selectorLabels" . | nindent 8 }}
|
|
||||||
spec:
|
|
||||||
{{- with .Values.imagePullSecrets }}
|
|
||||||
imagePullSecrets:
|
|
||||||
{{- toYaml . | nindent 8 }}
|
|
||||||
{{- end }}
|
|
||||||
serviceAccountName: {{ include "didi-km.serviceAccountName" . }}
|
|
||||||
securityContext:
|
|
||||||
{{- toYaml .Values.podSecurityContext | nindent 8 }}
|
|
||||||
containers:
|
|
||||||
- name: {{ .Chart.Name }}
|
|
||||||
securityContext:
|
|
||||||
{{- toYaml .Values.securityContext | nindent 12 }}
|
|
||||||
image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
|
|
||||||
imagePullPolicy: {{ .Values.image.pullPolicy }}
|
|
||||||
ports:
|
|
||||||
- name: http
|
|
||||||
containerPort: 8080
|
|
||||||
protocol: TCP
|
|
||||||
- name: jmx-metrics
|
|
||||||
containerPort: 9999
|
|
||||||
protocol: TCP
|
|
||||||
resources:
|
|
||||||
{{- toYaml .Values.resources | nindent 12 }}
|
|
||||||
{{- with .Values.nodeSelector }}
|
|
||||||
nodeSelector:
|
|
||||||
{{- toYaml . | nindent 8 }}
|
|
||||||
{{- end }}
|
|
||||||
{{- with .Values.affinity }}
|
|
||||||
affinity:
|
|
||||||
{{- toYaml . | nindent 8 }}
|
|
||||||
{{- end }}
|
|
||||||
{{- with .Values.tolerations }}
|
|
||||||
tolerations:
|
|
||||||
{{- toYaml . | nindent 8 }}
|
|
||||||
{{- end }}
|
|
||||||
@@ -1,28 +0,0 @@
|
|||||||
{{- if .Values.autoscaling.enabled }}
|
|
||||||
apiVersion: autoscaling/v2beta1
|
|
||||||
kind: HorizontalPodAutoscaler
|
|
||||||
metadata:
|
|
||||||
name: {{ include "didi-km.fullname" . }}
|
|
||||||
labels:
|
|
||||||
{{- include "didi-km.labels" . | nindent 4 }}
|
|
||||||
spec:
|
|
||||||
scaleTargetRef:
|
|
||||||
apiVersion: apps/v1
|
|
||||||
kind: Deployment
|
|
||||||
name: {{ include "didi-km.fullname" . }}
|
|
||||||
minReplicas: {{ .Values.autoscaling.minReplicas }}
|
|
||||||
maxReplicas: {{ .Values.autoscaling.maxReplicas }}
|
|
||||||
metrics:
|
|
||||||
{{- if .Values.autoscaling.targetCPUUtilizationPercentage }}
|
|
||||||
- type: Resource
|
|
||||||
resource:
|
|
||||||
name: cpu
|
|
||||||
targetAverageUtilization: {{ .Values.autoscaling.targetCPUUtilizationPercentage }}
|
|
||||||
{{- end }}
|
|
||||||
{{- if .Values.autoscaling.targetMemoryUtilizationPercentage }}
|
|
||||||
- type: Resource
|
|
||||||
resource:
|
|
||||||
name: memory
|
|
||||||
targetAverageUtilization: {{ .Values.autoscaling.targetMemoryUtilizationPercentage }}
|
|
||||||
{{- end }}
|
|
||||||
{{- end }}
|
|
||||||
@@ -1,41 +0,0 @@
|
|||||||
{{- if .Values.ingress.enabled -}}
|
|
||||||
{{- $fullName := include "didi-km.fullname" . -}}
|
|
||||||
{{- $svcPort := .Values.service.port -}}
|
|
||||||
{{- if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
|
|
||||||
apiVersion: networking.k8s.io/v1beta1
|
|
||||||
{{- else -}}
|
|
||||||
apiVersion: extensions/v1beta1
|
|
||||||
{{- end }}
|
|
||||||
kind: Ingress
|
|
||||||
metadata:
|
|
||||||
name: {{ $fullName }}
|
|
||||||
labels:
|
|
||||||
{{- include "didi-km.labels" . | nindent 4 }}
|
|
||||||
{{- with .Values.ingress.annotations }}
|
|
||||||
annotations:
|
|
||||||
{{- toYaml . | nindent 4 }}
|
|
||||||
{{- end }}
|
|
||||||
spec:
|
|
||||||
{{- if .Values.ingress.tls }}
|
|
||||||
tls:
|
|
||||||
{{- range .Values.ingress.tls }}
|
|
||||||
- hosts:
|
|
||||||
{{- range .hosts }}
|
|
||||||
- {{ . | quote }}
|
|
||||||
{{- end }}
|
|
||||||
secretName: {{ .secretName }}
|
|
||||||
{{- end }}
|
|
||||||
{{- end }}
|
|
||||||
rules:
|
|
||||||
{{- range .Values.ingress.hosts }}
|
|
||||||
- host: {{ .host | quote }}
|
|
||||||
http:
|
|
||||||
paths:
|
|
||||||
{{- range .paths }}
|
|
||||||
- path: {{ .path }}
|
|
||||||
backend:
|
|
||||||
serviceName: {{ $fullName }}
|
|
||||||
servicePort: {{ $svcPort }}
|
|
||||||
{{- end }}
|
|
||||||
{{- end }}
|
|
||||||
{{- end }}
|
|
||||||
@@ -1,15 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Service
|
|
||||||
metadata:
|
|
||||||
name: {{ include "didi-km.fullname" . }}
|
|
||||||
labels:
|
|
||||||
{{- include "didi-km.labels" . | nindent 4 }}
|
|
||||||
spec:
|
|
||||||
type: {{ .Values.service.type }}
|
|
||||||
ports:
|
|
||||||
- port: {{ .Values.service.port }}
|
|
||||||
targetPort: http
|
|
||||||
protocol: TCP
|
|
||||||
name: http
|
|
||||||
selector:
|
|
||||||
{{- include "didi-km.selectorLabels" . | nindent 4 }}
|
|
||||||
@@ -1,12 +0,0 @@
|
|||||||
{{- if .Values.serviceAccount.create -}}
|
|
||||||
apiVersion: v1
|
|
||||||
kind: ServiceAccount
|
|
||||||
metadata:
|
|
||||||
name: {{ include "didi-km.serviceAccountName" . }}
|
|
||||||
labels:
|
|
||||||
{{- include "didi-km.labels" . | nindent 4 }}
|
|
||||||
{{- with .Values.serviceAccount.annotations }}
|
|
||||||
annotations:
|
|
||||||
{{- toYaml . | nindent 4 }}
|
|
||||||
{{- end }}
|
|
||||||
{{- end }}
|
|
||||||
@@ -1,15 +0,0 @@
|
|||||||
apiVersion: v1
|
|
||||||
kind: Pod
|
|
||||||
metadata:
|
|
||||||
name: "{{ include "didi-km.fullname" . }}-test-connection"
|
|
||||||
labels:
|
|
||||||
{{- include "didi-km.labels" . | nindent 4 }}
|
|
||||||
annotations:
|
|
||||||
"helm.sh/hook": test
|
|
||||||
spec:
|
|
||||||
containers:
|
|
||||||
- name: wget
|
|
||||||
image: busybox
|
|
||||||
command: ['wget']
|
|
||||||
args: ['{{ include "didi-km.fullname" . }}:{{ .Values.service.port }}']
|
|
||||||
restartPolicy: Never
|
|
||||||
@@ -1,79 +0,0 @@
|
|||||||
# Default values for didi-km.
|
|
||||||
# This is a YAML-formatted file.
|
|
||||||
# Declare variables to be passed into your templates.
|
|
||||||
|
|
||||||
replicaCount: 1
|
|
||||||
|
|
||||||
image:
|
|
||||||
repository: docker.io/yangvipguang/km
|
|
||||||
pullPolicy: IfNotPresent
|
|
||||||
# Overrides the image tag whose default is the chart appVersion.
|
|
||||||
tag: "v18"
|
|
||||||
|
|
||||||
imagePullSecrets: []
|
|
||||||
nameOverride: ""
|
|
||||||
fullnameOverride: "km"
|
|
||||||
|
|
||||||
serviceAccount:
|
|
||||||
# Specifies whether a service account should be created
|
|
||||||
create: true
|
|
||||||
# Annotations to add to the service account
|
|
||||||
annotations: {}
|
|
||||||
# The name of the service account to use.
|
|
||||||
# If not set and create is true, a name is generated using the fullname template
|
|
||||||
name: ""
|
|
||||||
|
|
||||||
podAnnotations: {}
|
|
||||||
|
|
||||||
podSecurityContext: {}
|
|
||||||
# fsGroup: 2000
|
|
||||||
|
|
||||||
securityContext: {}
|
|
||||||
# capabilities:
|
|
||||||
# drop:
|
|
||||||
# - ALL
|
|
||||||
# readOnlyRootFilesystem: true
|
|
||||||
# runAsNonRoot: true
|
|
||||||
# runAsUser: 1000
|
|
||||||
|
|
||||||
service:
|
|
||||||
type: ClusterIP
|
|
||||||
port: 8080
|
|
||||||
|
|
||||||
ingress:
|
|
||||||
enabled: false
|
|
||||||
annotations: {}
|
|
||||||
# kubernetes.io/ingress.class: nginx
|
|
||||||
# kubernetes.io/tls-acme: "true"
|
|
||||||
hosts:
|
|
||||||
- host: chart-example.local
|
|
||||||
paths: []
|
|
||||||
tls: []
|
|
||||||
# - secretName: chart-example-tls
|
|
||||||
# hosts:
|
|
||||||
# - chart-example.local
|
|
||||||
|
|
||||||
resources:
|
|
||||||
# We usually recommend not to specify default resources and to leave this as a conscious
|
|
||||||
# choice for the user. This also increases chances charts run on environments with little
|
|
||||||
# resources, such as Minikube. If you do want to specify resources, uncomment the following
|
|
||||||
# lines, adjust them as necessary, and remove the curly braces after 'resources:'.
|
|
||||||
limits:
|
|
||||||
cpu: 50m
|
|
||||||
memory: 2048Mi
|
|
||||||
requests:
|
|
||||||
cpu: 10m
|
|
||||||
memory: 200Mi
|
|
||||||
|
|
||||||
autoscaling:
|
|
||||||
enabled: false
|
|
||||||
minReplicas: 1
|
|
||||||
maxReplicas: 100
|
|
||||||
targetCPUUtilizationPercentage: 80
|
|
||||||
# targetMemoryUtilizationPercentage: 80
|
|
||||||
|
|
||||||
nodeSelector: {}
|
|
||||||
|
|
||||||
tolerations: []
|
|
||||||
|
|
||||||
affinity: {}
|
|
||||||
@@ -7,7 +7,7 @@
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# 监控系统集成——夜莺
|
# 夜莺监控集成
|
||||||
|
|
||||||
- `Kafka-Manager`通过将 监控的数据 以及 监控的规则 都提交给夜莺，然后依赖夜莺的监控系统从而实现监控告警功能。
|
- `Kafka-Manager`通过将 监控的数据 以及 监控的规则 都提交给夜莺，然后依赖夜莺的监控系统从而实现监控告警功能。
|
||||||
|
|
||||||
@@ -22,13 +22,10 @@ monitor:
|
|||||||
n9e:
|
n9e:
|
||||||
nid: 2
|
nid: 2
|
||||||
user-token: 123456
|
user-token: 123456
|
||||||
# 夜莺 mon监控服务 地址
|
|
||||||
mon:
|
mon:
|
||||||
base-url: http://127.0.0.1:8006
|
base-url: http://127.0.0.1:8032
|
||||||
# 夜莺 transfer上传服务 地址
|
|
||||||
sink:
|
sink:
|
||||||
base-url: http://127.0.0.1:8008
|
base-url: http://127.0.0.1:8006
|
||||||
# 夜莺 rdb资源服务 地址
|
|
||||||
rdb:
|
rdb:
|
||||||
base-url: http://127.0.0.1:80
|
base-url: http://127.0.0.1:80
|
||||||
|
|
||||||
Binary file not shown.
|
Before Width: | Height: | Size: 382 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 270 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 589 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 652 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 511 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 672 KiB |
@@ -1,101 +0,0 @@
|
|||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
**一站式`Apache Kafka`集群指标监控与运维管控平台**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## JMX-连接失败问题解决
|
|
||||||
|
|
||||||
集群正常接入Logi-KafkaManager之后，即可以看到集群的Broker列表，此时如果查看不了Topic的实时流量，或者是Broker的实时流量信息时，那么大概率就是JMX连接的问题了。
|
|
||||||
|
|
||||||
下面我们按照步骤来一步一步的检查。
|
|
||||||
|
|
||||||
### 1、问题&说明
|
|
||||||
|
|
||||||
**类型一：JMX配置未开启**
|
|
||||||
|
|
||||||
未开启时，直接到`2、解决方法`查看如何开启即可。
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
**类型二：配置错误**
|
|
||||||
|
|
||||||
`JMX`端口已经开启的情况下，有的时候开启的配置不正确，此时也会导致出现连接失败的问题。这里大概列举几种原因：
|
|
||||||
|
|
||||||
- `JMX`配置错误：见`2、解决方法`。
|
|
||||||
- 存在防火墙或者网络限制：网络通的另外一台机器`telnet`试一下看是否可以连接上。
|
|
||||||
- 需要进行用户名及密码的认证：见`3、解决方法 —— 认证的JMX`。
|
|
||||||
|
|
||||||
|
|
||||||
错误日志例子：
|
|
||||||
```
|
|
||||||
# 错误一： 错误提示的是真实的IP，这样的话基本就是JMX配置的有问题了。
|
|
||||||
2021-01-27 10:06:20.730 ERROR 50901 --- [ics-Thread-1-62] c.x.k.m.c.utils.jmx.JmxConnectorWrap : JMX connect exception, host:192.168.0.1 port:9999.
|
|
||||||
java.rmi.ConnectException: Connection refused to host: 192.168.0.1; nested exception is:
|
|
||||||
|
|
||||||
|
|
||||||
# 错误二：错误提示的是127.0.0.1这个IP，这个是机器的hostname配置的可能有问题。
|
|
||||||
2021-01-27 10:06:20.730 ERROR 50901 --- [ics-Thread-1-62] c.x.k.m.c.utils.jmx.JmxConnectorWrap : JMX connect exception, host:127.0.0.1 port:9999.
|
|
||||||
java.rmi.ConnectException: Connection refused to host: 127.0.0.1;; nested exception is:
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2、解决方法
|
|
||||||
|
|
||||||
这里仅介绍一下比较通用的解决方式，如若有更好的方式，欢迎大家指导告知一下。
|
|
||||||
|
|
||||||
修改`kafka-server-start.sh`文件：
|
|
||||||
```
|
|
||||||
# 在这个下面增加JMX端口的配置
|
|
||||||
if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
|
|
||||||
export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
|
|
||||||
export JMX_PORT=9999 # 增加这个配置, 这里的数值并不一定是要9999
|
|
||||||
fi
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
修改`kafka-run-class.sh`文件
|
|
||||||
```
|
|
||||||
# JMX settings
|
|
||||||
if [ -z "$KAFKA_JMX_OPTS" ]; then
|
|
||||||
KAFKA_JMX_OPTS="-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.rmi.server.hostname=${当前机器的IP}"
|
|
||||||
fi
|
|
||||||
|
|
||||||
# JMX port to use
|
|
||||||
if [ $JMX_PORT ]; then
|
|
||||||
KAFKA_JMX_OPTS="$KAFKA_JMX_OPTS -Dcom.sun.management.jmxremote.port=$JMX_PORT -Dcom.sun.management.jmxremote.rmi.port=$JMX_PORT"
|
|
||||||
fi
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
### 3、解决方法 —— 认证的JMX
|
|
||||||
|
|
||||||
如果您是直接看的这个部分，建议先看一下上一节：`2、解决方法`以确保`JMX`的配置没有问题了。
|
|
||||||
|
|
||||||
在JMX的配置等都没有问题的情况下，如果是因为认证的原因导致连接不了的，此时可以使用下面介绍的方法进行解决。
|
|
||||||
|
|
||||||
**当前这块后端刚刚开发完成，可能还不够完善，有问题随时沟通。**
|
|
||||||
|
|
||||||
`Logi-KafkaManager 2.2.0+`之后的版本后端已经支持`JMX`认证方式的连接，但是还没有界面，此时我们可以往`cluster`表的`jmx_properties`字段写入`JMX`的认证信息。
|
|
||||||
|
|
||||||
这个数据是`json`格式的字符串，例子如下所示：
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"maxConn": 10, # KM对单台Broker的最大JMX连接数
|
|
||||||
"username": "xxxxx", # 用户名
|
|
||||||
"password": "xxxx", # 密码
|
|
||||||
"openSSL": true, # 开启SSL, true表示开启ssl, false表示关闭
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
SQL的例子：
|
|
||||||
```sql
|
|
||||||
UPDATE cluster SET jmx_properties='{ "maxConn": 10, "username": "xxxxx", "password": "xxxx", "openSSL": false }' where id={xxx};
|
|
||||||
```
|
|
||||||
@@ -1,122 +0,0 @@
|
|||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
**一站式`Apache Kafka`集群指标监控与运维管控平台**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
# 动态配置管理
|
|
||||||
|
|
||||||
## 0、目录
|
|
||||||
|
|
||||||
- 1、Topic定时同步任务
|
|
||||||
- 2、专家服务——Topic分区热点
|
|
||||||
- 3、专家服务——Topic分区不足
|
|
||||||
|
|
||||||
|
|
||||||
## 1、Topic定时同步任务
|
|
||||||
|
|
||||||
### 1.1、配置的用途
|
|
||||||
`Logi-KafkaManager`在设计上，所有的资源都是挂在应用(app)下面。 如果接入的Kafka集群已经存在Topic了，那么会导致这些Topic不属于任何的应用，从而导致很多管理上的不便。
|
|
||||||
|
|
||||||
因此，需要有一个方式将这些无主的Topic挂到某个应用下面。
|
|
||||||
|
|
||||||
这里提供了一个配置，会定时自动将集群无主的Topic挂到某个应用下面下面。
|
|
||||||
|
|
||||||
### 1.2、相关实现
|
|
||||||
|
|
||||||
就是一个定时任务，该任务会定期做同步的工作。具体代码的位置在`com.xiaojukeji.kafka.manager.task.dispatch.op`包下面的`SyncTopic2DB`类。
|
|
||||||
|
|
||||||
### 1.3、配置说明
|
|
||||||
|
|
||||||
**步骤一：开启该功能**
|
|
||||||
|
|
||||||
在application.yml文件中，增加如下配置，已经有该配置的话，直接把false修改为true即可
|
|
||||||
```yml
|
|
||||||
# 任务相关的开关
|
|
||||||
task:
|
|
||||||
op:
|
|
||||||
sync-topic-enabled: true # 无主的Topic定期同步到DB中
|
|
||||||
```
|
|
||||||
|
|
||||||
**步骤二：配置管理中指定挂在那个应用下面**
|
|
||||||
|
|
||||||
配置的位置：
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
配置键：`SYNC_TOPIC_2_DB_CONFIG_KEY`
|
|
||||||
|
|
||||||
配置值(JSON数组)：
|
|
||||||
- clusterId：需要进行定时同步的集群ID
|
|
||||||
- defaultAppId：该集群无主的Topic将挂在哪个应用下面
|
|
||||||
- addAuthority：是否需要加上权限, 默认是false。因为考虑到这个挂载只是临时的，我们不希望用户使用这个App，同时后续可能移交给真正的所属的应用，因此默认是不加上权限。
|
|
||||||
|
|
||||||
**注意，这里的集群ID，或者是应用ID不存在的话，会导致配置不生效。该任务对已经在DB中的Topic不会进行修改**
|
|
||||||
```json
|
|
||||||
[
|
|
||||||
{
|
|
||||||
"clusterId": 1234567,
|
|
||||||
"defaultAppId": "ANONYMOUS",
|
|
||||||
"addAuthority": false
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"clusterId": 7654321,
|
|
||||||
"defaultAppId": "ANONYMOUS",
|
|
||||||
"addAuthority": false
|
|
||||||
}
|
|
||||||
]
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 2、专家服务——Topic分区热点
|
|
||||||
|
|
||||||
在`Region`所圈定的Broker范围内，某个Topic的Leader数在这些圈定的Broker上分布不均衡时，我们认为该Topic是存在热点的Topic。
|
|
||||||
|
|
||||||
备注：单纯的查看Leader数的分布，确实存在一定的局限性，这块欢迎贡献更多的热点定义于代码。
|
|
||||||
|
|
||||||
|
|
||||||
Topic分区热点相关的动态配置(页面在运维管控->平台管理->配置管理)：
|
|
||||||
|
|
||||||
配置Key：
|
|
||||||
```
|
|
||||||
REGION_HOT_TOPIC_CONFIG
|
|
||||||
```
|
|
||||||
|
|
||||||
配置Value：
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"maxDisPartitionNum": 2, # Region内Broker间的leader数差距超过2时，则认为是存在热点的Topic
|
|
||||||
"minTopicBytesInUnitB": 1048576, # 流量低于该值的Topic不做统计
|
|
||||||
"ignoreClusterIdList": [ # 忽略的集群
|
|
||||||
50
|
|
||||||
]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3、专家服务——Topic分区不足
|
|
||||||
|
|
||||||
总流量除以分区数，超过指定值时，则我们认为存在Topic分区不足。
|
|
||||||
|
|
||||||
Topic分区不足相关的动态配置(页面在运维管控->平台管理->配置管理)：
|
|
||||||
|
|
||||||
配置Key：
|
|
||||||
```
|
|
||||||
TOPIC_INSUFFICIENT_PARTITION_CONFIG
|
|
||||||
```
|
|
||||||
|
|
||||||
配置Value：
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"maxBytesInPerPartitionUnitB": 3145728, # 单分区流量超过该值, 则认为分区不去
|
|
||||||
"minTopicBytesInUnitB": 1048576, # 流量低于该值的Topic不做统计
|
|
||||||
"ignoreClusterIdList": [ # 忽略的集群
|
|
||||||
50
|
|
||||||
]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
@@ -1,10 +0,0 @@
|
|||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
**一站式`Apache Kafka`集群指标监控与运维管控平台**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
# Kafka-Gateway 配置说明
|
|
||||||
@@ -1,54 +0,0 @@
|
|||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
**一站式`Apache Kafka`集群指标监控与运维管控平台**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
# 监控系统集成
|
|
||||||
|
|
||||||
- 监控系统默认与 [夜莺] (https://github.com/didi/nightingale) 进行集成；
|
|
||||||
- 对接自有的监控系统需要进行简单的二次开发，即实现部分监控告警模块的相关接口即可；
|
|
||||||
- 集成会有两块内容，一个是指标数据上报的集成，还有一个是监控告警规则的集成；
|
|
||||||
|
|
||||||
## 1、指标数据上报集成
|
|
||||||
|
|
||||||
仅完成这一步的集成之后，即可将监控数据上报到监控系统中，此时已能够在自己的监控系统进行监控告警规则的配置了。
|
|
||||||
|
|
||||||
**步骤一：实现指标上报的接口**
|
|
||||||
|
|
||||||
- 按照自己内部监控系统的数据格式要求，将数据进行组装成符合自己内部监控系统要求的数据进行上报，具体的可以参考夜莺集成的实现代码。
|
|
||||||
- 至于会上报哪些指标，可以查看有哪些地方调用了该接口。
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
**步骤二：相关配置修改**
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
**步骤三：开启上报任务**
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## 2、监控告警规则集成
|
|
||||||
|
|
||||||
完成**1、指标数据上报集成**之后，即可在自己的监控系统进行监控告警规则的配置了。完成该步骤的集成之后，可以在`Logi-KafkaManager`中进行监控告警规则的增删改查等等。
|
|
||||||
|
|
||||||
大体上和**1、指标数据上报集成**一致，
|
|
||||||
|
|
||||||
**步骤一：实现相关接口**
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
实现完成步骤一之后，接下来的步骤和**1、指标数据上报集成**中的步骤二、步骤三一致，都需要进行相关配置的修改即可。
|
|
||||||
|
|
||||||
|
|
||||||
## 3、总结
|
|
||||||
|
|
||||||
简单介绍了一下监控告警的集成，嫌麻烦的同学可以仅做 **1、指标数据上报集成** 这一节的内容即可满足一定场景下的需求。
|
|
||||||
|
|
||||||
|
|
||||||
**集成过程中，有任何觉得文档没有说清楚的地方或者建议，欢迎入群交流，也欢迎贡献代码，觉得好也辛苦给个star。**
|
|
||||||
@@ -1,27 +0,0 @@
|
|||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
**一站式`Apache Kafka`集群指标监控与运维管控平台**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
# 升级至`2.2.0`版本
|
|
||||||
|
|
||||||
`2.2.0`版本在`cluster`表及`logical_cluster`各增加了一个字段，因此需要执行下面的sql进行字段的增加。
|
|
||||||
|
|
||||||
```sql
|
|
||||||
# 往cluster表中增加jmx_properties字段, 这个字段会用于存储jmx相关的认证以及配置信息
|
|
||||||
ALTER TABLE `cluster` ADD COLUMN `jmx_properties` TEXT NULL COMMENT 'JMX配置' AFTER `security_properties`;
|
|
||||||
|
|
||||||
# 往logical_cluster中增加identification字段, 同时数据和原先name数据相同, 最后增加一个唯一键.
|
|
||||||
# 此后, name字段还是表示集群名称, 而identification字段表示的是集群标识, 只能是字母数字及下划线组成,
|
|
||||||
# 数据上报到监控系统时, 集群这个标识采用的字段就是identification字段, 之前使用的是name字段.
|
|
||||||
ALTER TABLE `logical_cluster` ADD COLUMN `identification` VARCHAR(192) NOT NULL DEFAULT '' COMMENT '逻辑集群标识' AFTER `name`;
|
|
||||||
|
|
||||||
UPDATE `logical_cluster` SET `identification`=`name` WHERE id>=0;
|
|
||||||
|
|
||||||
ALTER TABLE `logical_cluster` ADD INDEX `uniq_identification` (`identification` ASC);
|
|
||||||
```
|
|
||||||
|
|
||||||
@@ -1,17 +0,0 @@
|
|||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
**一站式`Apache Kafka`集群指标监控与运维管控平台**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
# 升级至`2.3.0`版本
|
|
||||||
|
|
||||||
`2.3.0`版本在`gateway_config`表增加了一个描述说明的字段，因此需要执行下面的sql进行字段的增加。
|
|
||||||
|
|
||||||
```sql
|
|
||||||
ALTER TABLE `gateway_config`
|
|
||||||
ADD COLUMN `description` TEXT NULL COMMENT '描述信息' AFTER `version`;
|
|
||||||
```
|
|
||||||
@@ -15,7 +15,7 @@
|
|||||||
当前因为无法同时兼容`MySQL 8`与`MySQL 5.7`，因此代码中默认的版本还是`MySQL 5.7`。
|
当前因为无法同时兼容`MySQL 8`与`MySQL 5.7`，因此代码中默认的版本还是`MySQL 5.7`。
|
||||||
|
|
||||||
|
|
||||||
当前如需使用`MySQL 8`，则需按照下述流程进行简单修改代码。
|
当前如需使用`MySQL 8`，则续按照下述流程进行简单修改代码。
|
||||||
|
|
||||||
|
|
||||||
- Step1. 修改application.yml中的MySQL驱动类
|
- Step1. 修改application.yml中的MySQL驱动类
|
||||||
|
|||||||
@@ -1,104 +0,0 @@
|
|||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
**一站式`Apache Kafka`集群指标监控与运维管控平台**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
# 配置说明
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
server:
|
|
||||||
port: 8080 # 服务端口
|
|
||||||
tomcat:
|
|
||||||
accept-count: 1000
|
|
||||||
max-connections: 10000
|
|
||||||
max-threads: 800
|
|
||||||
min-spare-threads: 100
|
|
||||||
|
|
||||||
spring:
|
|
||||||
application:
|
|
||||||
name: kafkamanager
|
|
||||||
datasource:
|
|
||||||
kafka-manager: # 数据库连接配置
|
|
||||||
jdbc-url: jdbc:mysql://127.0.0.1:3306/kafka_manager?characterEncoding=UTF-8&serverTimezone=GMT%2B8 #数据库的地址
|
|
||||||
username: admin # 用户名
|
|
||||||
password: admin # 密码
|
|
||||||
driver-class-name: com.mysql.jdbc.Driver
|
|
||||||
main:
|
|
||||||
allow-bean-definition-overriding: true
|
|
||||||
|
|
||||||
profiles:
|
|
||||||
active: dev # 启用的配置
|
|
||||||
servlet:
|
|
||||||
multipart:
|
|
||||||
max-file-size: 100MB
|
|
||||||
max-request-size: 100MB
|
|
||||||
|
|
||||||
logging:
|
|
||||||
config: classpath:logback-spring.xml
|
|
||||||
|
|
||||||
custom:
|
|
||||||
idc: cn # 部署的数据中心, 忽略该配置, 后续会进行删除
|
|
||||||
jmx:
|
|
||||||
max-conn: 10 # 和单台 broker 的最大JMX连接数
|
|
||||||
store-metrics-task:
|
|
||||||
community:
|
|
||||||
broker-metrics-enabled: true # 社区部分broker metrics信息收集开关, 关闭之后metrics信息将不会进行收集及写DB
|
|
||||||
topic-metrics-enabled: true # 社区部分topic的metrics信息收集开关, 关闭之后metrics信息将不会进行收集及写DB
|
|
||||||
didi:
|
|
||||||
app-topic-metrics-enabled: false # 滴滴埋入的指标, 社区AK不存在该指标，因此默认关闭
|
|
||||||
topic-request-time-metrics-enabled: false # 滴滴埋入的指标, 社区AK不存在该指标，因此默认关闭
|
|
||||||
topic-throttled-metrics: false # 滴滴埋入的指标, 社区AK不存在该指标，因此默认关闭
|
|
||||||
save-days: 7 #指标在DB中保持的天数，-1表示永久保存，7表示保存近7天的数据
|
|
||||||
|
|
||||||
# 任务相关的开关
|
|
||||||
task:
|
|
||||||
op:
|
|
||||||
sync-topic-enabled: false # 未落盘的Topic定期同步到DB中
|
|
||||||
|
|
||||||
account: # ldap相关的配置, 社区版本暂时支持不够完善，可以先忽略，欢迎贡献代码对这块做优化
|
|
||||||
ldap:
|
|
||||||
|
|
||||||
kcm: # 集群升级部署相关的功能，需要配合夜莺及S3进行使用，这块我们后续专门补充一个文档细化一下，牵扯到kcm_script.sh脚本的修改
|
|
||||||
enabled: false # 默认关闭
|
|
||||||
storage:
|
|
||||||
base-url: http://127.0.0.1 # 存储地址
|
|
||||||
n9e:
|
|
||||||
base-url: http://127.0.0.1:8004 # 夜莺任务中心的地址
|
|
||||||
user-token: 12345678 # 夜莺用户的token
|
|
||||||
timeout: 300 # 集群任务的超时时间，单位秒
|
|
||||||
account: root # 集群任务使用的账号
|
|
||||||
script-file: kcm_script.sh # 集群任务的脚本
|
|
||||||
|
|
||||||
monitor: # 监控告警相关的功能，需要配合夜莺进行使用
|
|
||||||
enabled: false # 默认关闭，true就是开启
|
|
||||||
n9e:
|
|
||||||
nid: 2
|
|
||||||
user-token: 1234567890
|
|
||||||
mon:
|
|
||||||
# 夜莺 mon监控服务 地址
|
|
||||||
base-url: http://127.0.0.1:8032
|
|
||||||
sink:
|
|
||||||
# 夜莺 transfer上传服务 地址
|
|
||||||
base-url: http://127.0.0.1:8006
|
|
||||||
rdb:
|
|
||||||
# 夜莺 rdb资源服务 地址
|
|
||||||
base-url: http://127.0.0.1:80
|
|
||||||
|
|
||||||
# enabled: 表示是否开启监控告警的功能, true: 开启, false: 不开启
|
|
||||||
# n9e.nid: 夜莺的节点ID
|
|
||||||
# n9e.user-token: 用户的密钥，在夜莺的个人设置中
|
|
||||||
# n9e.mon.base-url: 监控地址
|
|
||||||
# n9e.sink.base-url: 数据上报地址
|
|
||||||
# n9e.rdb.base-url: 用户资源中心地址
|
|
||||||
|
|
||||||
notify: # 通知的功能
|
|
||||||
kafka: # 默认通知发送到kafka的指定Topic中
|
|
||||||
cluster-id: 95 # Topic的集群ID
|
|
||||||
topic-name: didi-kafka-notify # Topic名称
|
|
||||||
order: # 部署的KM的地址
|
|
||||||
detail-url: http://127.0.0.1
|
|
||||||
```
|
|
||||||
@@ -1,8 +1,3 @@
|
|||||||
-- create database
|
|
||||||
CREATE DATABASE logi_kafka_manager;
|
|
||||||
|
|
||||||
USE logi_kafka_manager;
|
|
||||||
|
|
||||||
--
|
--
|
||||||
-- Table structure for table `account`
|
-- Table structure for table `account`
|
||||||
--
|
--
|
||||||
@@ -109,8 +104,7 @@ CREATE TABLE `cluster` (
|
|||||||
`zookeeper` varchar(512) NOT NULL DEFAULT '' COMMENT 'zk地址',
|
`zookeeper` varchar(512) NOT NULL DEFAULT '' COMMENT 'zk地址',
|
||||||
`bootstrap_servers` varchar(512) NOT NULL DEFAULT '' COMMENT 'server地址',
|
`bootstrap_servers` varchar(512) NOT NULL DEFAULT '' COMMENT 'server地址',
|
||||||
`kafka_version` varchar(32) NOT NULL DEFAULT '' COMMENT 'kafka版本',
|
`kafka_version` varchar(32) NOT NULL DEFAULT '' COMMENT 'kafka版本',
|
||||||
`security_properties` text COMMENT 'Kafka安全认证参数',
|
`security_properties` text COMMENT '安全认证参数',
|
||||||
`jmx_properties` text COMMENT 'JMX配置',
|
|
||||||
`status` tinyint(4) NOT NULL DEFAULT '1' COMMENT ' 监控标记, 0表示未监控, 1表示监控中',
|
`status` tinyint(4) NOT NULL DEFAULT '1' COMMENT ' 监控标记, 0表示未监控, 1表示监控中',
|
||||||
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
|
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
|
||||||
`gmt_modify` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
|
`gmt_modify` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
|
||||||
@@ -203,8 +197,7 @@ CREATE TABLE `gateway_config` (
|
|||||||
`type` varchar(128) NOT NULL DEFAULT '' COMMENT '配置类型',
|
`type` varchar(128) NOT NULL DEFAULT '' COMMENT '配置类型',
|
||||||
`name` varchar(128) NOT NULL DEFAULT '' COMMENT '配置名称',
|
`name` varchar(128) NOT NULL DEFAULT '' COMMENT '配置名称',
|
||||||
`value` text COMMENT '配置值',
|
`value` text COMMENT '配置值',
|
||||||
`version` bigint(20) unsigned NOT NULL DEFAULT '1' COMMENT '版本信息',
|
`version` bigint(20) unsigned NOT NULL DEFAULT '0' COMMENT '版本信息',
|
||||||
`description` text COMMENT '描述信息',
|
|
||||||
`create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
|
`create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
|
||||||
`modify_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
|
`modify_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
|
||||||
PRIMARY KEY (`id`),
|
PRIMARY KEY (`id`),
|
||||||
@@ -309,22 +302,20 @@ INSERT INTO kafka_user(app_id, password, user_type, operation) VALUES ('dkm_admi
|
|||||||
-- Table structure for table `logical_cluster`
|
-- Table structure for table `logical_cluster`
|
||||||
--
|
--
|
||||||
|
|
||||||
|
-- DROP TABLE IF EXISTS `logical_cluster`;
|
||||||
CREATE TABLE `logical_cluster` (
|
CREATE TABLE `logical_cluster` (
|
||||||
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
|
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'id',
|
||||||
`name` varchar(192) NOT NULL DEFAULT '' COMMENT '逻辑集群名称',
|
`name` varchar(192) NOT NULL DEFAULT '' COMMENT '逻辑集群名称',
|
||||||
`identification` varchar(192) NOT NULL DEFAULT '' COMMENT '逻辑集群标识',
|
`mode` int(16) NOT NULL DEFAULT '0' COMMENT '逻辑集群类型, 0:共享集群, 1:独享集群, 2:独立集群',
|
||||||
`mode` int(16) NOT NULL DEFAULT '0' COMMENT '逻辑集群类型, 0:共享集群, 1:独享集群, 2:独立集群',
|
`app_id` varchar(64) NOT NULL DEFAULT '' COMMENT '所属应用',
|
||||||
`app_id` varchar(64) NOT NULL DEFAULT '' COMMENT '所属应用',
|
`cluster_id` bigint(20) NOT NULL DEFAULT '-1' COMMENT '集群id',
|
||||||
`cluster_id` bigint(20) NOT NULL DEFAULT '-1' COMMENT '集群id',
|
`region_list` varchar(256) NOT NULL DEFAULT '' COMMENT 'regionid列表',
|
||||||
`region_list` varchar(256) NOT NULL DEFAULT '' COMMENT 'regionid列表',
|
`description` text COMMENT '备注说明',
|
||||||
`description` text COMMENT '备注说明',
|
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
|
||||||
`gmt_create` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
|
`gmt_modify` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
|
||||||
`gmt_modify` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '修改时间',
|
PRIMARY KEY (`id`),
|
||||||
PRIMARY KEY (`id`),
|
UNIQUE KEY `uniq_name` (`name`)
|
||||||
UNIQUE KEY `uniq_name` (`name`),
|
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='逻辑集群信息表';
|
||||||
UNIQUE KEY `uniq_identification` (`identification`)
|
|
||||||
) ENGINE=InnoDB AUTO_INCREMENT=7 DEFAULT CHARSET=utf8 COMMENT='逻辑集群信息表';
|
|
||||||
|
|
||||||
|
|
||||||
--
|
--
|
||||||
-- Table structure for table `monitor_rule`
|
-- Table structure for table `monitor_rule`
|
||||||
|
|||||||
@@ -9,39 +9,19 @@
|
|||||||
|
|
||||||
# 安装手册
|
# 安装手册
|
||||||
|
|
||||||
## 1、环境依赖
|
|
||||||
|
|
||||||
如果是以Release包进行安装的，则仅安装`Java`及`MySQL`即可。如果是要先进行源码包进行打包，然后再使用，则需要安装`Maven`及`Node`环境。
|
## 环境依赖
|
||||||
|
|
||||||
|
- `Maven 3.5+`(后端打包依赖)
|
||||||
|
- `node 10+`(前端打包依赖)
|
||||||
- `Java 8+`(运行环境需要)
|
- `Java 8+`(运行环境需要)
|
||||||
- `MySQL 5.7`(数据存储)
|
- `MySQL 5.7`(数据存储)
|
||||||
- `Maven 3.5+`(后端打包依赖)
|
|
||||||
- `Node 10+`(前端打包依赖)
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 2、获取安装包
|
## 环境初始化
|
||||||
|
|
||||||
**1、Release直接下载**
|
执行[create_mysql_table.sql](create_mysql_table.sql)中的SQL命令，从而创建所需的MySQL库及表，默认创建的库名是`kafka_manager`。
|
||||||
|
|
||||||
这里如果觉得麻烦，然后也不想进行二次开发，则可以直接下载Release包，下载地址：[Github Release包下载地址](https://github.com/didi/Logi-KafkaManager/releases)
|
|
||||||
|
|
||||||
如果觉得Github的下载地址太慢了，也可以进入`Logi-KafkaManager`的用户群获取，群地址在README中。
|
|
||||||
|
|
||||||
|
|
||||||
**2、源代码进行打包**
|
|
||||||
|
|
||||||
下载好代码之后，进入`Logi-KafkaManager`的主目录，执行`sh build.sh`命令即可，执行完成之后会在`output/kafka-manager-xxx`目录下面生成一个jar包。
|
|
||||||
|
|
||||||
对于`windows`环境的用户，估计执行不了`sh build.sh`命令，因此可以直接执行`mvn install`，然后在`kafka-manager-web/target`目录下生成一个kafka-manager-web-xxx.jar的包。
|
|
||||||
|
|
||||||
获取到jar包之后，我们继续下面的步骤。
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## 3、MySQL-DB初始化
|
|
||||||
|
|
||||||
执行[create_mysql_table.sql](create_mysql_table.sql)中的SQL命令，从而创建所需的MySQL库及表，默认创建的库名是`logi_kafka_manager`。
|
|
||||||
|
|
||||||
```
|
```
|
||||||
# 示例：
|
# 示例：
|
||||||
@@ -50,15 +30,29 @@ mysql -uXXXX -pXXX -h XXX.XXX.XXX.XXX -PXXXX < ./create_mysql_table.sql
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 4、启动
|
## 打包
|
||||||
|
|
||||||
|
```bash
|
||||||
|
|
||||||
|
# 一次性打包
|
||||||
|
cd ..
|
||||||
|
mvn install
|
||||||
|
|
||||||
```
|
```
|
||||||
# application.yml 是配置文件，最简单的是仅修改MySQL相关的配置即可启动
|
|
||||||
|
|
||||||
nohup java -jar kafka-manager.jar --spring.config.location=./application.yml > /dev/null 2>&1 &
|
---
|
||||||
|
|
||||||
|
## 启动
|
||||||
|
|
||||||
|
```
|
||||||
|
# application.yml 是配置文件
|
||||||
|
|
||||||
|
cp kafka-manager-web/src/main/resources/application.yml kafka-manager-web/target/
|
||||||
|
cd kafka-manager-web/target/
|
||||||
|
nohup java -jar kafka-manager-web-2.0.0-SNAPSHOT.jar --spring.config.location=./application.yml > /dev/null 2>&1 &
|
||||||
```
|
```
|
||||||
|
|
||||||
### 5、使用
|
## 使用
|
||||||
|
|
||||||
本地启动的话，访问`http://localhost:8080`，输入帐号及密码(默认`admin/admin`)进行登录。更多参考：[kafka-manager 用户使用手册](../user_guide/user_guide_cn.md)
|
本地启动的话，访问`http://localhost:8080`，输入帐号及密码(默认`admin/admin`)进行登录。更多参考：[kafka-manager 用户使用手册](../user_guide/user_guide_cn.md)
|
||||||
|
|
||||||
|
|||||||
@@ -1,94 +0,0 @@
|
|||||||
---
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
**一站式`Apache Kafka`集群指标监控与运维管控平台**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## nginx配置-安装手册
|
|
||||||
|
|
||||||
# 一、独立部署
|
|
||||||
|
|
||||||
请参考参考：[kafka-manager 安装手册](install_guide_cn.md)
|
|
||||||
|
|
||||||
# 二、nginx配置
|
|
||||||
|
|
||||||
## 1、独立部署配置
|
|
||||||
|
|
||||||
```
|
|
||||||
#nginx 根目录访问配置如下
|
|
||||||
location / {
|
|
||||||
proxy_pass http://ip:port;
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## 2、前后端分离&配置多个静态资源
|
|
||||||
|
|
||||||
以下配置解决`nginx代理多个静态资源`，实现项目前后端分离，版本更新迭代。
|
|
||||||
|
|
||||||
### 1、源码下载
|
|
||||||
|
|
||||||
根据所需版本下载对应代码，下载地址：[Github 下载地址](https://github.com/didi/Logi-KafkaManager)
|
|
||||||
|
|
||||||
### 2、修改webpack.config.js 配置文件
|
|
||||||
|
|
||||||
修改`kafka-manager-console`模块 `webpack.config.js`
|
|
||||||
以下所有<font color='red'>xxxx</font>为nginx代理路径和打包静态文件加载前缀,<font color='red'>xxxx</font>可根据需求自行更改。
|
|
||||||
|
|
||||||
```
|
|
||||||
cd kafka-manager-console
|
|
||||||
vi webpack.config.js
|
|
||||||
|
|
||||||
# publicPath默认打包方式根目录下，修改为nginx代理访问路径。
|
|
||||||
let publicPath = '/xxxx';
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3、打包
|
|
||||||
|
|
||||||
```
|
|
||||||
|
|
||||||
npm cache clean --force && npm install
|
|
||||||
|
|
||||||
```
|
|
||||||
|
|
||||||
ps：如果打包过程中报错，运行`npm install clipboard@2.0.6`，相反请忽略！
|
|
||||||
|
|
||||||
### 4、部署
|
|
||||||
|
|
||||||
#### 1、前段静态文件部署
|
|
||||||
|
|
||||||
静态资源 `../kafka-manager-web/src/main/resources/templates`
|
|
||||||
|
|
||||||
上传到指定目录，目前以`root目录`做demo
|
|
||||||
|
|
||||||
#### 2、上传jar包并启动，请参考：[kafka-manager 安装手册](install_guide_cn.md)
|
|
||||||
|
|
||||||
#### 3、修改nginx 配置
|
|
||||||
|
|
||||||
```
|
|
||||||
location /xxxx {
|
|
||||||
# 静态文件存放位置
|
|
||||||
alias /root/templates;
|
|
||||||
try_files $uri $uri/ /xxxx/index.html;
|
|
||||||
index index.html;
|
|
||||||
}
|
|
||||||
|
|
||||||
location /api {
|
|
||||||
proxy_pass http://ip:port;
|
|
||||||
}
|
|
||||||
#后代端口建议使用/api，如果冲突可以使用以下配置
|
|
||||||
#location /api/v2 {
|
|
||||||
# proxy_pass http://ip:port;
|
|
||||||
#}
|
|
||||||
#location /api/v1 {
|
|
||||||
# proxy_pass http://ip:port;
|
|
||||||
#}
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@@ -5,26 +5,16 @@
|
|||||||
|
|
||||||
**一站式`Apache Kafka`集群指标监控与运维管控平台**
|
**一站式`Apache Kafka`集群指标监控与运维管控平台**
|
||||||
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# 集群接入
|
# 集群接入
|
||||||
|
|
||||||
## 主要概念讲解
|
|
||||||
面对大规模集群、业务场景复杂的情况，引入Region、逻辑集群的概念
|
|
||||||
- Region：划分部分Broker作为一个 Region，用Region定义资源划分的单位，提高扩展性和隔离性。如果部分Topic异常也不会影响大面积的Broker
|
|
||||||
- 逻辑集群：逻辑集群由部分Region组成，便于对大规模集群按照业务划分、保障能力进行管理
|
|
||||||
|
|
||||||
|
|
||||||
集群的接入总共需要三个步骤，分别是：
|
集群的接入总共需要三个步骤，分别是：
|
||||||
1. 接入物理集群：填写机器地址、安全协议等配置信息，接入真实的物理集群
|
1. 接入物理集群
|
||||||
2. 创建Region：将部分Broker划分为一个Region
|
2. 创建Region
|
||||||
3. 创建逻辑集群：逻辑集群由部分Region组成，可根据业务划分、保障等级来创建相应的逻辑集群
|
3. 创建逻辑集群
|
||||||
|
|
||||||
|
备注：接入集群需要2、3两步是因为普通用户的视角下，看到的都是逻辑集群，如果没有2、3两步，那么普通用户看不到任何信息。
|
||||||
|
|
||||||
|
|
||||||
**备注：接入集群需要2、3两步是因为普通用户的视角下，看到的都是逻辑集群，如果没有2、3两步，那么普通用户看不到任何信息。**
|
|
||||||
|
|
||||||
|
|
||||||
## 1、接入物理集群
|
## 1、接入物理集群
|
||||||
@@ -46,4 +36,4 @@
|
|||||||
|
|
||||||
|
|
||||||
|
|
||||||
如上图所示，填写逻辑集群信息，然后点击确定，即可完成逻辑集群的创建。
|
如上图所示，填写逻辑集群信息，然后点击确定，即可完成逻辑集群的创建。
|
||||||
Binary file not shown.
|
Before Width: | Height: | Size: 124 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 105 KiB |
@@ -1,25 +0,0 @@
|
|||||||
|
|
||||||
|
|
||||||
**一站式`Apache Kafka`集群指标监控与运维管控平台**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
|
|
||||||
## 报警策略-报警函数介绍
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
| 类别 | 函数 | 含义 |函数文案 |备注 |
|
|
||||||
| --- | --- | --- | --- | --- |
|
|
||||||
| 发生次数 |all，n | 最近$n个周期内，全发生 | 连续发生(all) | |
|
|
||||||
| 发生次数 | happen, n, m | 最近$n个周期内，发生m次 | 出现(happen) | null点也计算在n内 |
|
|
||||||
| 数学统计 | sum, n | 最近$n个周期取值 的 和 | 求和(sum) | sum_over_time |
|
|
||||||
| 数学统计 | avg, n | 最近$n个周期取值 的 平均值 | 平均值(avg) | avg_over_time |
|
|
||||||
| 数学统计 | min, n | 最近$n个周期取值 的 最小值 | 最小值(min) | min_over_time |
|
|
||||||
| 数学统计 | max, n | 最近$n个周期取值 的 最大值 | 最大值(max | max_over_time |
|
|
||||||
| 变化率 | pdiff, n | 最近$n个点的变化率, 有一个满足 则触发 | 突增突降率(pdiff) | 假设, 最近3个周期的值分别为 v, v2, v3（v为最新值）那么计算公式为 any( (v-v2)/v2, (v-v3)/v3 )**区分正负** |
|
|
||||||
| 变化量 | diff, n | 最近$n个点的变化量, 有一个满足 则触发 | 突增突降值(diff) | 假设, 最近3个周期的值分别为 v, v2, v3（v为最新值）那么计算公式为 any( (v-v2), (v-v3) )**区分正负** |
|
|
||||||
| 变化量 | ndiff | 最近n个周期，发生m次 v(t) - v(t-1) $OP threshold其中 v(t) 为最新值 | 连续变化（区分正负） - ndiff | |
|
|
||||||
| 数据中断 | nodata, t | 最近 $t 秒内 无数据上报 | 数据上报中断(nodata) | |
|
|
||||||
| 同环比 | c_avg_rate_abs, n | 最近$n个周期的取值，相比 1天或7天前取值 的变化率 的绝对值 | 同比变化率(c_avg_rate_abs) | 假设最近的n个值为 v1, v2, v3历史取到的对应n'个值为 v1', v2'那么计算公式为abs((avg(v1,v2,v3) / avg(v1',v2') -1)* 100%) |
|
|
||||||
| 同环比 | c_avg_rate, n | 最近$n个周期的取值，相比 1天或7天前取值 的变化率（**区分正负**) | 同比变化率(c_avg_rate) | 假设最近的n个值为 v1, v2, v3历史取到的对应n'个值为 v1', v2'那么计算公式为(avg(v1,v2,v3) / avg(v1',v2') -1)* 100% |
|
|
||||||
Binary file not shown.
|
Before Width: | Height: | Size: 119 KiB |
@@ -7,37 +7,20 @@
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# FAQ
|
# FAQ
|
||||||
|
|
||||||
- 0、支持哪些Kafka版本？
|
- 1、Topic申请时没有可选择的集群？
|
||||||
- 1、Topic申请、新建监控告警等操作时没有可选择的集群？
|
|
||||||
- 2、逻辑集群 & Region的用途？
|
- 2、逻辑集群 & Region的用途？
|
||||||
- 3、登录失败？
|
- 3、登录失败？
|
||||||
- 4、页面流量信息等无数据？
|
- 4、页面流量信息等无数据？
|
||||||
- 5、如何对接夜莺的监控告警功能？
|
- 5、如何对接夜莺的监控告警功能？
|
||||||
- 6、如何使用`MySQL 8`？
|
- 6、如何使用`MySQL 8`？
|
||||||
- 7、`Jmx`连接失败如何解决？
|
|
||||||
- 8、`topic biz data not exist`错误及处理方式
|
|
||||||
- 9、进程启动后，如何查看API文档
|
|
||||||
- 10、如何创建告警组？
|
|
||||||
- 11、连接信息、耗时信息为什么没有数据？
|
|
||||||
- 12、逻辑集群申请审批通过之后为什么看不到逻辑集群？
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 0、支持哪些Kafka版本？
|
### 1、Topic申请时没有可选择的集群？
|
||||||
|
|
||||||
基本上只要所使用的Kafka还依赖于Zookeeper，那么该版本的主要功能基本上应该就是支持的。
|
- 参看 [kafka-manager 接入集群](docs/user_guide/add_cluster/add_cluster.md) 手册，这里的Region和逻辑集群都必须添加。
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 1、Topic申请、新建监控告警等操作时没有可选择的集群？
|
|
||||||
|
|
||||||
缺少逻辑集群导致的，在Topic管理、监控告警、集群管理这三个Tab下面都是普通用户视角，普通用户看到的集群都是逻辑集群，因此在这三个Tab下进行操作时，都需要有逻辑集群。
|
|
||||||
|
|
||||||
逻辑集群的创建参看：
|
|
||||||
|
|
||||||
- [kafka-manager 接入集群](add_cluster/add_cluster.md) 手册，这里的Region和逻辑集群都必须添加。
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -60,7 +43,7 @@
|
|||||||
|
|
||||||
- 1、检查`Broker JMX`是否正确开启。
|
- 1、检查`Broker JMX`是否正确开启。
|
||||||
|
|
||||||
如若还未开启，具体可百度一下看如何开启，或者参看：[Jmx连接配置&问题解决说明文档](../dev_guide/connect_jmx_failed.md)
|
如若还未开启，具体可百度一下看如何开启。
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
@@ -70,59 +53,16 @@
|
|||||||
|
|
||||||
- 3、数据库时区问题。
|
- 3、数据库时区问题。
|
||||||
|
|
||||||
检查MySQL的topic_metrics表，查看是否有数据，如果有数据，那么再检查设置的时区是否正确。
|
检查MySQL的topic表，查看是否有数据，如果有数据，那么再检查设置的时区是否正确。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 5、如何对接夜莺的监控告警功能？
|
### 5、如何对接夜莺的监控告警功能？
|
||||||
|
|
||||||
- 参看 [kafka-manager 对接夜莺监控](../dev_guide/monitor_system_integrate_with_n9e.md) 说明。
|
- 参看 [kafka-manager 对接夜莺监控](../dev_guide/Intergration_n9e_monitor.md) 说明。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 6、如何使用`MySQL 8`？
|
### 6、如何使用`MySQL 8`？
|
||||||
|
|
||||||
- 参看 [kafka-manager 使用`MySQL 8`](../dev_guide/use_mysql_8.md) 说明。
|
- 参看 [kafka-manager 使用`MySQL 8`](../dev_guide/use_mysql_8.md) 说明。
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 7、`Jmx`连接失败如何解决？
|
|
||||||
|
|
||||||
- 参看 [Jmx连接配置&问题解决](../dev_guide/connect_jmx_failed.md) 说明。
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 8、`topic biz data not exist`错误及处理方式
|
|
||||||
|
|
||||||
**错误原因**
|
|
||||||
|
|
||||||
在进行权限审批的时候，可能会出现这个错误，出现这个错误的原因是因为Topic相关的业务信息没有在DB中存储，或者更具体的说就是该Topic不属于任何应用导致的，只需要将这些无主的Topic挂在某个应用下面即可。
|
|
||||||
|
|
||||||
**解决方式**
|
|
||||||
|
|
||||||
可以在`运维管控->集群列表->Topic信息`下面，编辑申请权限的Topic，为Topic选择一个应用即可。
|
|
||||||
|
|
||||||
以上仅仅只是针对单个Topic的场景，如果你有非常多的Topic需要进行初始化的，那么此时可以在配置管理中增加一个配置，来定时的对无主的Topic进行同步，具体见：[动态配置管理 - 1、Topic定时同步任务](../dev_guide/dynamic_config_manager.md)
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
### 9、进程启动后，如何查看API文档
|
|
||||||
|
|
||||||
- 滴滴Logi-KafkaManager采用Swagger-API工具记录API文档。Swagger-API地址： [http://IP:PORT/swagger-ui.html#/](http://IP:PORT/swagger-ui.html#/)
|
|
||||||
|
|
||||||
|
|
||||||
### 10、如何创建告警组？
|
|
||||||
|
|
||||||
这块需要配合监控系统进行使用，现在默认已经实现了夜莺的对接，当然也可以对接自己内部的监控系统，不过需要实现一些接口。
|
|
||||||
|
|
||||||
具体的文档可见：[监控功能对接夜莺](../dev_guide/monitor_system_integrate_with_n9e.md)、[监控功能对接其他系统](../dev_guide/monitor_system_integrate_with_self.md)
|
|
||||||
|
|
||||||
### 11、连接信息、耗时信息为什么没有数据？
|
|
||||||
|
|
||||||
这块需要结合滴滴内部的kafka-gateway一同使用才会有数据，滴滴kafka-gateway暂未开源。
|
|
||||||
|
|
||||||
### 12、逻辑集群申请审批通过之后为什么看不到逻辑集群？
|
|
||||||
|
|
||||||
逻辑集群的申请与审批仅仅只是一个工单流程，并不会去实际创建逻辑集群，逻辑集群的创建还需要手动去创建。
|
|
||||||
|
|
||||||
具体的操作可见：[kafka-manager 接入集群](add_cluster/add_cluster.md)。
|
|
||||||
|
|||||||
@@ -1,72 +0,0 @@
|
|||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
**一站式`Apache Kafka`集群指标监控与运维管控平台**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
# Topic 指标说明
|
|
||||||
|
|
||||||
## 1. 实时流量指标说明
|
|
||||||
|
|
||||||
|
|
||||||
| 指标名称| 单位| 指标含义|
|
|
||||||
|-- |---- |---|
|
|
||||||
| messagesIn| 条/s | 每秒发送到kafka的消息条数 |
|
|
||||||
| byteIn| B/s | 每秒发送到kafka的字节数 |
|
|
||||||
| byteOut| B/s | 每秒流出kafka的字节数（所有消费组消费的流量，如果是Kafka版本较低，这个还包括副本同步的流量） |
|
|
||||||
| byteRejected| B/s | 每秒被拒绝的字节数 |
|
|
||||||
| failedFetchRequest| qps | 每秒拉取失败的请求数 |
|
|
||||||
| failedProduceRequest| qps | 每秒发送失败的请求数 |
|
|
||||||
| totalProduceRequest| qps | 每秒总共发送的请求数，与messagesIn的区别是一个是发送请求里面可能会有多条消息 |
|
|
||||||
| totalFetchRequest| qps | 每秒总共拉取消息的请求数 |
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## 2. 历史流量指标说明
|
|
||||||
|
|
||||||
| 指标名称| 单位| 指标含义|
|
|
||||||
|-- |---- |---|
|
|
||||||
| messagesIn| 条/s | 近一分钟每秒发送到kafka的消息条数 |
|
|
||||||
| byteIn| B/s | 近一分钟每秒发送到kafka的字节数 |
|
|
||||||
| byteOut| B/s | 近一分钟每秒流出kafka的字节数（所有消费组消费的流量，如果是Kafka版本较低，副本同步的流量） |
|
|
||||||
| byteRejected| B/s | 近一分钟每秒被拒绝的字节数 |
|
|
||||||
| totalProduceRequest| qps | 近一分钟每秒总共发送的请求数，与messagesIn的区别是一个是发送请求里面可能会有多条消息 |
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
## 3. 实时耗时指标说明
|
|
||||||
|
|
||||||
**基于滴滴加强版Kafka引擎的特性，可以获取Broker的实时耗时信息和历史耗时信息**
|
|
||||||
|
|
||||||
| 指标名称| 单位 | 指标含义 | 耗时高原因 | 解决方案|
|
|
||||||
|-- |-- |-- |-- |--|
|
|
||||||
| RequestQueueTimeMs| ms | 请求队列排队时间 | 请求多，服务端处理不过来 | 联系运维人员处理 |
|
|
||||||
| LocalTimeMs| ms | Broker本地处理时间 | 服务端读写数据慢，可能是读写锁竞争 | 联系运维人员处理 |
|
|
||||||
| RemoteTimeMs| ms | 请求等待远程完成时间，对于发送请求，如果ack=-1，该时间表示副本同步时间，对于消费请求，如果当前没有数据，该时间为等待新数据时间，如果请求的版本与topic存储的版本不同，需要做版本转换，也会拉高该时间 | 对于生产，ack=-1必然会导致该指标耗时高，对于消费，如果topic数据写入很慢，该指标高也正常。如果需要版本转换，该指标耗时也会高 | 对于生产，可以考虑修改ack=1，消费端问题可以联系运维人员具体分析 |
|
|
||||||
| ThrottleTimeMs| ms | 请求限流时间 | 生产／消费被限流 | 申请提升限流值 |
|
|
||||||
| ResponseQueueTimeMs| ms | 响应队列排队时间 | 响应多，服务端处理不过来 | 联系运维人员处理 |
|
|
||||||
| ResponseSendTimeMs| ms | 响应返回客户端时间 | 1：下游消费能力差，导致向consumer发送数据时写网络缓冲区过慢；2：消费lag过大，一直从磁盘读取数据 | 1:提升客户端消费性能；2: 联系运维人员确认是否读取磁盘问题 |
|
|
||||||
| TotalTimeMs| ms | 接收到请求到完成总时间，理论上该时间等于上述六项时间之和，但由于各时间都是单独统计，总时间只是约等于上述六部分时间之和 | 上面六项有些耗时高 | 具体针对高的指标解决 |
|
|
||||||
|
|
||||||
**备注：由于kafka消费端实现方式，消费端一次会发送多个Fetch请求，在接收到一个Response之后就会开始处理数据，使Broker端返回其他Response等待，因此ResponseSendTimeMs并不完全是服务端发送时间，有时会包含一部分消费端处理数据时间**
|
|
||||||
|
|
||||||
## 4. 历史耗时指标说明
|
|
||||||
|
|
||||||
**基于滴滴加强版Kafka引擎的特性，可以获取Broker的实时耗时信息和历史耗时信息**
|
|
||||||
|
|
||||||
| 指标名称| 单位| 指标含义|
|
|
||||||
|-- | ---- |---|
|
|
||||||
| produceRequestTime99thPercentile|ms|Topic近一分钟发送99分位耗时|
|
|
||||||
| fetchRequestTime99thPercentile|ms|Topic近一分钟拉取99分位耗时|
|
|
||||||
| produceRequestTime95thPercentile|ms|Topic近一分钟发送95分位耗时|
|
|
||||||
| fetchRequestTime95thPercentile|ms|Topic近一分钟拉取95分位耗时|
|
|
||||||
| produceRequestTime75thPercentile|ms|Topic近一分钟发送75分位耗时|
|
|
||||||
| fetchRequestTime75thPercentile|ms|Topic近一分钟拉取75分位耗时|
|
|
||||||
| produceRequestTime50thPercentile|ms|Topic近一分钟发送50分位耗时|
|
|
||||||
| fetchRequestTime50thPercentile|ms|Topic近一分钟拉取50分位耗时|
|
|
||||||
|
|
||||||
@@ -1,32 +0,0 @@
|
|||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
**一站式`Apache Kafka`集群指标监控与运维管控平台**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
|
|
||||||
# 资源申请文档
|
|
||||||
|
|
||||||
## 主要名词解释
|
|
||||||
|
|
||||||
- 应用（App）：作为Kafka中的账户，使用AppID+password作为身份标识
|
|
||||||
- 集群：可使用平台提供的共享集群，也可为某一应用申请单独的集群
|
|
||||||
- Topic：可申请创建Topic或申请其他Topic的生产/消费权限。进行生产/消费时通过Topic+AppID进行身份鉴权
|
|
||||||
|
|
||||||
|
|
||||||
## 应用申请
|
|
||||||
应用（App）作为Kafka中的账户，使用AppID+password作为身份标识。对Topic进行生产/消费时通过Topic+AppID进行身份鉴权。
|
|
||||||
|
|
||||||
用户申请应用，经由运维人员审批，审批通过后获得AppID和密钥
|
|
||||||
|
|
||||||
## 集群申请
|
|
||||||
可使用平台提供的共享集群，若对隔离性、稳定性、生产消费速率有更高的需求，可对某一应用申请单独的集群
|
|
||||||
|
|
||||||
## Topic申请
|
|
||||||
- 用户可根据已申请的应用创建Topic。创建后，应用负责人默认拥有该Topic的生产/消费权限和管理权限
|
|
||||||
- 也可申请其他Topic的生产、消费权限。经由Topic所属应用的负责人审批后，即可拥有相应权限。
|
|
||||||
|
|
||||||
|
|
||||||
@@ -622,9 +622,6 @@ Lag：表示该消费客户端是否有堆积；等于 partition offset-consume
|
|||||||
|
|
||||||
<font size=2>步骤3：</font>填写完成后，点击提交即可提交申请。
|
<font size=2>步骤3：</font>填写完成后，点击提交即可提交申请。
|
||||||
|
|
||||||
备注说明：集群创建后，还需在此基础上创建region、逻辑集群。具体操作可参照 [集群接入手册](https://github.com/didi/Logi-KafkaManager/blob/master/docs/user_guide/add_cluster/add_cluster.md)
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
#### 申请集群下线 ####
|
#### 申请集群下线 ####
|
||||||
|
|||||||
@@ -5,13 +5,13 @@
|
|||||||
<modelVersion>4.0.0</modelVersion>
|
<modelVersion>4.0.0</modelVersion>
|
||||||
<groupId>com.xiaojukeji.kafka</groupId>
|
<groupId>com.xiaojukeji.kafka</groupId>
|
||||||
<artifactId>kafka-manager-common</artifactId>
|
<artifactId>kafka-manager-common</artifactId>
|
||||||
<version>${kafka-manager.revision}</version>
|
<version>2.0.0-SNAPSHOT</version>
|
||||||
<packaging>jar</packaging>
|
<packaging>jar</packaging>
|
||||||
|
|
||||||
<parent>
|
<parent>
|
||||||
<artifactId>kafka-manager</artifactId>
|
<artifactId>kafka-manager</artifactId>
|
||||||
<groupId>com.xiaojukeji.kafka</groupId>
|
<groupId>com.xiaojukeji.kafka</groupId>
|
||||||
<version>${kafka-manager.revision}</version>
|
<version>2.0.0-SNAPSHOT</version>
|
||||||
</parent>
|
</parent>
|
||||||
|
|
||||||
<properties>
|
<properties>
|
||||||
@@ -104,10 +104,5 @@
|
|||||||
<groupId>javax.servlet</groupId>
|
<groupId>javax.servlet</groupId>
|
||||||
<artifactId>javax.servlet-api</artifactId>
|
<artifactId>javax.servlet-api</artifactId>
|
||||||
</dependency>
|
</dependency>
|
||||||
|
|
||||||
<dependency>
|
|
||||||
<groupId>junit</groupId>
|
|
||||||
<artifactId>junit</artifactId>
|
|
||||||
</dependency>
|
|
||||||
</dependencies>
|
</dependencies>
|
||||||
</project>
|
</project>
|
||||||
@@ -47,13 +47,4 @@ public enum AccountRoleEnum {
|
|||||||
}
|
}
|
||||||
return AccountRoleEnum.UNKNOWN;
|
return AccountRoleEnum.UNKNOWN;
|
||||||
}
|
}
|
||||||
|
|
||||||
public static AccountRoleEnum getUserRoleEnum(String roleName) {
|
|
||||||
for (AccountRoleEnum elem: AccountRoleEnum.values()) {
|
|
||||||
if (elem.message.equalsIgnoreCase(roleName)) {
|
|
||||||
return elem;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return AccountRoleEnum.UNKNOWN;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -6,6 +6,8 @@ package com.xiaojukeji.kafka.manager.common.bizenum;
|
|||||||
*/
|
*/
|
||||||
public enum IDCEnum {
|
public enum IDCEnum {
|
||||||
CN("cn", "国内"),
|
CN("cn", "国内"),
|
||||||
|
US("us", "美东"),
|
||||||
|
RU("ru", "俄罗斯"),
|
||||||
;
|
;
|
||||||
|
|
||||||
private String idc;
|
private String idc;
|
||||||
|
|||||||
@@ -21,8 +21,6 @@ public enum ModuleEnum {
|
|||||||
|
|
||||||
PARTITION(5, "分区"),
|
PARTITION(5, "分区"),
|
||||||
|
|
||||||
GATEWAY_CONFIG(6, "Gateway配置"),
|
|
||||||
|
|
||||||
UNKNOWN(-1, "未知")
|
UNKNOWN(-1, "未知")
|
||||||
;
|
;
|
||||||
ModuleEnum(int code, String message) {
|
ModuleEnum(int code, String message) {
|
||||||
|
|||||||
@@ -46,7 +46,7 @@ public enum OperateEnum {
|
|||||||
|
|
||||||
public static boolean validate(Integer code) {
|
public static boolean validate(Integer code) {
|
||||||
if (code == null) {
|
if (code == null) {
|
||||||
return true;
|
return false;
|
||||||
}
|
}
|
||||||
for (OperateEnum state : OperateEnum.values()) {
|
for (OperateEnum state : OperateEnum.values()) {
|
||||||
if (state.getCode() == code) {
|
if (state.getCode() == code) {
|
||||||
|
|||||||
@@ -10,7 +10,6 @@ public enum RebalanceDimensionEnum {
|
|||||||
REGION(1, "Region维度"),
|
REGION(1, "Region维度"),
|
||||||
BROKER(2, "Broker维度"),
|
BROKER(2, "Broker维度"),
|
||||||
TOPIC(3, "Topic维度"),
|
TOPIC(3, "Topic维度"),
|
||||||
PARTITION(4, "Partition维度"),
|
|
||||||
;
|
;
|
||||||
|
|
||||||
private Integer code;
|
private Integer code;
|
||||||
|
|||||||
@@ -0,0 +1,45 @@
|
|||||||
|
package com.xiaojukeji.kafka.manager.common.bizenum;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 是否上报监控系统
|
||||||
|
* @author zengqiao
|
||||||
|
* @date 20/9/25
|
||||||
|
*/
|
||||||
|
public enum SinkMonitorSystemEnum {
|
||||||
|
SINK_MONITOR_SYSTEM(0, "上报监控系统"),
|
||||||
|
NOT_SINK_MONITOR_SYSTEM(1, "不上报监控系统"),
|
||||||
|
;
|
||||||
|
|
||||||
|
private Integer code;
|
||||||
|
|
||||||
|
private String message;
|
||||||
|
|
||||||
|
SinkMonitorSystemEnum(Integer code, String message) {
|
||||||
|
this.code = code;
|
||||||
|
this.message = message;
|
||||||
|
}
|
||||||
|
|
||||||
|
public Integer getCode() {
|
||||||
|
return code;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void setCode(Integer code) {
|
||||||
|
this.code = code;
|
||||||
|
}
|
||||||
|
|
||||||
|
public String getMessage() {
|
||||||
|
return message;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void setMessage(String message) {
|
||||||
|
this.message = message;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public String toString() {
|
||||||
|
return "SinkMonitorSystemEnum{" +
|
||||||
|
"code=" + code +
|
||||||
|
", message='" + message + '\'' +
|
||||||
|
'}';
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,32 +0,0 @@
|
|||||||
package com.xiaojukeji.kafka.manager.common.bizenum;
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 过期Topic状态
|
|
||||||
* @author zengqiao
|
|
||||||
* @date 21/01/25
|
|
||||||
*/
|
|
||||||
public enum TopicExpiredStatusEnum {
|
|
||||||
ALREADY_NOTIFIED_AND_DELETED(-2, "已通知, 已下线"),
|
|
||||||
ALREADY_NOTIFIED_AND_CAN_DELETE(-1, "已通知, 可下线"),
|
|
||||||
ALREADY_EXPIRED_AND_WAIT_NOTIFY(0, "已过期, 待通知"),
|
|
||||||
ALREADY_NOTIFIED_AND_WAIT_RESPONSE(1, "已通知, 待反馈"),
|
|
||||||
|
|
||||||
;
|
|
||||||
|
|
||||||
private int status;
|
|
||||||
|
|
||||||
private String message;
|
|
||||||
|
|
||||||
TopicExpiredStatusEnum(int status, String message) {
|
|
||||||
this.status = status;
|
|
||||||
this.message = message;
|
|
||||||
}
|
|
||||||
|
|
||||||
public int getStatus() {
|
|
||||||
return status;
|
|
||||||
}
|
|
||||||
|
|
||||||
public String getMessage() {
|
|
||||||
return message;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -45,13 +45,4 @@ public enum GatewayConfigKeyEnum {
|
|||||||
", configName='" + configName + '\'' +
|
", configName='" + configName + '\'' +
|
||||||
'}';
|
'}';
|
||||||
}
|
}
|
||||||
|
|
||||||
public static GatewayConfigKeyEnum getByConfigType(String configType) {
|
|
||||||
for (GatewayConfigKeyEnum configKeyEnum: GatewayConfigKeyEnum.values()) {
|
|
||||||
if (configKeyEnum.getConfigType().equals(configType)) {
|
|
||||||
return configKeyEnum;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
@@ -7,18 +7,18 @@ package com.xiaojukeji.kafka.manager.common.constant;
|
|||||||
*/
|
*/
|
||||||
public class ApiPrefix {
|
public class ApiPrefix {
|
||||||
public static final String API_PREFIX = "/api/";
|
public static final String API_PREFIX = "/api/";
|
||||||
private static final String API_V1_PREFIX = API_PREFIX + "v1/";
|
public static final String API_V1_PREFIX = API_PREFIX + "v1/";
|
||||||
|
public static final String API_V2_PREFIX = API_PREFIX + "v2/";
|
||||||
// login
|
|
||||||
public static final String API_V1_SSO_PREFIX = API_V1_PREFIX + "sso/";
|
|
||||||
|
|
||||||
// console
|
// console
|
||||||
|
public static final String API_V1_SSO_PREFIX = API_V1_PREFIX + "sso/";
|
||||||
public static final String API_V1_NORMAL_PREFIX = API_V1_PREFIX + "normal/";
|
public static final String API_V1_NORMAL_PREFIX = API_V1_PREFIX + "normal/";
|
||||||
public static final String API_V1_RD_PREFIX = API_V1_PREFIX + "rd/";
|
public static final String API_V1_RD_PREFIX = API_V1_PREFIX + "rd/";
|
||||||
public static final String API_V1_OP_PREFIX = API_V1_PREFIX + "op/";
|
public static final String API_V1_OP_PREFIX = API_V1_PREFIX + "op/";
|
||||||
|
|
||||||
// open
|
// open
|
||||||
public static final String API_V1_THIRD_PART_PREFIX = API_V1_PREFIX + "third-part/";
|
public static final String API_V1_THIRD_PART_PREFIX = API_V1_PREFIX + "third-part/";
|
||||||
|
public static final String API_V2_THIRD_PART_PREFIX = API_V2_PREFIX + "third-part/";
|
||||||
|
|
||||||
// gateway
|
// gateway
|
||||||
public static final String GATEWAY_API_V1_PREFIX = "/gateway" + API_V1_PREFIX;
|
public static final String GATEWAY_API_V1_PREFIX = "/gateway" + API_V1_PREFIX;
|
||||||
|
|||||||
@@ -7,8 +7,6 @@ package com.xiaojukeji.kafka.manager.common.constant;
|
|||||||
public class KafkaConstant {
|
public class KafkaConstant {
|
||||||
public static final String COORDINATOR_TOPIC_NAME = "__consumer_offsets";
|
public static final String COORDINATOR_TOPIC_NAME = "__consumer_offsets";
|
||||||
|
|
||||||
public static final String TRANSACTION_TOPIC_NAME = "__transaction_state";
|
|
||||||
|
|
||||||
public static final String BROKER_HOST_NAME_SUFFIX = ".diditaxi.com";
|
public static final String BROKER_HOST_NAME_SUFFIX = ".diditaxi.com";
|
||||||
|
|
||||||
public static final String CLIENT_VERSION_CODE_UNKNOWN = "-1";
|
public static final String CLIENT_VERSION_CODE_UNKNOWN = "-1";
|
||||||
|
|||||||
@@ -12,6 +12,11 @@ public class TopicCreationConstant {
|
|||||||
*/
|
*/
|
||||||
public static final String LOG_X_CREATE_TOPIC_CONFIG_KEY_NAME = "LOG_X_CREATE_TOPIC_CONFIG";
|
public static final String LOG_X_CREATE_TOPIC_CONFIG_KEY_NAME = "LOG_X_CREATE_TOPIC_CONFIG";
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 治理平台创建Topic配置KEY
|
||||||
|
*/
|
||||||
|
public static final String CHORUS_CREATE_TOPIC_CONFIG_KEY_NAME = "CHORUS_CREATE_TOPIC_CONFIG";
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 内部创建Topic配置KEY
|
* 内部创建Topic配置KEY
|
||||||
*/
|
*/
|
||||||
@@ -25,8 +30,6 @@ public class TopicCreationConstant {
|
|||||||
|
|
||||||
public static final String TOPIC_RETENTION_TIME_KEY_NAME = "retention.ms";
|
public static final String TOPIC_RETENTION_TIME_KEY_NAME = "retention.ms";
|
||||||
|
|
||||||
public static final Long DEFAULT_QUOTA = 3 * 1024 * 1024L;
|
|
||||||
|
|
||||||
public static Properties createNewProperties(Long retentionTime) {
|
public static Properties createNewProperties(Long retentionTime) {
|
||||||
Properties properties = new Properties();
|
Properties properties = new Properties();
|
||||||
properties.put(TOPIC_RETENTION_TIME_KEY_NAME, String.valueOf(retentionTime));
|
properties.put(TOPIC_RETENTION_TIME_KEY_NAME, String.valueOf(retentionTime));
|
||||||
@@ -43,15 +46,4 @@ public class TopicCreationConstant {
|
|||||||
public static final String TOPIC_NAME_PREFIX_RU = "ru01_";
|
public static final String TOPIC_NAME_PREFIX_RU = "ru01_";
|
||||||
|
|
||||||
public static final Integer TOPIC_NAME_MAX_LENGTH = 255;
|
public static final Integer TOPIC_NAME_MAX_LENGTH = 255;
|
||||||
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 单次自动化审批, 默认允许的通过单子
|
|
||||||
*/
|
|
||||||
public static final Integer DEFAULT_MAX_PASSED_ORDER_NUM_PER_TASK = 1;
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 单次自动化审批, 最多允许的通过单子
|
|
||||||
*/
|
|
||||||
public static final Integer MAX_PASSED_ORDER_NUM_PER_TASK = 200;
|
|
||||||
}
|
}
|
||||||
@@ -3,6 +3,7 @@ package com.xiaojukeji.kafka.manager.common.entity;
|
|||||||
import kafka.admin.AdminClient;
|
import kafka.admin.AdminClient;
|
||||||
|
|
||||||
import java.util.*;
|
import java.util.*;
|
||||||
|
import java.util.concurrent.ConcurrentHashMap;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author zengqiao
|
* @author zengqiao
|
||||||
@@ -15,12 +16,17 @@ public class ConsumerMetadata {
|
|||||||
|
|
||||||
private Map<String, AdminClient.ConsumerGroupSummary> consumerGroupSummaryMap = new HashMap<>();
|
private Map<String, AdminClient.ConsumerGroupSummary> consumerGroupSummaryMap = new HashMap<>();
|
||||||
|
|
||||||
|
private Map<String, List<String>> consumerGroupAppMap = new ConcurrentHashMap<>();
|
||||||
|
|
||||||
|
|
||||||
public ConsumerMetadata(Set<String> consumerGroupSet,
|
public ConsumerMetadata(Set<String> consumerGroupSet,
|
||||||
Map<String, Set<String>> topicNameConsumerGroupMap,
|
Map<String, Set<String>> topicNameConsumerGroupMap,
|
||||||
Map<String, AdminClient.ConsumerGroupSummary> consumerGroupSummaryMap) {
|
Map<String, AdminClient.ConsumerGroupSummary> consumerGroupSummaryMap,
|
||||||
|
Map<String, List<String>> consumerGroupAppMap) {
|
||||||
this.consumerGroupSet = consumerGroupSet;
|
this.consumerGroupSet = consumerGroupSet;
|
||||||
this.topicNameConsumerGroupMap = topicNameConsumerGroupMap;
|
this.topicNameConsumerGroupMap = topicNameConsumerGroupMap;
|
||||||
this.consumerGroupSummaryMap = consumerGroupSummaryMap;
|
this.consumerGroupSummaryMap = consumerGroupSummaryMap;
|
||||||
|
this.consumerGroupAppMap = consumerGroupAppMap;
|
||||||
}
|
}
|
||||||
|
|
||||||
public Set<String> getConsumerGroupSet() {
|
public Set<String> getConsumerGroupSet() {
|
||||||
@@ -34,4 +40,8 @@ public class ConsumerMetadata {
|
|||||||
public Map<String, AdminClient.ConsumerGroupSummary> getConsumerGroupSummaryMap() {
|
public Map<String, AdminClient.ConsumerGroupSummary> getConsumerGroupSummaryMap() {
|
||||||
return consumerGroupSummaryMap;
|
return consumerGroupSummaryMap;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public Map<String, List<String>> getConsumerGroupAppMap() {
|
||||||
|
return consumerGroupAppMap;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
@@ -1,7 +1,6 @@
|
|||||||
package com.xiaojukeji.kafka.manager.common.entity;
|
package com.xiaojukeji.kafka.manager.common.entity;
|
||||||
|
|
||||||
import com.alibaba.fastjson.JSON;
|
import com.alibaba.fastjson.JSON;
|
||||||
import com.xiaojukeji.kafka.manager.common.constant.Constant;
|
|
||||||
|
|
||||||
import java.io.Serializable;
|
import java.io.Serializable;
|
||||||
|
|
||||||
@@ -97,17 +96,9 @@ public class Result<T> implements Serializable {
|
|||||||
return result;
|
return result;
|
||||||
}
|
}
|
||||||
|
|
||||||
public static <T> Result<T> buildGatewayFailure(String message) {
|
|
||||||
Result<T> result = new Result<T>();
|
|
||||||
result.setCode(ResultStatus.GATEWAY_INVALID_REQUEST.getCode());
|
|
||||||
result.setMessage(message);
|
|
||||||
result.setData(null);
|
|
||||||
return result;
|
|
||||||
}
|
|
||||||
|
|
||||||
public static <T> Result<T> buildFailure(String message) {
|
public static <T> Result<T> buildFailure(String message) {
|
||||||
Result<T> result = new Result<T>();
|
Result<T> result = new Result<T>();
|
||||||
result.setCode(ResultStatus.FAIL.getCode());
|
result.setCode(ResultStatus.GATEWAY_INVALID_REQUEST.getCode());
|
||||||
result.setMessage(message);
|
result.setMessage(message);
|
||||||
result.setData(null);
|
result.setData(null);
|
||||||
return result;
|
return result;
|
||||||
@@ -127,9 +118,4 @@ public class Result<T> implements Serializable {
|
|||||||
result.setData(data);
|
result.setData(data);
|
||||||
return result;
|
return result;
|
||||||
}
|
}
|
||||||
|
|
||||||
public boolean failed() {
|
|
||||||
return !Constant.SUCCESS.equals(code);
|
|
||||||
}
|
|
||||||
|
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -12,105 +12,123 @@ public enum ResultStatus {
|
|||||||
|
|
||||||
SUCCESS(Constant.SUCCESS, "success"),
|
SUCCESS(Constant.SUCCESS, "success"),
|
||||||
|
|
||||||
FAIL(1, "操作失败"),
|
LOGIN_FAILED(1, "login failed, please check username and password"),
|
||||||
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 操作错误[1000, 2000)
|
* 内部依赖错误, [1000, 1200)
|
||||||
* ------------------------------------------------------------------------------------------
|
* ------------------------------------------------------------------------------------------
|
||||||
*/
|
*/
|
||||||
|
MYSQL_ERROR(1000, "operate database failed"),
|
||||||
|
|
||||||
|
CONNECT_ZOOKEEPER_FAILED(1000, "connect zookeeper failed"),
|
||||||
|
READ_ZOOKEEPER_FAILED(1000, "read zookeeper failed"),
|
||||||
|
READ_JMX_FAILED(1000, "read jmx failed"),
|
||||||
|
|
||||||
|
|
||||||
|
// 内部依赖错误 —— Kafka特定错误, [1000, 1100)
|
||||||
|
BROKER_NUM_NOT_ENOUGH(1000, "broker not enough"),
|
||||||
|
CONTROLLER_NOT_ALIVE(1000, "controller not alive"),
|
||||||
|
CLUSTER_METADATA_ERROR(1000, "cluster metadata error"),
|
||||||
|
TOPIC_CONFIG_ERROR(1000, "topic config error"),
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 外部依赖错误, [1200, 1400)
|
||||||
|
* ------------------------------------------------------------------------------------------
|
||||||
|
*/
|
||||||
|
CALL_CLUSTER_TASK_AGENT_FAILED(1000, " call cluster task agent failed"),
|
||||||
|
CALL_MONITOR_SYSTEM_ERROR(1000, " call monitor-system failed"),
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 外部用户操作错误, [1400, 1600)
|
||||||
|
* ------------------------------------------------------------------------------------------
|
||||||
|
*/
|
||||||
|
PARAM_ILLEGAL(1400, "param illegal"),
|
||||||
OPERATION_FAILED(1401, "operation failed"),
|
OPERATION_FAILED(1401, "operation failed"),
|
||||||
OPERATION_FORBIDDEN(1402, "operation forbidden"),
|
OPERATION_FORBIDDEN(1402, "operation forbidden"),
|
||||||
API_CALL_EXCEED_LIMIT(1403, "api call exceed limit"),
|
API_CALL_EXCEED_LIMIT(1403, "api call exceed limit"),
|
||||||
USER_WITHOUT_AUTHORITY(1404, "user without authority"),
|
|
||||||
CHANGE_ZOOKEEPER_FORBIDDEN(1405, "change zookeeper forbidden"),
|
|
||||||
|
|
||||||
|
// 资源不存在
|
||||||
|
CLUSTER_NOT_EXIST(10000, "cluster not exist"),
|
||||||
|
BROKER_NOT_EXIST(10000, "broker not exist"),
|
||||||
|
TOPIC_NOT_EXIST(10000, "topic not exist"),
|
||||||
|
PARTITION_NOT_EXIST(10000, "partition not exist"),
|
||||||
|
|
||||||
APP_OFFLINE_FORBIDDEN(1406, "先下线topic，才能下线应用～"),
|
ACCOUNT_NOT_EXIST(10000, "account not exist"),
|
||||||
|
APP_NOT_EXIST(1000, "app not exist"),
|
||||||
|
ORDER_NOT_EXIST(1000, "order not exist"),
|
||||||
|
CONFIG_NOT_EXIST(1000, "config not exist"),
|
||||||
|
IDC_NOT_EXIST(1000, "idc not exist"),
|
||||||
|
TASK_NOT_EXIST(1110, "task not exist"),
|
||||||
|
|
||||||
|
AUTHORITY_NOT_EXIST(1000, "authority not exist"),
|
||||||
|
|
||||||
TOPIC_OPERATION_PARAM_NULL_POINTER(1450, "参数错误"),
|
MONITOR_NOT_EXIST(1110, "monitor not exist"),
|
||||||
TOPIC_OPERATION_PARTITION_NUM_ILLEGAL(1451, "分区数错误"),
|
|
||||||
TOPIC_OPERATION_BROKER_NUM_NOT_ENOUGH(1452, "Broker数不足错误"),
|
|
||||||
TOPIC_OPERATION_TOPIC_NAME_ILLEGAL(1453, "Topic名称非法"),
|
|
||||||
TOPIC_OPERATION_TOPIC_EXISTED(1454, "Topic已存在"),
|
|
||||||
TOPIC_OPERATION_UNKNOWN_TOPIC_PARTITION(1455, "Topic未知"),
|
|
||||||
TOPIC_OPERATION_TOPIC_CONFIG_ILLEGAL(1456, "Topic配置错误"),
|
|
||||||
TOPIC_OPERATION_TOPIC_IN_DELETING(1457, "Topic正在删除"),
|
|
||||||
TOPIC_OPERATION_UNKNOWN_ERROR(1458, "未知错误"),
|
|
||||||
|
|
||||||
/**
|
QUOTA_NOT_EXIST(1000, "quota not exist, please check clusterId, topicName and appId"),
|
||||||
* 参数错误[2000, 3000)
|
|
||||||
* ------------------------------------------------------------------------------------------
|
|
||||||
*/
|
|
||||||
PARAM_ILLEGAL(2000, "param illegal"),
|
|
||||||
CG_LOCATION_ILLEGAL(2001, "consumer group location illegal"),
|
|
||||||
ORDER_ALREADY_HANDLED(2002, "order already handled"),
|
|
||||||
APP_ID_OR_PASSWORD_ILLEGAL(2003, "app or password illegal"),
|
|
||||||
SYSTEM_CODE_ILLEGAL(2004, "system code illegal"),
|
|
||||||
CLUSTER_TASK_HOST_LIST_ILLEGAL(2005, "主机列表错误，请检查主机列表"),
|
|
||||||
JSON_PARSER_ERROR(2006, "json parser error"),
|
|
||||||
|
|
||||||
BROKER_NUM_NOT_ENOUGH(2050, "broker not enough"),
|
// 资源不存在, 已存在, 已被使用
|
||||||
CONTROLLER_NOT_ALIVE(2051, "controller not alive"),
|
RESOURCE_NOT_EXIST(1200, "资源不存在"),
|
||||||
CLUSTER_METADATA_ERROR(2052, "cluster metadata error"),
|
RESOURCE_ALREADY_EXISTED(1200, "资源已经存在"),
|
||||||
TOPIC_CONFIG_ERROR(2053, "topic config error"),
|
RESOURCE_NAME_DUPLICATED(1200, "资源名称重复"),
|
||||||
|
RESOURCE_ALREADY_USED(1000, "资源早已被使用"),
|
||||||
/**
|
|
||||||
* 参数错误 - 资源检查错误
|
|
||||||
* 因为外部系统的问题, 操作时引起的错误, [7000, 8000)
|
|
||||||
* ------------------------------------------------------------------------------------------
|
|
||||||
*/
|
|
||||||
RESOURCE_NOT_EXIST(7100, "资源不存在"),
|
|
||||||
CLUSTER_NOT_EXIST(7101, "cluster not exist"),
|
|
||||||
BROKER_NOT_EXIST(7102, "broker not exist"),
|
|
||||||
TOPIC_NOT_EXIST(7103, "topic not exist"),
|
|
||||||
PARTITION_NOT_EXIST(7104, "partition not exist"),
|
|
||||||
ACCOUNT_NOT_EXIST(7105, "account not exist"),
|
|
||||||
APP_NOT_EXIST(7106, "app not exist"),
|
|
||||||
ORDER_NOT_EXIST(7107, "order not exist"),
|
|
||||||
CONFIG_NOT_EXIST(7108, "config not exist"),
|
|
||||||
IDC_NOT_EXIST(7109, "idc not exist"),
|
|
||||||
TASK_NOT_EXIST(7110, "task not exist"),
|
|
||||||
AUTHORITY_NOT_EXIST(7111, "authority not exist"),
|
|
||||||
MONITOR_NOT_EXIST(7112, "monitor not exist"),
|
|
||||||
QUOTA_NOT_EXIST(7113, "quota not exist, please check clusterId, topicName and appId"),
|
|
||||||
CONSUMER_GROUP_NOT_EXIST(7114, "consumerGroup not exist"),
|
|
||||||
TOPIC_BIZ_DATA_NOT_EXIST(7115, "topic biz data not exist, please sync topic to db"),
|
|
||||||
|
|
||||||
// 资源已存在
|
|
||||||
RESOURCE_ALREADY_EXISTED(7200, "资源已经存在"),
|
|
||||||
TOPIC_ALREADY_EXIST(7201, "topic already existed"),
|
|
||||||
|
|
||||||
// 资源重名
|
|
||||||
RESOURCE_NAME_DUPLICATED(7300, "资源名称重复"),
|
|
||||||
|
|
||||||
// 资源已被使用
|
|
||||||
RESOURCE_ALREADY_USED(7400, "资源早已被使用"),
|
|
||||||
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 因为外部系统的问题, 操作时引起的错误, [8000, 9000)
|
* 资源参数错误
|
||||||
* ------------------------------------------------------------------------------------------
|
|
||||||
*/
|
*/
|
||||||
MYSQL_ERROR(8010, "operate database failed"),
|
CG_LOCATION_ILLEGAL(10000, "consumer group location illegal"),
|
||||||
|
ORDER_ALREADY_HANDLED(1000, "order already handled"),
|
||||||
|
|
||||||
ZOOKEEPER_CONNECT_FAILED(8020, "zookeeper connect failed"),
|
APP_ID_OR_PASSWORD_ILLEGAL(1000, "app or password illegal"),
|
||||||
ZOOKEEPER_READ_FAILED(8021, "zookeeper read failed"),
|
SYSTEM_CODE_ILLEGAL(1000, "system code illegal"),
|
||||||
ZOOKEEPER_WRITE_FAILED(8022, "zookeeper write failed"),
|
|
||||||
ZOOKEEPER_DELETE_FAILED(8023, "zookeeper delete failed"),
|
|
||||||
|
|
||||||
// 调用集群任务里面的agent失败
|
|
||||||
CALL_CLUSTER_TASK_AGENT_FAILED(8030, " call cluster task agent failed"),
|
|
||||||
|
|
||||||
// 调用监控系统失败
|
|
||||||
CALL_MONITOR_SYSTEM_ERROR(8040, " call monitor-system failed"),
|
|
||||||
|
|
||||||
// 存储相关的调用失败
|
|
||||||
STORAGE_UPLOAD_FILE_FAILED(8050, "upload file failed"),
|
|
||||||
STORAGE_FILE_TYPE_NOT_SUPPORT(8051, "File type not support"),
|
|
||||||
STORAGE_DOWNLOAD_FILE_FAILED(8052, "download file failed"),
|
|
||||||
LDAP_AUTHENTICATION_FAILED(8053, "ldap authentication failed"),
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
///////////////////////////////////////////////////////////////
|
||||||
|
|
||||||
|
USER_WITHOUT_AUTHORITY(1000, "user without authority"),
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
JSON_PARSER_ERROR(1000, "json parser error"),
|
||||||
|
|
||||||
|
|
||||||
|
TOPIC_OPERATION_PARAM_NULL_POINTER(2, "参数错误"),
|
||||||
|
TOPIC_OPERATION_PARTITION_NUM_ILLEGAL(3, "分区数错误"),
|
||||||
|
TOPIC_OPERATION_BROKER_NUM_NOT_ENOUGH(4, "Broker数不足错误"),
|
||||||
|
TOPIC_OPERATION_TOPIC_NAME_ILLEGAL(5, "Topic名称非法"),
|
||||||
|
TOPIC_OPERATION_TOPIC_EXISTED(6, "Topic已存在"),
|
||||||
|
TOPIC_OPERATION_UNKNOWN_TOPIC_PARTITION(7, "Topic未知"),
|
||||||
|
TOPIC_OPERATION_TOPIC_CONFIG_ILLEGAL(8, "Topic配置错误"),
|
||||||
|
TOPIC_OPERATION_TOPIC_IN_DELETING(9, "Topic正在删除"),
|
||||||
|
TOPIC_OPERATION_UNKNOWN_ERROR(10, "未知错误"),
|
||||||
|
TOPIC_EXIST_CONNECT_CANNOT_DELETE(10, "topic exist connect cannot delete"),
|
||||||
|
EXIST_TOPIC_CANNOT_DELETE(10, "exist topic cannot delete"),
|
||||||
|
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 工单
|
||||||
|
*/
|
||||||
|
CHANGE_ZOOKEEPER_FORBIDEN(100, "change zookeeper forbiden"),
|
||||||
|
// APP_EXIST_TOPIC_AUTHORITY_CANNOT_DELETE(1000, "app exist topic authority cannot delete"),
|
||||||
|
|
||||||
|
UPLOAD_FILE_FAIL(1000, "upload file fail"),
|
||||||
|
FILE_TYPE_NOT_SUPPORT(1000, "File type not support"),
|
||||||
|
DOWNLOAD_FILE_FAIL(1000, "download file fail"),
|
||||||
|
|
||||||
|
|
||||||
|
TOPIC_ALREADY_EXIST(17400, "topic already existed"),
|
||||||
|
CONSUMER_GROUP_NOT_EXIST(17411, "consumerGroup not exist"),
|
||||||
;
|
;
|
||||||
|
|
||||||
private int code;
|
private int code;
|
||||||
|
|||||||
@@ -23,8 +23,6 @@ public class ClusterDetailDTO {
|
|||||||
|
|
||||||
private String securityProperties;
|
private String securityProperties;
|
||||||
|
|
||||||
private String jmxProperties;
|
|
||||||
|
|
||||||
private Integer status;
|
private Integer status;
|
||||||
|
|
||||||
private Date gmtCreate;
|
private Date gmtCreate;
|
||||||
@@ -105,14 +103,6 @@ public class ClusterDetailDTO {
|
|||||||
this.securityProperties = securityProperties;
|
this.securityProperties = securityProperties;
|
||||||
}
|
}
|
||||||
|
|
||||||
public String getJmxProperties() {
|
|
||||||
return jmxProperties;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setJmxProperties(String jmxProperties) {
|
|
||||||
this.jmxProperties = jmxProperties;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Integer getStatus() {
|
public Integer getStatus() {
|
||||||
return status;
|
return status;
|
||||||
}
|
}
|
||||||
@@ -186,9 +176,8 @@ public class ClusterDetailDTO {
|
|||||||
", bootstrapServers='" + bootstrapServers + '\'' +
|
", bootstrapServers='" + bootstrapServers + '\'' +
|
||||||
", kafkaVersion='" + kafkaVersion + '\'' +
|
", kafkaVersion='" + kafkaVersion + '\'' +
|
||||||
", idc='" + idc + '\'' +
|
", idc='" + idc + '\'' +
|
||||||
", mode=" + mode +
|
", mode='" + mode + '\'' +
|
||||||
", securityProperties='" + securityProperties + '\'' +
|
", securityProperties='" + securityProperties + '\'' +
|
||||||
", jmxProperties='" + jmxProperties + '\'' +
|
|
||||||
", status=" + status +
|
", status=" + status +
|
||||||
", gmtCreate=" + gmtCreate +
|
", gmtCreate=" + gmtCreate +
|
||||||
", gmtModify=" + gmtModify +
|
", gmtModify=" + gmtModify +
|
||||||
|
|||||||
@@ -1,6 +1,5 @@
|
|||||||
package com.xiaojukeji.kafka.manager.common.entity.ao;
|
package com.xiaojukeji.kafka.manager.common.entity.ao;
|
||||||
|
|
||||||
import java.util.List;
|
|
||||||
import java.util.Properties;
|
import java.util.Properties;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -24,8 +23,6 @@ public class RdTopicBasic {
|
|||||||
|
|
||||||
private String description;
|
private String description;
|
||||||
|
|
||||||
private List<String> regionNameList;
|
|
||||||
|
|
||||||
public Long getClusterId() {
|
public Long getClusterId() {
|
||||||
return clusterId;
|
return clusterId;
|
||||||
}
|
}
|
||||||
@@ -90,14 +87,6 @@ public class RdTopicBasic {
|
|||||||
this.description = description;
|
this.description = description;
|
||||||
}
|
}
|
||||||
|
|
||||||
public List<String> getRegionNameList() {
|
|
||||||
return regionNameList;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setRegionNameList(List<String> regionNameList) {
|
|
||||||
this.regionNameList = regionNameList;
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public String toString() {
|
public String toString() {
|
||||||
return "RdTopicBasic{" +
|
return "RdTopicBasic{" +
|
||||||
@@ -109,7 +98,6 @@ public class RdTopicBasic {
|
|||||||
", appName='" + appName + '\'' +
|
", appName='" + appName + '\'' +
|
||||||
", properties=" + properties +
|
", properties=" + properties +
|
||||||
", description='" + description + '\'' +
|
", description='" + description + '\'' +
|
||||||
", regionNameList='" + regionNameList + '\'' +
|
|
||||||
'}';
|
'}';
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1,53 +0,0 @@
|
|||||||
package com.xiaojukeji.kafka.manager.common.entity.ao.cluster;
|
|
||||||
|
|
||||||
public class ControllerPreferredCandidate {
|
|
||||||
private Integer brokerId;
|
|
||||||
|
|
||||||
private String host;
|
|
||||||
|
|
||||||
private Long startTime;
|
|
||||||
|
|
||||||
private Integer status;
|
|
||||||
|
|
||||||
public Integer getBrokerId() {
|
|
||||||
return brokerId;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setBrokerId(Integer brokerId) {
|
|
||||||
this.brokerId = brokerId;
|
|
||||||
}
|
|
||||||
|
|
||||||
public String getHost() {
|
|
||||||
return host;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setHost(String host) {
|
|
||||||
this.host = host;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Long getStartTime() {
|
|
||||||
return startTime;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setStartTime(Long startTime) {
|
|
||||||
this.startTime = startTime;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Integer getStatus() {
|
|
||||||
return status;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setStatus(Integer status) {
|
|
||||||
this.status = status;
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
|
||||||
public String toString() {
|
|
||||||
return "ControllerPreferredBroker{" +
|
|
||||||
"brokerId=" + brokerId +
|
|
||||||
", host='" + host + '\'' +
|
|
||||||
", startTime=" + startTime +
|
|
||||||
", status=" + status +
|
|
||||||
'}';
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -9,8 +9,6 @@ public class LogicalCluster {
|
|||||||
|
|
||||||
private String logicalClusterName;
|
private String logicalClusterName;
|
||||||
|
|
||||||
private String logicalClusterIdentification;
|
|
||||||
|
|
||||||
private Integer mode;
|
private Integer mode;
|
||||||
|
|
||||||
private Integer topicNum;
|
private Integer topicNum;
|
||||||
@@ -43,14 +41,6 @@ public class LogicalCluster {
|
|||||||
this.logicalClusterName = logicalClusterName;
|
this.logicalClusterName = logicalClusterName;
|
||||||
}
|
}
|
||||||
|
|
||||||
public String getLogicalClusterIdentification() {
|
|
||||||
return logicalClusterIdentification;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setLogicalClusterIdentification(String logicalClusterIdentification) {
|
|
||||||
this.logicalClusterIdentification = logicalClusterIdentification;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Integer getMode() {
|
public Integer getMode() {
|
||||||
return mode;
|
return mode;
|
||||||
}
|
}
|
||||||
@@ -91,14 +81,6 @@ public class LogicalCluster {
|
|||||||
this.bootstrapServers = bootstrapServers;
|
this.bootstrapServers = bootstrapServers;
|
||||||
}
|
}
|
||||||
|
|
||||||
public String getDescription() {
|
|
||||||
return description;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setDescription(String description) {
|
|
||||||
this.description = description;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Long getGmtCreate() {
|
public Long getGmtCreate() {
|
||||||
return gmtCreate;
|
return gmtCreate;
|
||||||
}
|
}
|
||||||
@@ -115,12 +97,19 @@ public class LogicalCluster {
|
|||||||
this.gmtModify = gmtModify;
|
this.gmtModify = gmtModify;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public String getDescription() {
|
||||||
|
return description;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void setDescription(String description) {
|
||||||
|
this.description = description;
|
||||||
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public String toString() {
|
public String toString() {
|
||||||
return "LogicalCluster{" +
|
return "LogicalCluster{" +
|
||||||
"logicalClusterId=" + logicalClusterId +
|
"logicalClusterId=" + logicalClusterId +
|
||||||
", logicalClusterName='" + logicalClusterName + '\'' +
|
", logicalClusterName='" + logicalClusterName + '\'' +
|
||||||
", logicalClusterIdentification='" + logicalClusterIdentification + '\'' +
|
|
||||||
", mode=" + mode +
|
", mode=" + mode +
|
||||||
", topicNum=" + topicNum +
|
", topicNum=" + topicNum +
|
||||||
", clusterVersion='" + clusterVersion + '\'' +
|
", clusterVersion='" + clusterVersion + '\'' +
|
||||||
|
|||||||
@@ -1,8 +1,5 @@
|
|||||||
package com.xiaojukeji.kafka.manager.common.entity.ao.config;
|
package com.xiaojukeji.kafka.manager.common.entity.ao.config;
|
||||||
|
|
||||||
import com.xiaojukeji.kafka.manager.common.constant.TopicCreationConstant;
|
|
||||||
import com.xiaojukeji.kafka.manager.common.utils.ValidateUtils;
|
|
||||||
|
|
||||||
import java.util.List;
|
import java.util.List;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -10,27 +7,8 @@ import java.util.List;
|
|||||||
* @date 20/7/24
|
* @date 20/7/24
|
||||||
*/
|
*/
|
||||||
public class CreateTopicConfig {
|
public class CreateTopicConfig {
|
||||||
/**
|
|
||||||
* 单次自动化审批, 允许的通过单子
|
|
||||||
*/
|
|
||||||
private Integer maxPassedOrderNumPerTask;
|
|
||||||
|
|
||||||
private List<CreateTopicElemConfig> configList;
|
private List<CreateTopicElemConfig> configList;
|
||||||
|
|
||||||
public Integer getMaxPassedOrderNumPerTask() {
|
|
||||||
if (ValidateUtils.isNull(maxPassedOrderNumPerTask)) {
|
|
||||||
return TopicCreationConstant.DEFAULT_MAX_PASSED_ORDER_NUM_PER_TASK;
|
|
||||||
}
|
|
||||||
if (maxPassedOrderNumPerTask > TopicCreationConstant.MAX_PASSED_ORDER_NUM_PER_TASK) {
|
|
||||||
return TopicCreationConstant.MAX_PASSED_ORDER_NUM_PER_TASK;
|
|
||||||
}
|
|
||||||
return maxPassedOrderNumPerTask;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setMaxPassedOrderNumPerTask(Integer maxPassedOrderNumPerTask) {
|
|
||||||
this.maxPassedOrderNumPerTask = maxPassedOrderNumPerTask;
|
|
||||||
}
|
|
||||||
|
|
||||||
public List<CreateTopicElemConfig> getConfigList() {
|
public List<CreateTopicElemConfig> getConfigList() {
|
||||||
return configList;
|
return configList;
|
||||||
}
|
}
|
||||||
@@ -42,8 +20,7 @@ public class CreateTopicConfig {
|
|||||||
@Override
|
@Override
|
||||||
public String toString() {
|
public String toString() {
|
||||||
return "CreateTopicConfig{" +
|
return "CreateTopicConfig{" +
|
||||||
"maxPassedOrderNumPerTask=" + maxPassedOrderNumPerTask +
|
"configList=" + configList +
|
||||||
", configList=" + configList +
|
|
||||||
'}';
|
'}';
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,57 @@
|
|||||||
|
package com.xiaojukeji.kafka.manager.common.entity.ao.config;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* @author zengqiao
|
||||||
|
* @date 20/9/7
|
||||||
|
*/
|
||||||
|
public class SinkTopicRequestTimeMetricsConfig {
|
||||||
|
private Long clusterId;
|
||||||
|
|
||||||
|
private String topicName;
|
||||||
|
|
||||||
|
private Long startId;
|
||||||
|
|
||||||
|
private Long step;
|
||||||
|
|
||||||
|
public Long getClusterId() {
|
||||||
|
return clusterId;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void setClusterId(Long clusterId) {
|
||||||
|
this.clusterId = clusterId;
|
||||||
|
}
|
||||||
|
|
||||||
|
public String getTopicName() {
|
||||||
|
return topicName;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void setTopicName(String topicName) {
|
||||||
|
this.topicName = topicName;
|
||||||
|
}
|
||||||
|
|
||||||
|
public Long getStartId() {
|
||||||
|
return startId;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void setStartId(Long startId) {
|
||||||
|
this.startId = startId;
|
||||||
|
}
|
||||||
|
|
||||||
|
public Long getStep() {
|
||||||
|
return step;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void setStep(Long step) {
|
||||||
|
this.step = step;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public String toString() {
|
||||||
|
return "SinkTopicRequestTimeMetricsConfig{" +
|
||||||
|
"clusterId=" + clusterId +
|
||||||
|
", topicName='" + topicName + '\'' +
|
||||||
|
", startId=" + startId +
|
||||||
|
", step=" + step +
|
||||||
|
'}';
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -2,18 +2,30 @@ package com.xiaojukeji.kafka.manager.common.entity.ao.consumer;
|
|||||||
|
|
||||||
import com.xiaojukeji.kafka.manager.common.bizenum.OffsetLocationEnum;
|
import com.xiaojukeji.kafka.manager.common.bizenum.OffsetLocationEnum;
|
||||||
|
|
||||||
|
import java.util.List;
|
||||||
import java.util.Objects;
|
import java.util.Objects;
|
||||||
|
|
||||||
public class ConsumerGroup {
|
/**
|
||||||
|
* 消费组信息
|
||||||
|
* @author zengqiao
|
||||||
|
* @date 19/4/18
|
||||||
|
*/
|
||||||
|
public class ConsumerGroupDTO {
|
||||||
private Long clusterId;
|
private Long clusterId;
|
||||||
|
|
||||||
private String consumerGroup;
|
private String consumerGroup;
|
||||||
|
|
||||||
|
private List<String> appIdList;
|
||||||
|
|
||||||
private OffsetLocationEnum offsetStoreLocation;
|
private OffsetLocationEnum offsetStoreLocation;
|
||||||
|
|
||||||
public ConsumerGroup(Long clusterId, String consumerGroup, OffsetLocationEnum offsetStoreLocation) {
|
public ConsumerGroupDTO(Long clusterId,
|
||||||
|
String consumerGroup,
|
||||||
|
List<String> appIdList,
|
||||||
|
OffsetLocationEnum offsetStoreLocation) {
|
||||||
this.clusterId = clusterId;
|
this.clusterId = clusterId;
|
||||||
this.consumerGroup = consumerGroup;
|
this.consumerGroup = consumerGroup;
|
||||||
|
this.appIdList = appIdList;
|
||||||
this.offsetStoreLocation = offsetStoreLocation;
|
this.offsetStoreLocation = offsetStoreLocation;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -33,6 +45,14 @@ public class ConsumerGroup {
|
|||||||
this.consumerGroup = consumerGroup;
|
this.consumerGroup = consumerGroup;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public List<String> getAppIdList() {
|
||||||
|
return appIdList;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void setAppIdList(List<String> appIdList) {
|
||||||
|
this.appIdList = appIdList;
|
||||||
|
}
|
||||||
|
|
||||||
public OffsetLocationEnum getOffsetStoreLocation() {
|
public OffsetLocationEnum getOffsetStoreLocation() {
|
||||||
return offsetStoreLocation;
|
return offsetStoreLocation;
|
||||||
}
|
}
|
||||||
@@ -43,9 +63,10 @@ public class ConsumerGroup {
|
|||||||
|
|
||||||
@Override
|
@Override
|
||||||
public String toString() {
|
public String toString() {
|
||||||
return "ConsumerGroup{" +
|
return "ConsumerGroupDTO{" +
|
||||||
"clusterId=" + clusterId +
|
"clusterId=" + clusterId +
|
||||||
", consumerGroup='" + consumerGroup + '\'' +
|
", consumerGroup='" + consumerGroup + '\'' +
|
||||||
|
", appIdList=" + appIdList +
|
||||||
", offsetStoreLocation=" + offsetStoreLocation +
|
", offsetStoreLocation=" + offsetStoreLocation +
|
||||||
'}';
|
'}';
|
||||||
}
|
}
|
||||||
@@ -58,7 +79,7 @@ public class ConsumerGroup {
|
|||||||
if (o == null || getClass() != o.getClass()) {
|
if (o == null || getClass() != o.getClass()) {
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
ConsumerGroup that = (ConsumerGroup) o;
|
ConsumerGroupDTO that = (ConsumerGroupDTO) o;
|
||||||
return clusterId.equals(that.clusterId)
|
return clusterId.equals(that.clusterId)
|
||||||
&& consumerGroup.equals(that.consumerGroup)
|
&& consumerGroup.equals(that.consumerGroup)
|
||||||
&& offsetStoreLocation == that.offsetStoreLocation;
|
&& offsetStoreLocation == that.offsetStoreLocation;
|
||||||
@@ -1,68 +0,0 @@
|
|||||||
package com.xiaojukeji.kafka.manager.common.entity.ao.consumer;
|
|
||||||
|
|
||||||
import com.xiaojukeji.kafka.manager.common.bizenum.OffsetLocationEnum;
|
|
||||||
|
|
||||||
import java.util.List;
|
|
||||||
|
|
||||||
public class ConsumerGroupSummary {
|
|
||||||
private Long clusterId;
|
|
||||||
|
|
||||||
private String consumerGroup;
|
|
||||||
|
|
||||||
private OffsetLocationEnum offsetStoreLocation;
|
|
||||||
|
|
||||||
private List<String> appIdList;
|
|
||||||
|
|
||||||
private String state;
|
|
||||||
|
|
||||||
public Long getClusterId() {
|
|
||||||
return clusterId;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setClusterId(Long clusterId) {
|
|
||||||
this.clusterId = clusterId;
|
|
||||||
}
|
|
||||||
|
|
||||||
public String getConsumerGroup() {
|
|
||||||
return consumerGroup;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setConsumerGroup(String consumerGroup) {
|
|
||||||
this.consumerGroup = consumerGroup;
|
|
||||||
}
|
|
||||||
|
|
||||||
public OffsetLocationEnum getOffsetStoreLocation() {
|
|
||||||
return offsetStoreLocation;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setOffsetStoreLocation(OffsetLocationEnum offsetStoreLocation) {
|
|
||||||
this.offsetStoreLocation = offsetStoreLocation;
|
|
||||||
}
|
|
||||||
|
|
||||||
public List<String> getAppIdList() {
|
|
||||||
return appIdList;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setAppIdList(List<String> appIdList) {
|
|
||||||
this.appIdList = appIdList;
|
|
||||||
}
|
|
||||||
|
|
||||||
public String getState() {
|
|
||||||
return state;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setState(String state) {
|
|
||||||
this.state = state;
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
|
||||||
public String toString() {
|
|
||||||
return "ConsumerGroupSummary{" +
|
|
||||||
"clusterId=" + clusterId +
|
|
||||||
", consumerGroup='" + consumerGroup + '\'' +
|
|
||||||
", offsetStoreLocation=" + offsetStoreLocation +
|
|
||||||
", appIdList=" + appIdList +
|
|
||||||
", state='" + state + '\'' +
|
|
||||||
'}';
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,7 +1,5 @@
|
|||||||
package com.xiaojukeji.kafka.manager.common.entity.ao.topic;
|
package com.xiaojukeji.kafka.manager.common.entity.ao.topic;
|
||||||
|
|
||||||
import java.util.List;
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author arthur
|
* @author arthur
|
||||||
* @date 2018/09/03
|
* @date 2018/09/03
|
||||||
@@ -19,7 +17,7 @@ public class TopicBasicDTO {
|
|||||||
|
|
||||||
private String description;
|
private String description;
|
||||||
|
|
||||||
private List<String> regionNameList;
|
private String region;
|
||||||
|
|
||||||
private Integer score;
|
private Integer score;
|
||||||
|
|
||||||
@@ -85,12 +83,12 @@ public class TopicBasicDTO {
|
|||||||
this.description = description;
|
this.description = description;
|
||||||
}
|
}
|
||||||
|
|
||||||
public List<String> getRegionNameList() {
|
public String getRegion() {
|
||||||
return regionNameList;
|
return region;
|
||||||
}
|
}
|
||||||
|
|
||||||
public void setRegionNameList(List<String> regionNameList) {
|
public void setRegion(String region) {
|
||||||
this.regionNameList = regionNameList;
|
this.region = region;
|
||||||
}
|
}
|
||||||
|
|
||||||
public Integer getScore() {
|
public Integer getScore() {
|
||||||
@@ -166,7 +164,7 @@ public class TopicBasicDTO {
|
|||||||
", principals='" + principals + '\'' +
|
", principals='" + principals + '\'' +
|
||||||
", topicName='" + topicName + '\'' +
|
", topicName='" + topicName + '\'' +
|
||||||
", description='" + description + '\'' +
|
", description='" + description + '\'' +
|
||||||
", regionNameList='" + regionNameList + '\'' +
|
", region='" + region + '\'' +
|
||||||
", score=" + score +
|
", score=" + score +
|
||||||
", topicCodeC='" + topicCodeC + '\'' +
|
", topicCodeC='" + topicCodeC + '\'' +
|
||||||
", partitionNum=" + partitionNum +
|
", partitionNum=" + partitionNum +
|
||||||
|
|||||||
@@ -18,8 +18,6 @@ public class TopicOverview {
|
|||||||
|
|
||||||
private Object byteIn;
|
private Object byteIn;
|
||||||
|
|
||||||
private Object byteOut;
|
|
||||||
|
|
||||||
private Object produceRequest;
|
private Object produceRequest;
|
||||||
|
|
||||||
private String appName;
|
private String appName;
|
||||||
@@ -80,14 +78,6 @@ public class TopicOverview {
|
|||||||
this.byteIn = byteIn;
|
this.byteIn = byteIn;
|
||||||
}
|
}
|
||||||
|
|
||||||
public Object getByteOut() {
|
|
||||||
return byteOut;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setByteOut(Object byteOut) {
|
|
||||||
this.byteOut = byteOut;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Object getProduceRequest() {
|
public Object getProduceRequest() {
|
||||||
return produceRequest;
|
return produceRequest;
|
||||||
}
|
}
|
||||||
@@ -145,7 +135,6 @@ public class TopicOverview {
|
|||||||
", partitionNum=" + partitionNum +
|
", partitionNum=" + partitionNum +
|
||||||
", retentionTime=" + retentionTime +
|
", retentionTime=" + retentionTime +
|
||||||
", byteIn=" + byteIn +
|
", byteIn=" + byteIn +
|
||||||
", byteOut=" + byteOut +
|
|
||||||
", produceRequest=" + produceRequest +
|
", produceRequest=" + produceRequest +
|
||||||
", appName='" + appName + '\'' +
|
", appName='" + appName + '\'' +
|
||||||
", appId='" + appId + '\'' +
|
", appId='" + appId + '\'' +
|
||||||
|
|||||||
@@ -1,45 +0,0 @@
|
|||||||
package com.xiaojukeji.kafka.manager.common.entity.dto.op;
|
|
||||||
|
|
||||||
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
|
|
||||||
import io.swagger.annotations.ApiModel;
|
|
||||||
import io.swagger.annotations.ApiModelProperty;
|
|
||||||
|
|
||||||
import java.util.List;
|
|
||||||
|
|
||||||
/**
|
|
||||||
* @author zengqiao
|
|
||||||
* @date 21/01/24
|
|
||||||
*/
|
|
||||||
@JsonIgnoreProperties(ignoreUnknown = true)
|
|
||||||
@ApiModel(description="优选为Controller的候选者")
|
|
||||||
public class ControllerPreferredCandidateDTO {
|
|
||||||
@ApiModelProperty(value="集群ID")
|
|
||||||
private Long clusterId;
|
|
||||||
|
|
||||||
@ApiModelProperty(value="优选为controller的BrokerId")
|
|
||||||
private List<Integer> brokerIdList;
|
|
||||||
|
|
||||||
public Long getClusterId() {
|
|
||||||
return clusterId;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setClusterId(Long clusterId) {
|
|
||||||
this.clusterId = clusterId;
|
|
||||||
}
|
|
||||||
|
|
||||||
public List<Integer> getBrokerIdList() {
|
|
||||||
return brokerIdList;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setBrokerIdList(List<Integer> brokerIdList) {
|
|
||||||
this.brokerIdList = brokerIdList;
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
|
||||||
public String toString() {
|
|
||||||
return "ControllerPreferredCandidateDTO{" +
|
|
||||||
"clusterId=" + clusterId +
|
|
||||||
", brokerIdList=" + brokerIdList +
|
|
||||||
'}';
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -25,10 +25,7 @@ public class RebalanceDTO {
|
|||||||
@ApiModelProperty(value = "TopicName")
|
@ApiModelProperty(value = "TopicName")
|
||||||
private String topicName;
|
private String topicName;
|
||||||
|
|
||||||
@ApiModelProperty(value = "分区ID")
|
@ApiModelProperty(value = "维度[0: Cluster维度, 1: Region维度, 2:Broker维度, 3:Topic维度]")
|
||||||
private Integer partitionId;
|
|
||||||
|
|
||||||
@ApiModelProperty(value = "维度[0: Cluster维度, 1: Region维度, 2:Broker维度, 3:Topic维度, 4:Partition纬度]")
|
|
||||||
private Integer dimension;
|
private Integer dimension;
|
||||||
|
|
||||||
public Long getClusterId() {
|
public Long getClusterId() {
|
||||||
@@ -63,14 +60,6 @@ public class RebalanceDTO {
|
|||||||
this.topicName = topicName;
|
this.topicName = topicName;
|
||||||
}
|
}
|
||||||
|
|
||||||
public Integer getPartitionId() {
|
|
||||||
return partitionId;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setPartitionId(Integer partitionId) {
|
|
||||||
this.partitionId = partitionId;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Integer getDimension() {
|
public Integer getDimension() {
|
||||||
return dimension;
|
return dimension;
|
||||||
}
|
}
|
||||||
@@ -79,12 +68,22 @@ public class RebalanceDTO {
|
|||||||
this.dimension = dimension;
|
this.dimension = dimension;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public String toString() {
|
||||||
|
return "RebalanceDTO{" +
|
||||||
|
"clusterId=" + clusterId +
|
||||||
|
", regionId=" + regionId +
|
||||||
|
", brokerId=" + brokerId +
|
||||||
|
", topicName='" + topicName + '\'' +
|
||||||
|
", dimension=" + dimension +
|
||||||
|
'}';
|
||||||
|
}
|
||||||
|
|
||||||
public boolean paramLegal() {
|
public boolean paramLegal() {
|
||||||
if (ValidateUtils.isNull(clusterId)
|
if (ValidateUtils.isNull(clusterId)
|
||||||
|| (RebalanceDimensionEnum.REGION.getCode().equals(dimension) && ValidateUtils.isNull(regionId))
|
|| RebalanceDimensionEnum.REGION.getCode().equals(dimension) && ValidateUtils.isNull(regionId)
|
||||||
|| (RebalanceDimensionEnum.BROKER.getCode().equals(dimension) && ValidateUtils.isNull(brokerId))
|
|| RebalanceDimensionEnum.BROKER.getCode().equals(dimension) && ValidateUtils.isNull(brokerId)
|
||||||
|| (RebalanceDimensionEnum.TOPIC.getCode().equals(dimension) && ValidateUtils.isNull(topicName))
|
|| RebalanceDimensionEnum.TOPIC.getCode().equals(dimension) && ValidateUtils.isNull(topicName) ) {
|
||||||
|| (RebalanceDimensionEnum.PARTITION.getCode().equals(dimension) && (ValidateUtils.isNull(topicName) || ValidateUtils.isNull(partitionId))) ) {
|
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
return true;
|
return true;
|
||||||
|
|||||||
@@ -40,9 +40,6 @@ public class TopicCreationDTO extends ClusterTopicDTO {
|
|||||||
@ApiModelProperty(value = "Topic属性列表")
|
@ApiModelProperty(value = "Topic属性列表")
|
||||||
private Properties properties;
|
private Properties properties;
|
||||||
|
|
||||||
@ApiModelProperty(value = "最大写入字节数")
|
|
||||||
private Long peakBytesIn;
|
|
||||||
|
|
||||||
public String getAppId() {
|
public String getAppId() {
|
||||||
return appId;
|
return appId;
|
||||||
}
|
}
|
||||||
@@ -107,14 +104,6 @@ public class TopicCreationDTO extends ClusterTopicDTO {
|
|||||||
this.properties = properties;
|
this.properties = properties;
|
||||||
}
|
}
|
||||||
|
|
||||||
public Long getPeakBytesIn() {
|
|
||||||
return peakBytesIn;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setPeakBytesIn(Long peakBytesIn) {
|
|
||||||
this.peakBytesIn = peakBytesIn;
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public String toString() {
|
public String toString() {
|
||||||
return "TopicCreationDTO{" +
|
return "TopicCreationDTO{" +
|
||||||
@@ -146,4 +135,4 @@ public class TopicCreationDTO extends ClusterTopicDTO {
|
|||||||
}
|
}
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -27,12 +27,9 @@ public class ClusterDTO {
|
|||||||
@ApiModelProperty(value="数据中心")
|
@ApiModelProperty(value="数据中心")
|
||||||
private String idc;
|
private String idc;
|
||||||
|
|
||||||
@ApiModelProperty(value="Kafka安全配置")
|
@ApiModelProperty(value="安全配置参数")
|
||||||
private String securityProperties;
|
private String securityProperties;
|
||||||
|
|
||||||
@ApiModelProperty(value="Jmx配置")
|
|
||||||
private String jmxProperties;
|
|
||||||
|
|
||||||
public Long getClusterId() {
|
public Long getClusterId() {
|
||||||
return clusterId;
|
return clusterId;
|
||||||
}
|
}
|
||||||
@@ -81,14 +78,6 @@ public class ClusterDTO {
|
|||||||
this.securityProperties = securityProperties;
|
this.securityProperties = securityProperties;
|
||||||
}
|
}
|
||||||
|
|
||||||
public String getJmxProperties() {
|
|
||||||
return jmxProperties;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setJmxProperties(String jmxProperties) {
|
|
||||||
this.jmxProperties = jmxProperties;
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public String toString() {
|
public String toString() {
|
||||||
return "ClusterDTO{" +
|
return "ClusterDTO{" +
|
||||||
@@ -98,15 +87,15 @@ public class ClusterDTO {
|
|||||||
", bootstrapServers='" + bootstrapServers + '\'' +
|
", bootstrapServers='" + bootstrapServers + '\'' +
|
||||||
", idc='" + idc + '\'' +
|
", idc='" + idc + '\'' +
|
||||||
", securityProperties='" + securityProperties + '\'' +
|
", securityProperties='" + securityProperties + '\'' +
|
||||||
", jmxProperties='" + jmxProperties + '\'' +
|
|
||||||
'}';
|
'}';
|
||||||
}
|
}
|
||||||
|
|
||||||
public boolean legal() {
|
public Boolean legal() {
|
||||||
if (ValidateUtils.isNull(clusterName)
|
if (ValidateUtils.isNull(clusterName)
|
||||||
|| ValidateUtils.isNull(zookeeper)
|
|| ValidateUtils.isNull(zookeeper)
|
||||||
|| ValidateUtils.isNull(idc)
|
|| ValidateUtils.isNull(idc)
|
||||||
|| ValidateUtils.isNull(bootstrapServers)) {
|
|| ValidateUtils.isNull(bootstrapServers)
|
||||||
|
) {
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
return true;
|
return true;
|
||||||
|
|||||||
@@ -1,7 +1,6 @@
|
|||||||
package com.xiaojukeji.kafka.manager.common.entity.dto.rd;
|
package com.xiaojukeji.kafka.manager.common.entity.dto.rd;
|
||||||
|
|
||||||
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
|
import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
|
||||||
import com.xiaojukeji.kafka.manager.common.bizenum.ClusterModeEnum;
|
|
||||||
import com.xiaojukeji.kafka.manager.common.utils.ValidateUtils;
|
import com.xiaojukeji.kafka.manager.common.utils.ValidateUtils;
|
||||||
import io.swagger.annotations.ApiModel;
|
import io.swagger.annotations.ApiModel;
|
||||||
import io.swagger.annotations.ApiModelProperty;
|
import io.swagger.annotations.ApiModelProperty;
|
||||||
@@ -21,9 +20,6 @@ public class LogicalClusterDTO {
|
|||||||
@ApiModelProperty(value = "名称")
|
@ApiModelProperty(value = "名称")
|
||||||
private String name;
|
private String name;
|
||||||
|
|
||||||
@ApiModelProperty(value = "集群标识, 用于告警的上报")
|
|
||||||
private String identification;
|
|
||||||
|
|
||||||
@ApiModelProperty(value = "集群模式")
|
@ApiModelProperty(value = "集群模式")
|
||||||
private Integer mode;
|
private Integer mode;
|
||||||
|
|
||||||
@@ -55,14 +51,6 @@ public class LogicalClusterDTO {
|
|||||||
this.name = name;
|
this.name = name;
|
||||||
}
|
}
|
||||||
|
|
||||||
public String getIdentification() {
|
|
||||||
return identification;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setIdentification(String identification) {
|
|
||||||
this.identification = identification;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Integer getMode() {
|
public Integer getMode() {
|
||||||
return mode;
|
return mode;
|
||||||
}
|
}
|
||||||
@@ -108,7 +96,6 @@ public class LogicalClusterDTO {
|
|||||||
return "LogicalClusterDTO{" +
|
return "LogicalClusterDTO{" +
|
||||||
"id=" + id +
|
"id=" + id +
|
||||||
", name='" + name + '\'' +
|
", name='" + name + '\'' +
|
||||||
", identification='" + identification + '\'' +
|
|
||||||
", mode=" + mode +
|
", mode=" + mode +
|
||||||
", clusterId=" + clusterId +
|
", clusterId=" + clusterId +
|
||||||
", regionIdList=" + regionIdList +
|
", regionIdList=" + regionIdList +
|
||||||
@@ -121,15 +108,11 @@ public class LogicalClusterDTO {
|
|||||||
if (ValidateUtils.isNull(clusterId)
|
if (ValidateUtils.isNull(clusterId)
|
||||||
|| ValidateUtils.isNull(clusterId)
|
|| ValidateUtils.isNull(clusterId)
|
||||||
|| ValidateUtils.isEmptyList(regionIdList)
|
|| ValidateUtils.isEmptyList(regionIdList)
|
||||||
|
|| ValidateUtils.isNull(appId)
|
||||||
|| ValidateUtils.isNull(mode)) {
|
|| ValidateUtils.isNull(mode)) {
|
||||||
return false;
|
return false;
|
||||||
}
|
}
|
||||||
if (!ClusterModeEnum.SHARED_MODE.getCode().equals(mode) && ValidateUtils.isNull(appId)) {
|
|
||||||
return false;
|
|
||||||
}
|
|
||||||
appId = ValidateUtils.isNull(appId)? "": appId;
|
|
||||||
description = ValidateUtils.isNull(description)? "": description;
|
description = ValidateUtils.isNull(description)? "": description;
|
||||||
identification = ValidateUtils.isNull(identification)? name: identification;
|
|
||||||
return true;
|
return true;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -81,6 +81,11 @@ public class OperateRecordDTO {
|
|||||||
}
|
}
|
||||||
|
|
||||||
public boolean legal() {
|
public boolean legal() {
|
||||||
return !ValidateUtils.isNull(moduleId) && ModuleEnum.validate(moduleId) && OperateEnum.validate(operateId);
|
if (!ModuleEnum.validate(moduleId) ||
|
||||||
|
(!ValidateUtils.isNull(operateId) && OperateEnum.validate(operateId))
|
||||||
|
) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
return true;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,7 +1,5 @@
|
|||||||
package com.xiaojukeji.kafka.manager.common.entity.metrics;
|
package com.xiaojukeji.kafka.manager.common.entity.metrics;
|
||||||
|
|
||||||
import java.util.List;
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author zengqiao
|
* @author zengqiao
|
||||||
* @date 20/6/17
|
* @date 20/6/17
|
||||||
@@ -13,8 +11,6 @@ public class TopicMetrics extends BaseMetrics {
|
|||||||
|
|
||||||
private String topicName;
|
private String topicName;
|
||||||
|
|
||||||
private List<BrokerMetrics> brokerMetricsList;
|
|
||||||
|
|
||||||
public TopicMetrics(Long clusterId, String topicName) {
|
public TopicMetrics(Long clusterId, String topicName) {
|
||||||
super();
|
super();
|
||||||
this.clusterId = clusterId;
|
this.clusterId = clusterId;
|
||||||
@@ -28,14 +24,6 @@ public class TopicMetrics extends BaseMetrics {
|
|||||||
this.topicName = topicName;
|
this.topicName = topicName;
|
||||||
}
|
}
|
||||||
|
|
||||||
public TopicMetrics(String appId, Long clusterId, String topicName, List<BrokerMetrics> brokerMetricsList) {
|
|
||||||
super();
|
|
||||||
this.appId = appId;
|
|
||||||
this.clusterId = clusterId;
|
|
||||||
this.topicName = topicName;
|
|
||||||
this.brokerMetricsList = brokerMetricsList;
|
|
||||||
}
|
|
||||||
|
|
||||||
public String getAppId() {
|
public String getAppId() {
|
||||||
return appId;
|
return appId;
|
||||||
}
|
}
|
||||||
@@ -48,14 +36,6 @@ public class TopicMetrics extends BaseMetrics {
|
|||||||
return topicName;
|
return topicName;
|
||||||
}
|
}
|
||||||
|
|
||||||
public void setBrokerMetricsList(List<BrokerMetrics> brokerMetricsList) {
|
|
||||||
this.brokerMetricsList = brokerMetricsList;
|
|
||||||
}
|
|
||||||
|
|
||||||
public List<BrokerMetrics> getBrokerMetricsList() {
|
|
||||||
return brokerMetricsList;
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public String toString() {
|
public String toString() {
|
||||||
return "TopicMetrics{" +
|
return "TopicMetrics{" +
|
||||||
|
|||||||
@@ -1,7 +1,6 @@
|
|||||||
package com.xiaojukeji.kafka.manager.common.entity.pojo;
|
package com.xiaojukeji.kafka.manager.common.entity.pojo;
|
||||||
|
|
||||||
import java.util.Date;
|
import java.util.Date;
|
||||||
import java.util.Objects;
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author zengqiao
|
* @author zengqiao
|
||||||
@@ -18,8 +17,6 @@ public class ClusterDO implements Comparable<ClusterDO> {
|
|||||||
|
|
||||||
private String securityProperties;
|
private String securityProperties;
|
||||||
|
|
||||||
private String jmxProperties;
|
|
||||||
|
|
||||||
private Integer status;
|
private Integer status;
|
||||||
|
|
||||||
private Date gmtCreate;
|
private Date gmtCreate;
|
||||||
@@ -34,6 +31,30 @@ public class ClusterDO implements Comparable<ClusterDO> {
|
|||||||
this.id = id;
|
this.id = id;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public Integer getStatus() {
|
||||||
|
return status;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void setStatus(Integer status) {
|
||||||
|
this.status = status;
|
||||||
|
}
|
||||||
|
|
||||||
|
public Date getGmtCreate() {
|
||||||
|
return gmtCreate;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void setGmtCreate(Date gmtCreate) {
|
||||||
|
this.gmtCreate = gmtCreate;
|
||||||
|
}
|
||||||
|
|
||||||
|
public Date getGmtModify() {
|
||||||
|
return gmtModify;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void setGmtModify(Date gmtModify) {
|
||||||
|
this.gmtModify = gmtModify;
|
||||||
|
}
|
||||||
|
|
||||||
public String getClusterName() {
|
public String getClusterName() {
|
||||||
return clusterName;
|
return clusterName;
|
||||||
}
|
}
|
||||||
@@ -66,38 +87,6 @@ public class ClusterDO implements Comparable<ClusterDO> {
|
|||||||
this.securityProperties = securityProperties;
|
this.securityProperties = securityProperties;
|
||||||
}
|
}
|
||||||
|
|
||||||
public String getJmxProperties() {
|
|
||||||
return jmxProperties;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setJmxProperties(String jmxProperties) {
|
|
||||||
this.jmxProperties = jmxProperties;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Integer getStatus() {
|
|
||||||
return status;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setStatus(Integer status) {
|
|
||||||
this.status = status;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Date getGmtCreate() {
|
|
||||||
return gmtCreate;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setGmtCreate(Date gmtCreate) {
|
|
||||||
this.gmtCreate = gmtCreate;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Date getGmtModify() {
|
|
||||||
return gmtModify;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setGmtModify(Date gmtModify) {
|
|
||||||
this.gmtModify = gmtModify;
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public String toString() {
|
public String toString() {
|
||||||
return "ClusterDO{" +
|
return "ClusterDO{" +
|
||||||
@@ -106,7 +95,6 @@ public class ClusterDO implements Comparable<ClusterDO> {
|
|||||||
", zookeeper='" + zookeeper + '\'' +
|
", zookeeper='" + zookeeper + '\'' +
|
||||||
", bootstrapServers='" + bootstrapServers + '\'' +
|
", bootstrapServers='" + bootstrapServers + '\'' +
|
||||||
", securityProperties='" + securityProperties + '\'' +
|
", securityProperties='" + securityProperties + '\'' +
|
||||||
", jmxProperties='" + jmxProperties + '\'' +
|
|
||||||
", status=" + status +
|
", status=" + status +
|
||||||
", gmtCreate=" + gmtCreate +
|
", gmtCreate=" + gmtCreate +
|
||||||
", gmtModify=" + gmtModify +
|
", gmtModify=" + gmtModify +
|
||||||
@@ -117,22 +105,4 @@ public class ClusterDO implements Comparable<ClusterDO> {
|
|||||||
public int compareTo(ClusterDO clusterDO) {
|
public int compareTo(ClusterDO clusterDO) {
|
||||||
return this.id.compareTo(clusterDO.id);
|
return this.id.compareTo(clusterDO.id);
|
||||||
}
|
}
|
||||||
|
|
||||||
@Override
|
|
||||||
public boolean equals(Object o) {
|
|
||||||
if (this == o) return true;
|
|
||||||
if (o == null || getClass() != o.getClass()) return false;
|
|
||||||
ClusterDO clusterDO = (ClusterDO) o;
|
|
||||||
return Objects.equals(id, clusterDO.id)
|
|
||||||
&& Objects.equals(clusterName, clusterDO.clusterName)
|
|
||||||
&& Objects.equals(zookeeper, clusterDO.zookeeper)
|
|
||||||
&& Objects.equals(bootstrapServers, clusterDO.bootstrapServers)
|
|
||||||
&& Objects.equals(securityProperties, clusterDO.securityProperties)
|
|
||||||
&& Objects.equals(jmxProperties, clusterDO.jmxProperties);
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
|
||||||
public int hashCode() {
|
|
||||||
return Objects.hash(id, clusterName, zookeeper, bootstrapServers, securityProperties, jmxProperties);
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
@@ -11,8 +11,6 @@ public class LogicalClusterDO {
|
|||||||
|
|
||||||
private String name;
|
private String name;
|
||||||
|
|
||||||
private String identification;
|
|
||||||
|
|
||||||
private Integer mode;
|
private Integer mode;
|
||||||
|
|
||||||
private String appId;
|
private String appId;
|
||||||
@@ -43,14 +41,6 @@ public class LogicalClusterDO {
|
|||||||
this.name = name;
|
this.name = name;
|
||||||
}
|
}
|
||||||
|
|
||||||
public String getIdentification() {
|
|
||||||
return identification;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setIdentification(String identification) {
|
|
||||||
this.identification = identification;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Integer getMode() {
|
public Integer getMode() {
|
||||||
return mode;
|
return mode;
|
||||||
}
|
}
|
||||||
@@ -112,7 +102,6 @@ public class LogicalClusterDO {
|
|||||||
return "LogicalClusterDO{" +
|
return "LogicalClusterDO{" +
|
||||||
"id=" + id +
|
"id=" + id +
|
||||||
", name='" + name + '\'' +
|
", name='" + name + '\'' +
|
||||||
", identification='" + identification + '\'' +
|
|
||||||
", mode=" + mode +
|
", mode=" + mode +
|
||||||
", appId='" + appId + '\'' +
|
", appId='" + appId + '\'' +
|
||||||
", clusterId=" + clusterId +
|
", clusterId=" + clusterId +
|
||||||
|
|||||||
@@ -1,7 +1,6 @@
|
|||||||
package com.xiaojukeji.kafka.manager.common.entity.pojo;
|
package com.xiaojukeji.kafka.manager.common.entity.pojo;
|
||||||
|
|
||||||
import com.xiaojukeji.kafka.manager.common.entity.dto.op.topic.TopicCreationDTO;
|
import com.xiaojukeji.kafka.manager.common.entity.dto.op.topic.TopicCreationDTO;
|
||||||
import com.xiaojukeji.kafka.manager.common.utils.ValidateUtils;
|
|
||||||
|
|
||||||
import java.util.Date;
|
import java.util.Date;
|
||||||
|
|
||||||
@@ -96,7 +95,6 @@ public class TopicDO {
|
|||||||
topicDO.setClusterId(dto.getClusterId());
|
topicDO.setClusterId(dto.getClusterId());
|
||||||
topicDO.setTopicName(dto.getTopicName());
|
topicDO.setTopicName(dto.getTopicName());
|
||||||
topicDO.setDescription(dto.getDescription());
|
topicDO.setDescription(dto.getDescription());
|
||||||
topicDO.setPeakBytesIn(ValidateUtils.isNull(dto.getPeakBytesIn()) ? -1L : dto.getPeakBytesIn());
|
|
||||||
return topicDO;
|
return topicDO;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -17,8 +17,6 @@ public class GatewayConfigDO {
|
|||||||
|
|
||||||
private Long version;
|
private Long version;
|
||||||
|
|
||||||
private String description;
|
|
||||||
|
|
||||||
private Date createTime;
|
private Date createTime;
|
||||||
|
|
||||||
private Date modifyTime;
|
private Date modifyTime;
|
||||||
@@ -63,14 +61,6 @@ public class GatewayConfigDO {
|
|||||||
this.version = version;
|
this.version = version;
|
||||||
}
|
}
|
||||||
|
|
||||||
public String getDescription() {
|
|
||||||
return description;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setDescription(String description) {
|
|
||||||
this.description = description;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Date getCreateTime() {
|
public Date getCreateTime() {
|
||||||
return createTime;
|
return createTime;
|
||||||
}
|
}
|
||||||
@@ -95,7 +85,6 @@ public class GatewayConfigDO {
|
|||||||
", name='" + name + '\'' +
|
", name='" + name + '\'' +
|
||||||
", value='" + value + '\'' +
|
", value='" + value + '\'' +
|
||||||
", version=" + version +
|
", version=" + version +
|
||||||
", description='" + description + '\'' +
|
|
||||||
", createTime=" + createTime +
|
", createTime=" + createTime +
|
||||||
", modifyTime=" + modifyTime +
|
", modifyTime=" + modifyTime +
|
||||||
'}';
|
'}';
|
||||||
|
|||||||
@@ -28,9 +28,6 @@ public class TopicOverviewVO {
|
|||||||
@ApiModelProperty(value = "每秒流入流量(B)")
|
@ApiModelProperty(value = "每秒流入流量(B)")
|
||||||
private Object byteIn;
|
private Object byteIn;
|
||||||
|
|
||||||
@ApiModelProperty(value = "每秒流出流量(B)")
|
|
||||||
private Object byteOut;
|
|
||||||
|
|
||||||
@ApiModelProperty(value = "发送请求数(个/秒)")
|
@ApiModelProperty(value = "发送请求数(个/秒)")
|
||||||
private Object produceRequest;
|
private Object produceRequest;
|
||||||
|
|
||||||
@@ -97,14 +94,6 @@ public class TopicOverviewVO {
|
|||||||
this.byteIn = byteIn;
|
this.byteIn = byteIn;
|
||||||
}
|
}
|
||||||
|
|
||||||
public Object getByteOut() {
|
|
||||||
return byteOut;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setByteOut(Object byteOut) {
|
|
||||||
this.byteOut = byteOut;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Object getProduceRequest() {
|
public Object getProduceRequest() {
|
||||||
return produceRequest;
|
return produceRequest;
|
||||||
}
|
}
|
||||||
@@ -162,7 +151,6 @@ public class TopicOverviewVO {
|
|||||||
", partitionNum=" + partitionNum +
|
", partitionNum=" + partitionNum +
|
||||||
", retentionTime=" + retentionTime +
|
", retentionTime=" + retentionTime +
|
||||||
", byteIn=" + byteIn +
|
", byteIn=" + byteIn +
|
||||||
", byteOut=" + byteOut +
|
|
||||||
", produceRequest=" + produceRequest +
|
", produceRequest=" + produceRequest +
|
||||||
", appName='" + appName + '\'' +
|
", appName='" + appName + '\'' +
|
||||||
", appId='" + appId + '\'' +
|
", appId='" + appId + '\'' +
|
||||||
|
|||||||
@@ -15,9 +15,6 @@ public class LogicClusterVO {
|
|||||||
@ApiModelProperty(value="逻辑集群名称")
|
@ApiModelProperty(value="逻辑集群名称")
|
||||||
private String clusterName;
|
private String clusterName;
|
||||||
|
|
||||||
@ApiModelProperty(value="逻辑标识")
|
|
||||||
private String clusterIdentification;
|
|
||||||
|
|
||||||
@ApiModelProperty(value="逻辑集群类型, 0:共享集群, 1:独享集群, 2:独立集群")
|
@ApiModelProperty(value="逻辑集群类型, 0:共享集群, 1:独享集群, 2:独立集群")
|
||||||
private Integer mode;
|
private Integer mode;
|
||||||
|
|
||||||
@@ -27,6 +24,9 @@ public class LogicClusterVO {
|
|||||||
@ApiModelProperty(value="集群版本")
|
@ApiModelProperty(value="集群版本")
|
||||||
private String clusterVersion;
|
private String clusterVersion;
|
||||||
|
|
||||||
|
@ApiModelProperty(value="物理集群ID")
|
||||||
|
private Long physicalClusterId;
|
||||||
|
|
||||||
@ApiModelProperty(value="集群服务地址")
|
@ApiModelProperty(value="集群服务地址")
|
||||||
private String bootstrapServers;
|
private String bootstrapServers;
|
||||||
|
|
||||||
@@ -55,22 +55,6 @@ public class LogicClusterVO {
|
|||||||
this.clusterName = clusterName;
|
this.clusterName = clusterName;
|
||||||
}
|
}
|
||||||
|
|
||||||
public String getClusterIdentification() {
|
|
||||||
return clusterIdentification;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setClusterIdentification(String clusterIdentification) {
|
|
||||||
this.clusterIdentification = clusterIdentification;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Integer getMode() {
|
|
||||||
return mode;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setMode(Integer mode) {
|
|
||||||
this.mode = mode;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Integer getTopicNum() {
|
public Integer getTopicNum() {
|
||||||
return topicNum;
|
return topicNum;
|
||||||
}
|
}
|
||||||
@@ -87,6 +71,14 @@ public class LogicClusterVO {
|
|||||||
this.clusterVersion = clusterVersion;
|
this.clusterVersion = clusterVersion;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public Long getPhysicalClusterId() {
|
||||||
|
return physicalClusterId;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void setPhysicalClusterId(Long physicalClusterId) {
|
||||||
|
this.physicalClusterId = physicalClusterId;
|
||||||
|
}
|
||||||
|
|
||||||
public String getBootstrapServers() {
|
public String getBootstrapServers() {
|
||||||
return bootstrapServers;
|
return bootstrapServers;
|
||||||
}
|
}
|
||||||
@@ -95,14 +87,6 @@ public class LogicClusterVO {
|
|||||||
this.bootstrapServers = bootstrapServers;
|
this.bootstrapServers = bootstrapServers;
|
||||||
}
|
}
|
||||||
|
|
||||||
public String getDescription() {
|
|
||||||
return description;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setDescription(String description) {
|
|
||||||
this.description = description;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Long getGmtCreate() {
|
public Long getGmtCreate() {
|
||||||
return gmtCreate;
|
return gmtCreate;
|
||||||
}
|
}
|
||||||
@@ -119,15 +103,32 @@ public class LogicClusterVO {
|
|||||||
this.gmtModify = gmtModify;
|
this.gmtModify = gmtModify;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public Integer getMode() {
|
||||||
|
return mode;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void setMode(Integer mode) {
|
||||||
|
this.mode = mode;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
public String getDescription() {
|
||||||
|
return description;
|
||||||
|
}
|
||||||
|
|
||||||
|
public void setDescription(String description) {
|
||||||
|
this.description = description;
|
||||||
|
}
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public String toString() {
|
public String toString() {
|
||||||
return "LogicClusterVO{" +
|
return "LogicClusterVO{" +
|
||||||
"clusterId=" + clusterId +
|
"clusterId=" + clusterId +
|
||||||
", clusterName='" + clusterName + '\'' +
|
", clusterName='" + clusterName + '\'' +
|
||||||
", clusterIdentification='" + clusterIdentification + '\'' +
|
|
||||||
", mode=" + mode +
|
", mode=" + mode +
|
||||||
", topicNum=" + topicNum +
|
", topicNum=" + topicNum +
|
||||||
", clusterVersion='" + clusterVersion + '\'' +
|
", clusterVersion='" + clusterVersion + '\'' +
|
||||||
|
", physicalClusterId=" + physicalClusterId +
|
||||||
", bootstrapServers='" + bootstrapServers + '\'' +
|
", bootstrapServers='" + bootstrapServers + '\'' +
|
||||||
", description='" + description + '\'' +
|
", description='" + description + '\'' +
|
||||||
", gmtCreate=" + gmtCreate +
|
", gmtCreate=" + gmtCreate +
|
||||||
|
|||||||
@@ -1,67 +0,0 @@
|
|||||||
package com.xiaojukeji.kafka.manager.common.entity.vo.normal.consumer;
|
|
||||||
|
|
||||||
import io.swagger.annotations.ApiModel;
|
|
||||||
import io.swagger.annotations.ApiModelProperty;
|
|
||||||
|
|
||||||
import java.util.List;
|
|
||||||
|
|
||||||
/**
|
|
||||||
* @author zengqiao
|
|
||||||
* @date 21/01/14
|
|
||||||
*/
|
|
||||||
@ApiModel(value = "Topic消费组概要信息")
|
|
||||||
public class ConsumerGroupSummaryVO {
|
|
||||||
@ApiModelProperty(value = "消费组名称")
|
|
||||||
private String consumerGroup;
|
|
||||||
|
|
||||||
@ApiModelProperty(value = "使用的AppID")
|
|
||||||
private String appIds;
|
|
||||||
|
|
||||||
@ApiModelProperty(value = "offset存储位置")
|
|
||||||
private String location;
|
|
||||||
|
|
||||||
@ApiModelProperty(value = "消费组状态")
|
|
||||||
private String state;
|
|
||||||
|
|
||||||
public String getConsumerGroup() {
|
|
||||||
return consumerGroup;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setConsumerGroup(String consumerGroup) {
|
|
||||||
this.consumerGroup = consumerGroup;
|
|
||||||
}
|
|
||||||
|
|
||||||
public String getAppIds() {
|
|
||||||
return appIds;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setAppIds(String appIds) {
|
|
||||||
this.appIds = appIds;
|
|
||||||
}
|
|
||||||
|
|
||||||
public String getLocation() {
|
|
||||||
return location;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setLocation(String location) {
|
|
||||||
this.location = location;
|
|
||||||
}
|
|
||||||
|
|
||||||
public String getState() {
|
|
||||||
return state;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setState(String state) {
|
|
||||||
this.state = state;
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
|
||||||
public String toString() {
|
|
||||||
return "ConsumerGroupSummaryVO{" +
|
|
||||||
"consumerGroup='" + consumerGroup + '\'' +
|
|
||||||
", appIds=" + appIds +
|
|
||||||
", location='" + location + '\'' +
|
|
||||||
", state='" + state + '\'' +
|
|
||||||
'}';
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -3,8 +3,6 @@ package com.xiaojukeji.kafka.manager.common.entity.vo.normal.topic;
|
|||||||
import io.swagger.annotations.ApiModel;
|
import io.swagger.annotations.ApiModel;
|
||||||
import io.swagger.annotations.ApiModelProperty;
|
import io.swagger.annotations.ApiModelProperty;
|
||||||
|
|
||||||
import java.util.List;
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Topic的基本信息
|
* Topic的基本信息
|
||||||
* @author zengqiao
|
* @author zengqiao
|
||||||
@@ -51,9 +49,6 @@ public class TopicBasicVO {
|
|||||||
@ApiModelProperty(value = "集群地址")
|
@ApiModelProperty(value = "集群地址")
|
||||||
private String bootstrapServers;
|
private String bootstrapServers;
|
||||||
|
|
||||||
@ApiModelProperty(value = "所属region")
|
|
||||||
private List<String> regionNameList;
|
|
||||||
|
|
||||||
public Long getClusterId() {
|
public Long getClusterId() {
|
||||||
return clusterId;
|
return clusterId;
|
||||||
}
|
}
|
||||||
@@ -158,14 +153,6 @@ public class TopicBasicVO {
|
|||||||
this.score = score;
|
this.score = score;
|
||||||
}
|
}
|
||||||
|
|
||||||
public List<String> getRegionNameList() {
|
|
||||||
return regionNameList;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setRegionNameList(List<String> regionNameList) {
|
|
||||||
this.regionNameList = regionNameList;
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public String toString() {
|
public String toString() {
|
||||||
return "TopicBasicVO{" +
|
return "TopicBasicVO{" +
|
||||||
@@ -182,7 +169,6 @@ public class TopicBasicVO {
|
|||||||
", topicCodeC='" + topicCodeC + '\'' +
|
", topicCodeC='" + topicCodeC + '\'' +
|
||||||
", description='" + description + '\'' +
|
", description='" + description + '\'' +
|
||||||
", bootstrapServers='" + bootstrapServers + '\'' +
|
", bootstrapServers='" + bootstrapServers + '\'' +
|
||||||
", regionNameList=" + regionNameList +
|
|
||||||
'}';
|
'}';
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,39 +0,0 @@
|
|||||||
package com.xiaojukeji.kafka.manager.common.entity.vo.normal.topic;
|
|
||||||
|
|
||||||
/**
|
|
||||||
* author: mrazkonglingxu
|
|
||||||
* Date: 2020/12/7
|
|
||||||
* Time: 7:40 下午
|
|
||||||
*/
|
|
||||||
public class TopicBrokerRequestTimeVO {
|
|
||||||
|
|
||||||
private Long clusterId;
|
|
||||||
|
|
||||||
private Integer brokerId;
|
|
||||||
|
|
||||||
private TopicRequestTimeDetailVO brokerRequestTime;
|
|
||||||
|
|
||||||
public Long getClusterId() {
|
|
||||||
return clusterId;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setClusterId(Long clusterId) {
|
|
||||||
this.clusterId = clusterId;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Integer getBrokerId() {
|
|
||||||
return brokerId;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setBrokerId(Integer brokerId) {
|
|
||||||
this.brokerId = brokerId;
|
|
||||||
}
|
|
||||||
|
|
||||||
public TopicRequestTimeDetailVO getBrokerRequestTime() {
|
|
||||||
return brokerRequestTime;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setBrokerRequestTime(TopicRequestTimeDetailVO brokerRequestTime) {
|
|
||||||
this.brokerRequestTime = brokerRequestTime;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -3,8 +3,6 @@ package com.xiaojukeji.kafka.manager.common.entity.vo.normal.topic;
|
|||||||
import io.swagger.annotations.ApiModel;
|
import io.swagger.annotations.ApiModel;
|
||||||
import io.swagger.annotations.ApiModelProperty;
|
import io.swagger.annotations.ApiModelProperty;
|
||||||
|
|
||||||
import java.util.List;
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* @author zengqiao
|
* @author zengqiao
|
||||||
* @date 20/4/8
|
* @date 20/4/8
|
||||||
@@ -35,8 +33,6 @@ public class TopicRequestTimeDetailVO {
|
|||||||
@ApiModelProperty(value = "totalTimeMs")
|
@ApiModelProperty(value = "totalTimeMs")
|
||||||
private Object totalTimeMs;
|
private Object totalTimeMs;
|
||||||
|
|
||||||
private List<TopicBrokerRequestTimeVO> brokerRequestTimeList;
|
|
||||||
|
|
||||||
public String getRequestTimeType() {
|
public String getRequestTimeType() {
|
||||||
return requestTimeType;
|
return requestTimeType;
|
||||||
}
|
}
|
||||||
@@ -101,14 +97,6 @@ public class TopicRequestTimeDetailVO {
|
|||||||
this.totalTimeMs = totalTimeMs;
|
this.totalTimeMs = totalTimeMs;
|
||||||
}
|
}
|
||||||
|
|
||||||
public List<TopicBrokerRequestTimeVO> getBrokerRequestTimeList() {
|
|
||||||
return brokerRequestTimeList;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setBrokerRequestTimeList(List<TopicBrokerRequestTimeVO> brokerRequestTimeList) {
|
|
||||||
this.brokerRequestTimeList = brokerRequestTimeList;
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public String toString() {
|
public String toString() {
|
||||||
return "TopicRequestTimeDetailVO{" +
|
return "TopicRequestTimeDetailVO{" +
|
||||||
|
|||||||
@@ -28,7 +28,7 @@ public class ExpiredTopicVO {
|
|||||||
@ApiModelProperty(value = "负责人")
|
@ApiModelProperty(value = "负责人")
|
||||||
private String principals;
|
private String principals;
|
||||||
|
|
||||||
@ApiModelProperty(value = "状态, -1:已通知可下线, 0:过期待通知, 1+:已通知待反馈")
|
@ApiModelProperty(value = "状态, -1:可下线, 0:过期待通知, 1+:已通知待反馈")
|
||||||
private Integer status;
|
private Integer status;
|
||||||
|
|
||||||
public Long getClusterId() {
|
public Long getClusterId() {
|
||||||
|
|||||||
@@ -1,115 +0,0 @@
|
|||||||
package com.xiaojukeji.kafka.manager.common.entity.vo.rd;
|
|
||||||
|
|
||||||
import io.swagger.annotations.ApiModel;
|
|
||||||
import io.swagger.annotations.ApiModelProperty;
|
|
||||||
|
|
||||||
import java.util.Date;
|
|
||||||
|
|
||||||
/**
|
|
||||||
* @author zengqiao
|
|
||||||
* @date 20/3/19
|
|
||||||
*/
|
|
||||||
@ApiModel(value = "GatewayConfigVO", description = "Gateway配置信息")
|
|
||||||
public class GatewayConfigVO {
|
|
||||||
@ApiModelProperty(value="ID")
|
|
||||||
private Long id;
|
|
||||||
|
|
||||||
@ApiModelProperty(value="配置类型")
|
|
||||||
private String type;
|
|
||||||
|
|
||||||
@ApiModelProperty(value="配置名称")
|
|
||||||
private String name;
|
|
||||||
|
|
||||||
@ApiModelProperty(value="配置值")
|
|
||||||
private String value;
|
|
||||||
|
|
||||||
@ApiModelProperty(value="版本")
|
|
||||||
private Long version;
|
|
||||||
|
|
||||||
@ApiModelProperty(value="描述说明")
|
|
||||||
private String description;
|
|
||||||
|
|
||||||
@ApiModelProperty(value="创建时间")
|
|
||||||
private Date createTime;
|
|
||||||
|
|
||||||
@ApiModelProperty(value="修改时间")
|
|
||||||
private Date modifyTime;
|
|
||||||
|
|
||||||
public Long getId() {
|
|
||||||
return id;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setId(Long id) {
|
|
||||||
this.id = id;
|
|
||||||
}
|
|
||||||
|
|
||||||
public String getType() {
|
|
||||||
return type;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setType(String type) {
|
|
||||||
this.type = type;
|
|
||||||
}
|
|
||||||
|
|
||||||
public String getName() {
|
|
||||||
return name;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setName(String name) {
|
|
||||||
this.name = name;
|
|
||||||
}
|
|
||||||
|
|
||||||
public String getValue() {
|
|
||||||
return value;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setValue(String value) {
|
|
||||||
this.value = value;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Long getVersion() {
|
|
||||||
return version;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setVersion(Long version) {
|
|
||||||
this.version = version;
|
|
||||||
}
|
|
||||||
|
|
||||||
public String getDescription() {
|
|
||||||
return description;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setDescription(String description) {
|
|
||||||
this.description = description;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Date getCreateTime() {
|
|
||||||
return createTime;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setCreateTime(Date createTime) {
|
|
||||||
this.createTime = createTime;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Date getModifyTime() {
|
|
||||||
return modifyTime;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setModifyTime(Date modifyTime) {
|
|
||||||
this.modifyTime = modifyTime;
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
|
||||||
public String toString() {
|
|
||||||
return "GatewayConfigVO{" +
|
|
||||||
"id=" + id +
|
|
||||||
", type='" + type + '\'' +
|
|
||||||
", name='" + name + '\'' +
|
|
||||||
", value='" + value + '\'' +
|
|
||||||
", version=" + version +
|
|
||||||
", description='" + description + '\'' +
|
|
||||||
", createTime=" + createTime +
|
|
||||||
", modifyTime=" + modifyTime +
|
|
||||||
'}';
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -3,7 +3,6 @@ package com.xiaojukeji.kafka.manager.common.entity.vo.rd;
|
|||||||
import io.swagger.annotations.ApiModel;
|
import io.swagger.annotations.ApiModel;
|
||||||
import io.swagger.annotations.ApiModelProperty;
|
import io.swagger.annotations.ApiModelProperty;
|
||||||
|
|
||||||
import java.util.List;
|
|
||||||
import java.util.Properties;
|
import java.util.Properties;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -36,9 +35,6 @@ public class RdTopicBasicVO {
|
|||||||
@ApiModelProperty(value = "备注")
|
@ApiModelProperty(value = "备注")
|
||||||
private String description;
|
private String description;
|
||||||
|
|
||||||
@ApiModelProperty(value = "所属region")
|
|
||||||
private List<String> regionNameList;
|
|
||||||
|
|
||||||
public Long getClusterId() {
|
public Long getClusterId() {
|
||||||
return clusterId;
|
return clusterId;
|
||||||
}
|
}
|
||||||
@@ -103,14 +99,6 @@ public class RdTopicBasicVO {
|
|||||||
this.description = description;
|
this.description = description;
|
||||||
}
|
}
|
||||||
|
|
||||||
public List<String> getRegionNameList() {
|
|
||||||
return regionNameList;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setRegionNameList(List<String> regionNameList) {
|
|
||||||
this.regionNameList = regionNameList;
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
@Override
|
||||||
public String toString() {
|
public String toString() {
|
||||||
return "RdTopicBasicVO{" +
|
return "RdTopicBasicVO{" +
|
||||||
@@ -122,7 +110,6 @@ public class RdTopicBasicVO {
|
|||||||
", appName='" + appName + '\'' +
|
", appName='" + appName + '\'' +
|
||||||
", properties=" + properties +
|
", properties=" + properties +
|
||||||
", description='" + description + '\'' +
|
", description='" + description + '\'' +
|
||||||
", regionNameList='" + regionNameList + '\'' +
|
|
||||||
'}';
|
'}';
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -32,12 +32,9 @@ public class ClusterBaseVO {
|
|||||||
@ApiModelProperty(value="集群类型")
|
@ApiModelProperty(value="集群类型")
|
||||||
private Integer mode;
|
private Integer mode;
|
||||||
|
|
||||||
@ApiModelProperty(value="Kafka安全配置")
|
@ApiModelProperty(value="安全配置参数")
|
||||||
private String securityProperties;
|
private String securityProperties;
|
||||||
|
|
||||||
@ApiModelProperty(value="Jmx配置")
|
|
||||||
private String jmxProperties;
|
|
||||||
|
|
||||||
@ApiModelProperty(value="1:监控中, 0:暂停监控")
|
@ApiModelProperty(value="1:监控中, 0:暂停监控")
|
||||||
private Integer status;
|
private Integer status;
|
||||||
|
|
||||||
@@ -111,14 +108,6 @@ public class ClusterBaseVO {
|
|||||||
this.securityProperties = securityProperties;
|
this.securityProperties = securityProperties;
|
||||||
}
|
}
|
||||||
|
|
||||||
public String getJmxProperties() {
|
|
||||||
return jmxProperties;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setJmxProperties(String jmxProperties) {
|
|
||||||
this.jmxProperties = jmxProperties;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Integer getStatus() {
|
public Integer getStatus() {
|
||||||
return status;
|
return status;
|
||||||
}
|
}
|
||||||
@@ -152,9 +141,8 @@ public class ClusterBaseVO {
|
|||||||
", bootstrapServers='" + bootstrapServers + '\'' +
|
", bootstrapServers='" + bootstrapServers + '\'' +
|
||||||
", kafkaVersion='" + kafkaVersion + '\'' +
|
", kafkaVersion='" + kafkaVersion + '\'' +
|
||||||
", idc='" + idc + '\'' +
|
", idc='" + idc + '\'' +
|
||||||
", mode=" + mode +
|
", mode='" + mode + '\'' +
|
||||||
", securityProperties='" + securityProperties + '\'' +
|
", securityProperties='" + securityProperties + '\'' +
|
||||||
", jmxProperties='" + jmxProperties + '\'' +
|
|
||||||
", status=" + status +
|
", status=" + status +
|
||||||
", gmtCreate=" + gmtCreate +
|
", gmtCreate=" + gmtCreate +
|
||||||
", gmtModify=" + gmtModify +
|
", gmtModify=" + gmtModify +
|
||||||
|
|||||||
@@ -1,61 +0,0 @@
|
|||||||
package com.xiaojukeji.kafka.manager.common.entity.vo.rd.cluster;
|
|
||||||
|
|
||||||
import io.swagger.annotations.ApiModel;
|
|
||||||
import io.swagger.annotations.ApiModelProperty;
|
|
||||||
|
|
||||||
@ApiModel(description = "Broker基本信息")
|
|
||||||
public class ControllerPreferredCandidateVO {
|
|
||||||
@ApiModelProperty(value = "brokerId")
|
|
||||||
private Integer brokerId;
|
|
||||||
|
|
||||||
@ApiModelProperty(value = "主机名")
|
|
||||||
private String host;
|
|
||||||
|
|
||||||
@ApiModelProperty(value = "启动时间")
|
|
||||||
private Long startTime;
|
|
||||||
|
|
||||||
@ApiModelProperty(value = "broker状态[0:在线, -1:不在线]")
|
|
||||||
private Integer status;
|
|
||||||
|
|
||||||
public Integer getBrokerId() {
|
|
||||||
return brokerId;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setBrokerId(Integer brokerId) {
|
|
||||||
this.brokerId = brokerId;
|
|
||||||
}
|
|
||||||
|
|
||||||
public String getHost() {
|
|
||||||
return host;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setHost(String host) {
|
|
||||||
this.host = host;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Long getStartTime() {
|
|
||||||
return startTime;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setStartTime(Long startTime) {
|
|
||||||
this.startTime = startTime;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Integer getStatus() {
|
|
||||||
return status;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setStatus(Integer status) {
|
|
||||||
this.status = status;
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
|
||||||
public String toString() {
|
|
||||||
return "ControllerPreferredBrokerVO{" +
|
|
||||||
"brokerId=" + brokerId +
|
|
||||||
", host='" + host + '\'' +
|
|
||||||
", startTime=" + startTime +
|
|
||||||
", status=" + status +
|
|
||||||
'}';
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -18,9 +18,6 @@ public class LogicalClusterVO {
|
|||||||
@ApiModelProperty(value = "逻辑集群名称")
|
@ApiModelProperty(value = "逻辑集群名称")
|
||||||
private String logicalClusterName;
|
private String logicalClusterName;
|
||||||
|
|
||||||
@ApiModelProperty(value = "逻辑集群标识")
|
|
||||||
private String logicalClusterIdentification;
|
|
||||||
|
|
||||||
@ApiModelProperty(value = "物理集群ID")
|
@ApiModelProperty(value = "物理集群ID")
|
||||||
private Long physicalClusterId;
|
private Long physicalClusterId;
|
||||||
|
|
||||||
@@ -58,14 +55,6 @@ public class LogicalClusterVO {
|
|||||||
this.logicalClusterName = logicalClusterName;
|
this.logicalClusterName = logicalClusterName;
|
||||||
}
|
}
|
||||||
|
|
||||||
public String getLogicalClusterIdentification() {
|
|
||||||
return logicalClusterIdentification;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setLogicalClusterIdentification(String logicalClusterIdentification) {
|
|
||||||
this.logicalClusterIdentification = logicalClusterIdentification;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Long getPhysicalClusterId() {
|
public Long getPhysicalClusterId() {
|
||||||
return physicalClusterId;
|
return physicalClusterId;
|
||||||
}
|
}
|
||||||
@@ -127,7 +116,6 @@ public class LogicalClusterVO {
|
|||||||
return "LogicalClusterVO{" +
|
return "LogicalClusterVO{" +
|
||||||
"logicalClusterId=" + logicalClusterId +
|
"logicalClusterId=" + logicalClusterId +
|
||||||
", logicalClusterName='" + logicalClusterName + '\'' +
|
", logicalClusterName='" + logicalClusterName + '\'' +
|
||||||
", logicalClusterIdentification='" + logicalClusterIdentification + '\'' +
|
|
||||||
", physicalClusterId=" + physicalClusterId +
|
", physicalClusterId=" + physicalClusterId +
|
||||||
", regionIdList=" + regionIdList +
|
", regionIdList=" + regionIdList +
|
||||||
", mode=" + mode +
|
", mode=" + mode +
|
||||||
|
|||||||
@@ -9,7 +9,6 @@ import com.xiaojukeji.kafka.manager.common.entity.pojo.gateway.TopicConnectionDO
|
|||||||
|
|
||||||
import java.lang.reflect.Method;
|
import java.lang.reflect.Method;
|
||||||
import java.util.ArrayList;
|
import java.util.ArrayList;
|
||||||
import java.util.Date;
|
|
||||||
import java.util.List;
|
import java.util.List;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -53,21 +52,7 @@ public class JsonUtils {
|
|||||||
return JSON.toJSONString(obj);
|
return JSON.toJSONString(obj);
|
||||||
}
|
}
|
||||||
|
|
||||||
public static <T> T stringToObj(String src, Class<T> clazz) {
|
public static List<TopicConnectionDO> parseTopicConnections(Long clusterId, JSONObject jsonObject) {
|
||||||
if (ValidateUtils.isBlank(src)) {
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
return JSON.parseObject(src, clazz);
|
|
||||||
}
|
|
||||||
|
|
||||||
public static <T> List<T> stringToArrObj(String src, Class<T> clazz) {
|
|
||||||
if (ValidateUtils.isBlank(src)) {
|
|
||||||
return null;
|
|
||||||
}
|
|
||||||
return JSON.parseArray(src, clazz);
|
|
||||||
}
|
|
||||||
|
|
||||||
public static List<TopicConnectionDO> parseTopicConnections(Long clusterId, JSONObject jsonObject, long postTime) {
|
|
||||||
List<TopicConnectionDO> connectionDOList = new ArrayList<>();
|
List<TopicConnectionDO> connectionDOList = new ArrayList<>();
|
||||||
for (String clientType: jsonObject.keySet()) {
|
for (String clientType: jsonObject.keySet()) {
|
||||||
JSONObject topicObject = jsonObject.getJSONObject(clientType);
|
JSONObject topicObject = jsonObject.getJSONObject(clientType);
|
||||||
@@ -79,7 +64,7 @@ public class JsonUtils {
|
|||||||
TopicConnectionDO connectionDO = new TopicConnectionDO();
|
TopicConnectionDO connectionDO = new TopicConnectionDO();
|
||||||
|
|
||||||
String[] appIdDetailArray = appIdDetail.toString().split("#");
|
String[] appIdDetailArray = appIdDetail.toString().split("#");
|
||||||
if (appIdDetailArray.length >= 3) {
|
if (appIdDetailArray.length == 3) {
|
||||||
connectionDO.setAppId(appIdDetailArray[0]);
|
connectionDO.setAppId(appIdDetailArray[0]);
|
||||||
connectionDO.setIp(appIdDetailArray[1]);
|
connectionDO.setIp(appIdDetailArray[1]);
|
||||||
connectionDO.setClientVersion(appIdDetailArray[2]);
|
connectionDO.setClientVersion(appIdDetailArray[2]);
|
||||||
@@ -88,7 +73,6 @@ public class JsonUtils {
|
|||||||
connectionDO.setClusterId(clusterId);
|
connectionDO.setClusterId(clusterId);
|
||||||
connectionDO.setTopicName(topicName);
|
connectionDO.setTopicName(topicName);
|
||||||
connectionDO.setType(clientType);
|
connectionDO.setType(clientType);
|
||||||
connectionDO.setCreateTime(new Date(postTime));
|
|
||||||
connectionDOList.add(connectionDO);
|
connectionDOList.add(connectionDO);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,8 +1,9 @@
|
|||||||
package com.xiaojukeji.kafka.manager.common.utils;
|
package com.xiaojukeji.kafka.manager.common.utils;
|
||||||
|
|
||||||
|
import com.xiaojukeji.kafka.manager.common.bizenum.IDCEnum;
|
||||||
|
import com.xiaojukeji.kafka.manager.common.constant.TopicCreationConstant;
|
||||||
import org.apache.commons.lang.StringUtils;
|
import org.apache.commons.lang.StringUtils;
|
||||||
|
|
||||||
import java.util.Arrays;
|
|
||||||
import java.util.List;
|
import java.util.List;
|
||||||
import java.util.Map;
|
import java.util.Map;
|
||||||
import java.util.Set;
|
import java.util.Set;
|
||||||
@@ -12,20 +13,6 @@ import java.util.Set;
|
|||||||
* @date 20/4/16
|
* @date 20/4/16
|
||||||
*/
|
*/
|
||||||
public class ValidateUtils {
|
public class ValidateUtils {
|
||||||
/**
|
|
||||||
* 任意一个为空, 则返回true
|
|
||||||
*/
|
|
||||||
public static boolean anyNull(Object... objects) {
|
|
||||||
return Arrays.stream(objects).anyMatch(ValidateUtils::isNull);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 是空字符串或者空
|
|
||||||
*/
|
|
||||||
public static boolean anyBlank(String... strings) {
|
|
||||||
return Arrays.stream(strings).anyMatch(StringUtils::isBlank);
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 为空
|
* 为空
|
||||||
*/
|
*/
|
||||||
@@ -96,4 +83,23 @@ public class ValidateUtils {
|
|||||||
public static boolean isNullOrLessThanZero(Double value) {
|
public static boolean isNullOrLessThanZero(Double value) {
|
||||||
return value == null || value < 0;
|
return value == null || value < 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
public static boolean topicNameLegal(String idc, String topicName) {
|
||||||
|
if (ValidateUtils.isNull(idc) || ValidateUtils.isNull(topicName)) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
// 校验Topic的长度
|
||||||
|
if (topicName.length() >= TopicCreationConstant.TOPIC_NAME_MAX_LENGTH) {
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
|
||||||
|
// 校验前缀
|
||||||
|
if (IDCEnum.CN.getIdc().equals(idc) ||
|
||||||
|
(IDCEnum.US.getIdc().equals(idc) && topicName.startsWith(TopicCreationConstant.TOPIC_NAME_PREFIX_US)) ||
|
||||||
|
(IDCEnum.RU.getIdc().equals(idc) && topicName.startsWith(TopicCreationConstant.TOPIC_NAME_PREFIX_RU))) {
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
return false;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
@@ -1,65 +0,0 @@
|
|||||||
package com.xiaojukeji.kafka.manager.common.utils.jmx;
|
|
||||||
|
|
||||||
public class JmxConfig {
|
|
||||||
/**
|
|
||||||
* 单台最大连接数
|
|
||||||
*/
|
|
||||||
private Integer maxConn;
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 用户名
|
|
||||||
*/
|
|
||||||
private String username;
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 密码
|
|
||||||
*/
|
|
||||||
private String password;
|
|
||||||
|
|
||||||
/**
|
|
||||||
* 开启SSL
|
|
||||||
*/
|
|
||||||
private Boolean openSSL;
|
|
||||||
|
|
||||||
public Integer getMaxConn() {
|
|
||||||
return maxConn;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setMaxConn(Integer maxConn) {
|
|
||||||
this.maxConn = maxConn;
|
|
||||||
}
|
|
||||||
|
|
||||||
public String getUsername() {
|
|
||||||
return username;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setUsername(String username) {
|
|
||||||
this.username = username;
|
|
||||||
}
|
|
||||||
|
|
||||||
public String getPassword() {
|
|
||||||
return password;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setPassword(String password) {
|
|
||||||
this.password = password;
|
|
||||||
}
|
|
||||||
|
|
||||||
public Boolean isOpenSSL() {
|
|
||||||
return openSSL;
|
|
||||||
}
|
|
||||||
|
|
||||||
public void setOpenSSL(Boolean openSSL) {
|
|
||||||
this.openSSL = openSSL;
|
|
||||||
}
|
|
||||||
|
|
||||||
@Override
|
|
||||||
public String toString() {
|
|
||||||
return "JmxConfig{" +
|
|
||||||
"maxConn=" + maxConn +
|
|
||||||
", username='" + username + '\'' +
|
|
||||||
", password='" + password + '\'' +
|
|
||||||
", openSSL=" + openSSL +
|
|
||||||
'}';
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,6 +1,5 @@
|
|||||||
package com.xiaojukeji.kafka.manager.common.utils.jmx;
|
package com.xiaojukeji.kafka.manager.common.utils.jmx;
|
||||||
|
|
||||||
import com.xiaojukeji.kafka.manager.common.utils.ValidateUtils;
|
|
||||||
import org.slf4j.Logger;
|
import org.slf4j.Logger;
|
||||||
import org.slf4j.LoggerFactory;
|
import org.slf4j.LoggerFactory;
|
||||||
|
|
||||||
@@ -8,14 +7,8 @@ import javax.management.*;
|
|||||||
import javax.management.remote.JMXConnector;
|
import javax.management.remote.JMXConnector;
|
||||||
import javax.management.remote.JMXConnectorFactory;
|
import javax.management.remote.JMXConnectorFactory;
|
||||||
import javax.management.remote.JMXServiceURL;
|
import javax.management.remote.JMXServiceURL;
|
||||||
import javax.management.remote.rmi.RMIConnectorServer;
|
|
||||||
import javax.naming.Context;
|
|
||||||
import javax.rmi.ssl.SslRMIClientSocketFactory;
|
|
||||||
import java.io.IOException;
|
import java.io.IOException;
|
||||||
import java.net.MalformedURLException;
|
import java.net.MalformedURLException;
|
||||||
import java.util.Arrays;
|
|
||||||
import java.util.HashMap;
|
|
||||||
import java.util.Map;
|
|
||||||
import java.util.Set;
|
import java.util.Set;
|
||||||
import java.util.concurrent.atomic.AtomicInteger;
|
import java.util.concurrent.atomic.AtomicInteger;
|
||||||
|
|
||||||
@@ -35,19 +28,13 @@ public class JmxConnectorWrap {
|
|||||||
|
|
||||||
private AtomicInteger atomicInteger;
|
private AtomicInteger atomicInteger;
|
||||||
|
|
||||||
private JmxConfig jmxConfig;
|
public JmxConnectorWrap(String host, int port, int maxConn) {
|
||||||
|
|
||||||
public JmxConnectorWrap(String host, int port, JmxConfig jmxConfig) {
|
|
||||||
this.host = host;
|
this.host = host;
|
||||||
this.port = port;
|
this.port = port;
|
||||||
this.jmxConfig = jmxConfig;
|
if (maxConn <= 0) {
|
||||||
if (ValidateUtils.isNull(this.jmxConfig)) {
|
maxConn = 1;
|
||||||
this.jmxConfig = new JmxConfig();
|
|
||||||
}
|
}
|
||||||
if (ValidateUtils.isNullOrLessThanZero(this.jmxConfig.getMaxConn())) {
|
this.atomicInteger = new AtomicInteger(maxConn);
|
||||||
this.jmxConfig.setMaxConn(1);
|
|
||||||
}
|
|
||||||
this.atomicInteger = new AtomicInteger(this.jmxConfig.getMaxConn());
|
|
||||||
}
|
}
|
||||||
|
|
||||||
public boolean checkJmxConnectionAndInitIfNeed() {
|
public boolean checkJmxConnectionAndInitIfNeed() {
|
||||||
@@ -77,18 +64,8 @@ public class JmxConnectorWrap {
|
|||||||
}
|
}
|
||||||
String jmxUrl = String.format("service:jmx:rmi:///jndi/rmi://%s:%d/jmxrmi", host, port);
|
String jmxUrl = String.format("service:jmx:rmi:///jndi/rmi://%s:%d/jmxrmi", host, port);
|
||||||
try {
|
try {
|
||||||
Map<String, Object> environment = new HashMap<String, Object>();
|
JMXServiceURL url = new JMXServiceURL(jmxUrl);
|
||||||
if (!ValidateUtils.isBlank(this.jmxConfig.getUsername()) && !ValidateUtils.isBlank(this.jmxConfig.getPassword())) {
|
jmxConnector = JMXConnectorFactory.connect(url, null);
|
||||||
environment.put(JMXConnector.CREDENTIALS, Arrays.asList(this.jmxConfig.getUsername(), this.jmxConfig.getPassword()));
|
|
||||||
}
|
|
||||||
if (jmxConfig.isOpenSSL() != null && this.jmxConfig.isOpenSSL()) {
|
|
||||||
environment.put(Context.SECURITY_PROTOCOL, "ssl");
|
|
||||||
SslRMIClientSocketFactory clientSocketFactory = new SslRMIClientSocketFactory();
|
|
||||||
environment.put(RMIConnectorServer.RMI_CLIENT_SOCKET_FACTORY_ATTRIBUTE, clientSocketFactory);
|
|
||||||
environment.put("com.sun.jndi.rmi.factory.socket", clientSocketFactory);
|
|
||||||
}
|
|
||||||
|
|
||||||
jmxConnector = JMXConnectorFactory.connect(new JMXServiceURL(jmxUrl), environment);
|
|
||||||
LOGGER.info("JMX connect success, host:{} port:{}.", host, port);
|
LOGGER.info("JMX connect success, host:{} port:{}.", host, port);
|
||||||
return true;
|
return true;
|
||||||
} catch (MalformedURLException e) {
|
} catch (MalformedURLException e) {
|
||||||
|
|||||||
@@ -170,10 +170,7 @@ public class MbeanNameUtilV2 {
|
|||||||
new MbeanV2(
|
new MbeanV2(
|
||||||
"TopicCodeC",
|
"TopicCodeC",
|
||||||
JmxAttributeEnum.VALUE_ATTRIBUTE,
|
JmxAttributeEnum.VALUE_ATTRIBUTE,
|
||||||
Arrays.asList(
|
"kafka.server:type=ReplicaManager,name=TopicCodeC"
|
||||||
new AbstractMap.SimpleEntry<>(KafkaVersion.VERSION_0_10_3, "kafka.server:type=ReplicaManager,name=TopicCodeC"),
|
|
||||||
new AbstractMap.SimpleEntry<>(KafkaVersion.VERSION_MAX, "kafka.server:type=AppIdTopicMetrics,name=RecordCompression,appId=")
|
|
||||||
)
|
|
||||||
),
|
),
|
||||||
Arrays.asList(
|
Arrays.asList(
|
||||||
KafkaMetricsCollections.TOPIC_BASIC_PAGE_METRICS
|
KafkaMetricsCollections.TOPIC_BASIC_PAGE_METRICS
|
||||||
|
|||||||
@@ -8,7 +8,7 @@ package com.xiaojukeji.kafka.manager.common.zookeeper;
|
|||||||
public class ZkPathUtil {
|
public class ZkPathUtil {
|
||||||
private static final String ZOOKEEPER_SEPARATOR = "/";
|
private static final String ZOOKEEPER_SEPARATOR = "/";
|
||||||
|
|
||||||
public static final String BROKER_ROOT_NODE = ZOOKEEPER_SEPARATOR + "brokers";
|
private static final String BROKER_ROOT_NODE = ZOOKEEPER_SEPARATOR + "brokers";
|
||||||
|
|
||||||
public static final String CONTROLLER_ROOT_NODE = ZOOKEEPER_SEPARATOR + "controller";
|
public static final String CONTROLLER_ROOT_NODE = ZOOKEEPER_SEPARATOR + "controller";
|
||||||
|
|
||||||
@@ -18,8 +18,6 @@ public class ZkPathUtil {
|
|||||||
|
|
||||||
public static final String CONSUMER_ROOT_NODE = ZOOKEEPER_SEPARATOR + "consumers";
|
public static final String CONSUMER_ROOT_NODE = ZOOKEEPER_SEPARATOR + "consumers";
|
||||||
|
|
||||||
public static final String REASSIGN_PARTITIONS_ROOT_NODE = "/admin/reassign_partitions";
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* config
|
* config
|
||||||
*/
|
*/
|
||||||
@@ -29,13 +27,11 @@ public class ZkPathUtil {
|
|||||||
|
|
||||||
public static final String CONFIG_CLIENTS_ROOT_NODE = CONFIG_ROOT_NODE + ZOOKEEPER_SEPARATOR + "clients";
|
public static final String CONFIG_CLIENTS_ROOT_NODE = CONFIG_ROOT_NODE + ZOOKEEPER_SEPARATOR + "clients";
|
||||||
|
|
||||||
public static final String CONFIG_ENTITY_CHANGES_ROOT_NODE = CONFIG_ROOT_NODE + ZOOKEEPER_SEPARATOR + "changes/config_change_";
|
public static final String CONFIG_ENTITY_CHANGES_ROOT_NODE = CONFIG_ROOT_NODE + ZOOKEEPER_SEPARATOR + "changes/config_change_";
|
||||||
|
|
||||||
private static final String D_METRICS_CONFIG_ROOT_NODE = CONFIG_ROOT_NODE + ZOOKEEPER_SEPARATOR + "KafkaExMetrics";
|
public static final String REASSIGN_PARTITIONS_ROOT_NODE = "/admin/reassign_partitions";
|
||||||
|
|
||||||
public static final String D_CONFIG_EXTENSION_ROOT_NODE = CONFIG_ROOT_NODE + ZOOKEEPER_SEPARATOR + "extension";
|
private static final String D_METRICS_CONFIG_ROOT_NODE = CONFIG_ROOT_NODE + ZOOKEEPER_SEPARATOR + "KafkaExMetrics";
|
||||||
|
|
||||||
public static final String D_CONTROLLER_CANDIDATES = D_CONFIG_EXTENSION_ROOT_NODE + ZOOKEEPER_SEPARATOR + "candidates";
|
|
||||||
|
|
||||||
public static String getBrokerIdNodePath(Integer brokerId) {
|
public static String getBrokerIdNodePath(Integer brokerId) {
|
||||||
return BROKER_IDS_ROOT + ZOOKEEPER_SEPARATOR + String.valueOf(brokerId);
|
return BROKER_IDS_ROOT + ZOOKEEPER_SEPARATOR + String.valueOf(brokerId);
|
||||||
@@ -113,10 +109,6 @@ public class ZkPathUtil {
|
|||||||
}
|
}
|
||||||
|
|
||||||
public static String getKafkaExtraMetricsPath(Integer brokerId) {
|
public static String getKafkaExtraMetricsPath(Integer brokerId) {
|
||||||
return D_METRICS_CONFIG_ROOT_NODE + ZOOKEEPER_SEPARATOR + brokerId;
|
return D_METRICS_CONFIG_ROOT_NODE + ZOOKEEPER_SEPARATOR + String.valueOf(brokerId);
|
||||||
}
|
|
||||||
|
|
||||||
public static String getControllerCandidatePath(Integer brokerId) {
|
|
||||||
return D_CONTROLLER_CANDIDATES + ZOOKEEPER_SEPARATOR + brokerId;
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user