diff --git a/README.md b/README.md index c5830ffe..70585679 100644 --- a/README.md +++ b/README.md @@ -51,16 +51,16 @@ - 无需侵入改造 `Apache Kafka` ，一键便能纳管 `0.10.x` ~ `3.x.x` 众多版本的Kafka，包括 `ZK` 或 `Raft` 运行模式的版本，同时在兼容架构上具备良好的扩展性，帮助您提升集群管理水平； - 🌪️  **零成本、界面化** - - 提炼高频 CLI 能力，设计合理的产品路径，提供清新美观的 GUI 界面，支持 Cluster、Broker、Topic、Group、Message、ACL 等组件 GUI 管理，普通用户5分钟即可上手； + - 提炼高频 CLI 能力，设计合理的产品路径，提供清新美观的 GUI 界面，支持 Cluster、Broker、Zookeeper、Topic、ConsumerGroup、Message、ACL、Connect 等组件 GUI 管理，普通用户5分钟即可上手； - 👏  **云原生、插件化** - 基于云原生构建，具备水平扩展能力，只需要增加节点即可获取更强的采集及对外服务能力，提供众多可热插拔的企业级特性，覆盖可观测性生态整合、资源治理、多活容灾等核心场景； - 🚀  **专业能力** - - 集群管理：支持集群一键纳管，健康分析、核心组件观测 等功能； + - 集群管理：支持一键纳管，健康分析、核心组件观测 等功能； - 观测提升：多维度指标观测大盘、观测指标最佳实践 等功能； - 异常巡检：集群多维度健康巡检、集群多维度健康分 等功能； - - 能力增强：Topic扩缩副本、Topic副本迁移 等功能； + - 能力增强：集群负载均衡、Topic扩缩副本、Topic副本迁移 等功能；   diff --git a/Releases_Notes.md b/Releases_Notes.md index 616d1eab..5dda1685 100644 --- a/Releases_Notes.md +++ b/Releases_Notes.md @@ -1,5 +1,39 @@ +## v3.0.0-beta.3 + +**文档** +- FAQ 补充权限识别失败问题的说明 +- 同步更新文档，保持与官网一致 + + +**Bug修复** +- Offset 信息获取时，过滤掉无 Leader 的分区 +- 升级 oshi-core 版本至 5.6.1 版本，修复 Windows 系统获取系统指标失败问题 +- 修复 JMX 连接被关闭后，未进行重建的问题 +- 修复因 DB 中 Broker 信息不存在导致 TotalLogSize 指标获取时抛空指针问题 +- 修复 dml-logi.sql 中，SQL 注释错误的问题 +- 修复 startup.sh 中，识别操作系统类型错误的问题 +- 修复配置管理页面删除配置失败的问题 +- 修复系统管理应用文件引用路径 +- 修复 Topic Messages 详情提示信息点击跳转 404 的问题 +- 修复扩副本时，当前副本数不显示问题 + + +**体验优化** +- Topic-Messages 页面，增加返回数据的排序以及按照Earliest/Latest的获取方式 +- 优化 GroupOffsetResetEnum 类名为 OffsetTypeEnum，使得类名含义更准确 +- 移动 KafkaZKDAO 类，及 Kafka Znode 实体类的位置，使得 Kafka Zookeeper DAO 更加内聚及便于识别 +- 后端补充 Overview 页面指标排序的功能 +- 前端 Webpack 配置优化 +- Cluster Overview 图表取消放大展示功能 +- 列表页增加手动刷新功能 +- 接入/编辑集群，优化 JMX-PORT，Version 信息的回显，优化JMX信息的展示 +- 提高登录页面图片展示清晰度 +- 部分样式和文案优化 + +--- + ## v3.0.0-beta.2 **文档** diff --git a/bin/startup.sh b/bin/startup.sh index 8081f2dd..cbde7c56 100644 --- a/bin/startup.sh +++ b/bin/startup.sh @@ -9,7 +9,7 @@ error_exit () [ ! -e "$JAVA_HOME/bin/java" ] && unset JAVA_HOME if [ -z "$JAVA_HOME" ]; then - if $darwin; then + if [ "Darwin" = "$(uname -s)" ]; then if [ -x '/usr/libexec/java_home' ] ; then export JAVA_HOME=`/usr/libexec/java_home` diff --git a/docs/install_guide/单机部署手册.md b/docs/install_guide/单机部署手册.md index f9f5ad1a..b429b9d5 100644 --- a/docs/install_guide/单机部署手册.md +++ b/docs/install_guide/单机部署手册.md @@ -59,6 +59,8 @@ sh deploy_KnowStreaming-offline.sh ### 2.1.3、容器部署 +#### 2.1.3.1、Helm + **环境依赖** - Kubernetes >= 1.14 ，Helm >= 2.17.0 @@ -87,6 +89,103 @@ helm pull knowstreaming/knowstreaming-manager   +#### 2.1.3.2、Docker Compose +```yml +version: "3" + +services: + + knowstreaming-manager: + image: knowstreaming/knowstreaming-manager:0.2.0-test + container_name: knowstreaming-manager + privileged: true + restart: always + depends_on: + - elasticsearch-single + - knowstreaming-mysql + expose: + - 80 + command: + - /bin/sh + - /ks-start.sh + environment: + TZ: Asia/Shanghai + + SERVER_MYSQL_ADDRESS: knowstreaming-mysql:3306 + SERVER_MYSQL_DB: know_streaming + SERVER_MYSQL_USER: root + SERVER_MYSQL_PASSWORD: admin2022_ + + SERVER_ES_ADDRESS: elasticsearch-single:9200 + + JAVA_OPTS: -Xmx1g -Xms1g + +# extra_hosts: +# - "hostname:x.x.x.x" +# volumes: +# - /ks/manage/log:/logs + knowstreaming-ui: + image: knowstreaming/knowstreaming-ui:0.2.0-test1 + container_name: knowstreaming-ui + restart: always + ports: + - '18092:80' + environment: + TZ: Asia/Shanghai + depends_on: + - knowstreaming-manager +# extra_hosts: +# - "hostname:x.x.x.x" + + elasticsearch-single: + image: docker.io/library/elasticsearch:7.6.2 + container_name: elasticsearch-single + restart: always + expose: + - 9200 + - 9300 +# ports: +# - '9200:9200' +# - '9300:9300' + environment: + TZ: Asia/Shanghai + ES_JAVA_OPTS: -Xms512m -Xmx512m + discovery.type: single-node +# volumes: +# - /ks/es/data:/usr/share/elasticsearch/data + + knowstreaming-init: + image: knowstreaming/knowstreaming-manager:0.2.0-test + container_name: knowstreaming_init + depends_on: + - elasticsearch-single + command: + - /bin/bash + - /es_template_create.sh + environment: + TZ: Asia/Shanghai + SERVER_ES_ADDRESS: elasticsearch-single:9200 + + + knowstreaming-mysql: + image: knowstreaming/knowstreaming-mysql:0.2.0-test + container_name: knowstreaming-mysql + restart: always + environment: + TZ: Asia/Shanghai + MYSQL_ROOT_PASSWORD: admin2022_ + MYSQL_DATABASE: know_streaming + MYSQL_ROOT_HOST: '%' + expose: + - 3306 +# ports: +# - '3306:3306' +# volumes: +# - /ks/mysql/data:/data/mysql +``` + +  + ### 2.1.4、手动部署 **部署流程** diff --git a/docs/install_guide/版本升级手册.md b/docs/install_guide/版本升级手册.md index 48417bb4..2af3f69a 100644 --- a/docs/install_guide/版本升级手册.md +++ b/docs/install_guide/版本升级手册.md @@ -40,8 +40,7 @@ thread-pool: ``` - -**SQL变更** +**SQL 变更** ```sql -- 多集群管理权限2022-09-06新增 @@ -80,12 +79,11 @@ ALTER TABLE `logi_security_oplog` ### 6.2.2、升级至 `v3.0.0-beta.1`版本 - -**SQL变更** +**SQL 变更** 1、在`ks_km_broker`表增加了一个监听信息字段。 -2、为`logi_security_oplog`表operation_methods字段设置默认值''。 -因此需要执行下面的sql对数据库表进行更新。 +2、为`logi_security_oplog`表 operation_methods 字段设置默认值''。 +因此需要执行下面的 sql 对数据库表进行更新。 ```sql ALTER TABLE `ks_km_broker` @@ -98,7 +96,6 @@ ALTER COLUMN `operation_methods` set default ''; --- - ### 6.2.3、`2.x`版本 升级至 `v3.0.0-beta.0`版本 **升级步骤：** @@ -123,14 +120,14 @@ ALTER COLUMN `operation_methods` set default ''; UPDATE ks_km_topic INNER JOIN (SELECT - topic.cluster_id AS cluster_id, - topic.topic_name AS topic_name, - topic.description AS description + topic.cluster_id AS cluster_id, + topic.topic_name AS topic_name, + topic.description AS description FROM topic WHERE description != '' ) AS t - ON ks_km_topic.cluster_phy_id = t.cluster_id - AND ks_km_topic.topic_name = t.topic_name - AND ks_km_topic.id > 0 -SET ks_km_topic.description = t.description; +ON ks_km_topic.cluster_phy_id = t.cluster_id + AND ks_km_topic.topic_name = t.topic_name + AND ks_km_topic.id > 0 + SET ks_km_topic.description = t.description; ``` \ No newline at end of file diff --git a/docs/user_guide/faq.md b/docs/user_guide/faq.md index 764c58b9..98dfbf83 100644 --- a/docs/user_guide/faq.md +++ b/docs/user_guide/faq.md @@ -166,3 +166,19 @@ Node 版本: v12.22.12 需要到具体的应用中执行 `npm run start`，例如 `cd packages/layout-clusters-fe` 后，执行 `npm run start`。 应用启动后需要到基座应用中查看（需要启动基座应用，即 layout-clusters-fe）。 + + +## 8.12、权限识别失败问题 +1、使用admin账号登陆KnowStreaming时，点击系统管理-用户管理-角色管理-新增角色，查看页面是否正常。 + + + +2、查看'/logi-security/api/v1/permission/tree'接口返回值，出现如下图所示乱码现象。 +![接口返回值](http://img-ys011.didistatic.com/static/dc2img/do1_jTxBkwNGU9vZuYQQbdNw) + +3、查看logi_security_permission表，看看是否出现了中文乱码现象。 + +根据以上几点，我们可以确定是由于数据库乱码造成的权限识别失败问题。 + ++ 原因：由于数据库编码和我们提供的脚本不一致，数据库里的数据发生了乱码，因此出现权限识别失败问题。 ++ 解决方案：清空数据库数据，将数据库字符集调整为utf8，最后重新执行[dml-logi.sql](https://github.com/didi/KnowStreaming/blob/master/km-dist/init/sql/dml-logi.sql)脚本导入数据即可。 diff --git a/docs/user_guide/用户使用手册.md b/docs/user_guide/用户使用手册.md index f2efc936..6179cb3b 100644 --- a/docs/user_guide/用户使用手册.md +++ b/docs/user_guide/用户使用手册.md @@ -11,7 +11,7 @@ 下面是用户第一次使用我们产品的典型体验路径： -![text](http://img-ys011.didistatic.com/static/dc2img/do1_YehqxqmsVaqU5gf3XphI) +![text](http://img-ys011.didistatic.com/static/dc2img/do1_qgqPsAY46sZeBaPUCwXY) ## 5.3、常用功能 diff --git a/km-biz/src/main/java/com/xiaojukeji/know/streaming/km/biz/group/impl/GroupManagerImpl.java b/km-biz/src/main/java/com/xiaojukeji/know/streaming/km/biz/group/impl/GroupManagerImpl.java index 15aafdb5..1095d5ee 100644 --- a/km-biz/src/main/java/com/xiaojukeji/know/streaming/km/biz/group/impl/GroupManagerImpl.java +++ b/km-biz/src/main/java/com/xiaojukeji/know/streaming/km/biz/group/impl/GroupManagerImpl.java @@ -19,7 +19,7 @@ import com.xiaojukeji.know.streaming.km.common.bean.vo.group.GroupTopicConsumedD import com.xiaojukeji.know.streaming.km.common.bean.vo.group.GroupTopicOverviewVO; import com.xiaojukeji.know.streaming.km.common.constant.MsgConstant; import com.xiaojukeji.know.streaming.km.common.enums.AggTypeEnum; -import com.xiaojukeji.know.streaming.km.common.enums.GroupOffsetResetEnum; +import com.xiaojukeji.know.streaming.km.common.enums.OffsetTypeEnum; import com.xiaojukeji.know.streaming.km.common.enums.group.GroupStateEnum; import com.xiaojukeji.know.streaming.km.common.exception.AdminOperateException; import com.xiaojukeji.know.streaming.km.common.exception.NotExistException; @@ -199,12 +199,12 @@ public class GroupManagerImpl implements GroupManager { return Result.buildFromRSAndMsg(ResultStatus.NOT_EXIST, MsgConstant.getTopicNotExist(dto.getClusterId(), dto.getTopicName())); } - if (GroupOffsetResetEnum.PRECISE_OFFSET.getResetType() == dto.getResetType() + if (OffsetTypeEnum.PRECISE_OFFSET.getResetType() == dto.getResetType() && ValidateUtils.isEmptyList(dto.getOffsetList())) { return Result.buildFromRSAndMsg(ResultStatus.PARAM_ILLEGAL, "参数错误，指定offset重置需传offset信息"); } - if (GroupOffsetResetEnum.PRECISE_TIMESTAMP.getResetType() == dto.getResetType() + if (OffsetTypeEnum.PRECISE_TIMESTAMP.getResetType() == dto.getResetType() && ValidateUtils.isNull(dto.getTimestamp())) { return Result.buildFromRSAndMsg(ResultStatus.PARAM_ILLEGAL, "参数错误，指定时间重置需传时间信息"); } @@ -213,7 +213,7 @@ public class GroupManagerImpl implements GroupManager { } private Result> getPartitionOffset(GroupOffsetResetDTO dto) { - if (GroupOffsetResetEnum.PRECISE_OFFSET.getResetType() == dto.getResetType()) { + if (OffsetTypeEnum.PRECISE_OFFSET.getResetType() == dto.getResetType()) { return Result.buildSuc(dto.getOffsetList().stream().collect(Collectors.toMap( elem -> new TopicPartition(dto.getTopicName(), elem.getPartitionId()), PartitionOffsetDTO::getOffset, @@ -222,9 +222,9 @@ public class GroupManagerImpl implements GroupManager { } OffsetSpec offsetSpec = null; - if (GroupOffsetResetEnum.PRECISE_TIMESTAMP.getResetType() == dto.getResetType()) { + if (OffsetTypeEnum.PRECISE_TIMESTAMP.getResetType() == dto.getResetType()) { offsetSpec = OffsetSpec.forTimestamp(dto.getTimestamp()); - } else if (GroupOffsetResetEnum.EARLIEST.getResetType() == dto.getResetType()) { + } else if (OffsetTypeEnum.EARLIEST.getResetType() == dto.getResetType()) { offsetSpec = OffsetSpec.earliest(); } else { offsetSpec = OffsetSpec.latest(); diff --git a/km-biz/src/main/java/com/xiaojukeji/know/streaming/km/biz/topic/TopicStateManager.java b/km-biz/src/main/java/com/xiaojukeji/know/streaming/km/biz/topic/TopicStateManager.java index 0e8436d9..ec3a3207 100644 --- a/km-biz/src/main/java/com/xiaojukeji/know/streaming/km/biz/topic/TopicStateManager.java +++ b/km-biz/src/main/java/com/xiaojukeji/know/streaming/km/biz/topic/TopicStateManager.java @@ -1,5 +1,6 @@ package com.xiaojukeji.know.streaming.km.biz.topic; +import com.xiaojukeji.know.streaming.km.common.bean.dto.pagination.PaginationSortDTO; import com.xiaojukeji.know.streaming.km.common.bean.dto.topic.TopicRecordDTO; import com.xiaojukeji.know.streaming.km.common.bean.entity.result.Result; import com.xiaojukeji.know.streaming.km.common.bean.vo.topic.TopicBrokersPartitionsSummaryVO; diff --git a/km-biz/src/main/java/com/xiaojukeji/know/streaming/km/biz/topic/impl/TopicStateManagerImpl.java b/km-biz/src/main/java/com/xiaojukeji/know/streaming/km/biz/topic/impl/TopicStateManagerImpl.java index a0418bb2..9c03737a 100644 --- a/km-biz/src/main/java/com/xiaojukeji/know/streaming/km/biz/topic/impl/TopicStateManagerImpl.java +++ b/km-biz/src/main/java/com/xiaojukeji/know/streaming/km/biz/topic/impl/TopicStateManagerImpl.java @@ -22,25 +22,26 @@ import com.xiaojukeji.know.streaming.km.common.bean.vo.topic.partition.TopicPart import com.xiaojukeji.know.streaming.km.common.constant.Constant; import com.xiaojukeji.know.streaming.km.common.constant.KafkaConstant; import com.xiaojukeji.know.streaming.km.common.constant.MsgConstant; -import com.xiaojukeji.know.streaming.km.common.converter.PartitionConverter; import com.xiaojukeji.know.streaming.km.common.converter.TopicVOConverter; +import com.xiaojukeji.know.streaming.km.common.enums.OffsetTypeEnum; +import com.xiaojukeji.know.streaming.km.common.enums.SortTypeEnum; import com.xiaojukeji.know.streaming.km.common.exception.AdminOperateException; import com.xiaojukeji.know.streaming.km.common.exception.NotExistException; import com.xiaojukeji.know.streaming.km.common.utils.ConvertUtil; +import com.xiaojukeji.know.streaming.km.common.utils.PaginationUtil; import com.xiaojukeji.know.streaming.km.common.utils.ValidateUtils; import com.xiaojukeji.know.streaming.km.core.service.broker.BrokerService; import com.xiaojukeji.know.streaming.km.core.service.cluster.ClusterPhyService; import com.xiaojukeji.know.streaming.km.core.service.partition.PartitionMetricService; -import com.xiaojukeji.know.streaming.km.core.service.topic.TopicConfigService; import com.xiaojukeji.know.streaming.km.core.service.partition.PartitionService; +import com.xiaojukeji.know.streaming.km.core.service.topic.TopicConfigService; import com.xiaojukeji.know.streaming.km.core.service.topic.TopicMetricService; import com.xiaojukeji.know.streaming.km.core.service.topic.TopicService; import com.xiaojukeji.know.streaming.km.core.service.version.metrics.TopicMetricVersionItems; +import org.apache.commons.lang3.ObjectUtils; +import org.apache.commons.lang3.StringUtils; import org.apache.kafka.clients.admin.OffsetSpec; -import org.apache.kafka.clients.consumer.ConsumerConfig; -import org.apache.kafka.clients.consumer.ConsumerRecord; -import org.apache.kafka.clients.consumer.ConsumerRecords; -import org.apache.kafka.clients.consumer.KafkaConsumer; +import org.apache.kafka.clients.consumer.*; import org.apache.kafka.common.TopicPartition; import org.apache.kafka.common.config.TopicConfig; import org.springframework.beans.factory.annotation.Autowired; @@ -160,8 +161,31 @@ public class TopicStateManagerImpl implements TopicStateManager { } maxMessage = Math.min(maxMessage, dto.getMaxRecords()); kafkaConsumer.assign(partitionList); + + Map partitionOffsetAndTimestampMap = new HashMap<>(); + // 获取指定时间每个分区的offset（按指定开始时间查询消息时） + if (OffsetTypeEnum.PRECISE_TIMESTAMP.getResetType() == dto.getFilterOffsetReset()) { + Map timestampsToSearch = new HashMap<>(); + partitionList.forEach(topicPartition -> { + timestampsToSearch.put(topicPartition, dto.getStartTimestampUnitMs()); + }); + partitionOffsetAndTimestampMap = kafkaConsumer.offsetsForTimes(timestampsToSearch); + } + for (TopicPartition partition : partitionList) { - kafkaConsumer.seek(partition, Math.max(beginOffsetsMapResult.getData().get(partition), endOffsetsMapResult.getData().get(partition) - dto.getMaxRecords())); + if (OffsetTypeEnum.EARLIEST.getResetType() == dto.getFilterOffsetReset()) { + // 重置到最旧 + kafkaConsumer.seek(partition, beginOffsetsMapResult.getData().get(partition)); + } else if (OffsetTypeEnum.PRECISE_TIMESTAMP.getResetType() == dto.getFilterOffsetReset()) { + // 重置到指定时间 + kafkaConsumer.seek(partition, partitionOffsetAndTimestampMap.get(partition).offset()); + } else if (OffsetTypeEnum.PRECISE_OFFSET.getResetType() == dto.getFilterOffsetReset()) { + // 重置到指定位置 + + } else { + // 默认，重置到最新 + kafkaConsumer.seek(partition, Math.max(beginOffsetsMapResult.getData().get(partition), endOffsetsMapResult.getData().get(partition) - dto.getMaxRecords())); + } } // 这里需要减去 KafkaConstant.POLL_ONCE_TIMEOUT_UNIT_MS 是因为poll一次需要耗时，如果这里不减去，则可能会导致poll之后，超过要求的时间 @@ -185,6 +209,15 @@ public class TopicStateManagerImpl implements TopicStateManager { } } + // 排序 + if (ObjectUtils.isNotEmpty(voList)) { + // 默认按时间倒序排序 + if (StringUtils.isBlank(dto.getSortType())) { + dto.setSortType(SortTypeEnum.DESC.getSortType()); + } + PaginationUtil.pageBySort(voList, dto.getSortField(), dto.getSortType()); + } + return Result.buildSuc(voList.subList(0, Math.min(dto.getMaxRecords(), voList.size()))); } catch (Exception e) { log.error("method=getTopicMessages||clusterPhyId={}||topicName={}||param={}||errMsg=exception", clusterPhyId, topicName, dto, e); diff --git a/km-biz/src/main/java/com/xiaojukeji/know/streaming/km/biz/version/impl/VersionControlManagerImpl.java b/km-biz/src/main/java/com/xiaojukeji/know/streaming/km/biz/version/impl/VersionControlManagerImpl.java index 994b2c8a..52a91520 100644 --- a/km-biz/src/main/java/com/xiaojukeji/know/streaming/km/biz/version/impl/VersionControlManagerImpl.java +++ b/km-biz/src/main/java/com/xiaojukeji/know/streaming/km/biz/version/impl/VersionControlManagerImpl.java @@ -7,12 +7,14 @@ import com.didiglobal.logi.log.LogFactory; import com.didiglobal.logi.security.common.dto.config.ConfigDTO; import com.didiglobal.logi.security.service.ConfigService; import com.xiaojukeji.know.streaming.km.biz.version.VersionControlManager; +import com.xiaojukeji.know.streaming.km.common.bean.dto.metrices.MetricDetailDTO; import com.xiaojukeji.know.streaming.km.common.bean.dto.metrices.UserMetricConfigDTO; import com.xiaojukeji.know.streaming.km.common.bean.entity.config.metric.UserMetricConfig; import com.xiaojukeji.know.streaming.km.common.bean.entity.result.Result; import com.xiaojukeji.know.streaming.km.common.bean.entity.version.VersionControlItem; import com.xiaojukeji.know.streaming.km.common.bean.vo.config.metric.UserMetricConfigVO; import com.xiaojukeji.know.streaming.km.common.bean.vo.version.VersionItemVO; +import com.xiaojukeji.know.streaming.km.common.constant.Constant; import com.xiaojukeji.know.streaming.km.common.enums.version.VersionEnum; import com.xiaojukeji.know.streaming.km.common.utils.ConvertUtil; import com.xiaojukeji.know.streaming.km.common.utils.VersionUtil; @@ -47,29 +49,29 @@ public class VersionControlManagerImpl implements VersionControlManager { @PostConstruct public void init(){ defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_HEALTH_SCORE, true)); - defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_TOTAL_PRODUCE_REQUESTS, true)); defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_FAILED_FETCH_REQ, true)); defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_FAILED_PRODUCE_REQ, true)); - defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_MESSAGE_IN, true)); defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_UNDER_REPLICA_PARTITIONS, true)); + defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_TOTAL_PRODUCE_REQUESTS, true)); defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_BYTES_IN, true)); defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_BYTES_OUT, true)); defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_BYTES_REJECTED, true)); + defaultMetrics.add(new UserMetricConfig(METRIC_TOPIC.getCode(), TOPIC_METRIC_MESSAGE_IN, true)); defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_HEALTH_SCORE, true)); - defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_TOTAL_REQ_QUEUE_SIZE, true)); - defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_TOTAL_RES_QUEUE_SIZE, true)); defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_ACTIVE_CONTROLLER_COUNT, true)); - defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_TOTAL_PRODUCE_REQ, true)); - defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_TOTAL_LOG_SIZE, true)); - defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_CONNECTIONS, true)); - defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_MESSAGES_IN, true)); defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_BYTES_IN, true)); defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_BYTES_OUT, true)); - defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_GROUP_REBALANCES, true)); - defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_JOB_RUNNING, true)); + defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_CONNECTIONS, true)); + defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_MESSAGES_IN, true)); defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_PARTITIONS_NO_LEADER, true)); defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_PARTITION_URP, true)); + defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_TOTAL_LOG_SIZE, true)); + defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_TOTAL_PRODUCE_REQ, true)); + defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_TOTAL_REQ_QUEUE_SIZE, true)); + defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_TOTAL_RES_QUEUE_SIZE, true)); + defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_GROUP_REBALANCES, true)); + defaultMetrics.add(new UserMetricConfig(METRIC_CLUSTER.getCode(), CLUSTER_METRIC_JOB_RUNNING, true)); defaultMetrics.add(new UserMetricConfig(METRIC_GROUP.getCode(), GROUP_METRIC_OFFSET_CONSUMED, true)); defaultMetrics.add(new UserMetricConfig(METRIC_GROUP.getCode(), GROUP_METRIC_LAG, true)); @@ -77,18 +79,18 @@ public class VersionControlManagerImpl implements VersionControlManager { defaultMetrics.add(new UserMetricConfig(METRIC_GROUP.getCode(), GROUP_METRIC_HEALTH_SCORE, true)); defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_HEALTH_SCORE, true)); - defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_TOTAL_REQ_QUEUE, true)); - defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_TOTAL_RES_QUEUE, true)); + defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_CONNECTION_COUNT, true)); defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_MESSAGE_IN, true)); - defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_TOTAL_PRODUCE_REQ, true)); defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_NETWORK_RPO_AVG_IDLE, true)); defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_REQ_AVG_IDLE, true)); - defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_CONNECTION_COUNT, true)); - defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_BYTES_IN, true)); - defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_BYTES_OUT, true)); - defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_PARTITIONS_SKEW, true)); + defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_TOTAL_PRODUCE_REQ, true)); + defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_TOTAL_REQ_QUEUE, true)); + defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_TOTAL_RES_QUEUE, true)); defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_LEADERS_SKEW, true)); defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_UNDER_REPLICATE_PARTITION, true)); + defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_PARTITIONS_SKEW, true)); + defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_BYTES_IN, true)); + defaultMetrics.add(new UserMetricConfig(METRIC_BROKER.getCode(), BROKER_METRIC_BYTES_OUT, true)); } @Autowired @@ -159,6 +161,9 @@ public class VersionControlManagerImpl implements VersionControlManager { UserMetricConfig umc = userMetricConfigMap.get(itemType + "@" + metric); userMetricConfigVO.setSet(null != umc && umc.isSet()); + if (umc != null) { + userMetricConfigVO.setRank(umc.getRank()); + } userMetricConfigVO.setName(itemVO.getName()); userMetricConfigVO.setType(itemVO.getType()); userMetricConfigVO.setDesc(itemVO.getDesc()); @@ -178,13 +183,29 @@ public class VersionControlManagerImpl implements VersionControlManager { @Override public Result updateUserMetricItem(Long clusterId, Integer type, UserMetricConfigDTO dto, String operator) { Map metricsSetMap = dto.getMetricsSet(); - if(null == metricsSetMap || metricsSetMap.isEmpty()){ + + //转换metricDetailDTOList + List metricDetailDTOList = dto.getMetricDetailDTOList(); + Map metricDetailMap = new HashMap<>(); + if (metricDetailDTOList != null && !metricDetailDTOList.isEmpty()) { + metricDetailMap = metricDetailDTOList.stream().collect(Collectors.toMap(MetricDetailDTO::getMetric, Function.identity())); + } + + //转换metricsSetMap + if (metricsSetMap != null && !metricsSetMap.isEmpty()) { + for (Map.Entry metricAndShowEntry : metricsSetMap.entrySet()) { + if (metricDetailMap.containsKey(metricAndShowEntry.getKey())) continue; + metricDetailMap.put(metricAndShowEntry.getKey(), new MetricDetailDTO(metricAndShowEntry.getKey(), metricAndShowEntry.getValue(), null)); + } + } + + if (metricDetailMap.isEmpty()) { return Result.buildSuc(); } Set userMetricConfigs = getUserMetricConfig(operator); - for(Map.Entry metricAndShowEntry : metricsSetMap.entrySet()){ - UserMetricConfig userMetricConfig = new UserMetricConfig(type, metricAndShowEntry.getKey(), metricAndShowEntry.getValue()); + for (MetricDetailDTO metricDetailDTO : metricDetailMap.values()) { + UserMetricConfig userMetricConfig = new UserMetricConfig(type, metricDetailDTO.getMetric(), metricDetailDTO.getSet(), metricDetailDTO.getRank()); userMetricConfigs.remove(userMetricConfig); userMetricConfigs.add(userMetricConfig); } @@ -228,7 +249,7 @@ public class VersionControlManagerImpl implements VersionControlManager { return defaultMetrics; } - return JSON.parseObject(value, new TypeReference>(){}); + return JSON.parseObject(value, new TypeReference>() {}); } public static void main(String[] args){ diff --git a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/dto/group/GroupOffsetResetDTO.java b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/dto/group/GroupOffsetResetDTO.java index 09d12bcd..378709dd 100644 --- a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/dto/group/GroupOffsetResetDTO.java +++ b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/dto/group/GroupOffsetResetDTO.java @@ -3,6 +3,7 @@ package com.xiaojukeji.know.streaming.km.common.bean.dto.group; import com.fasterxml.jackson.annotation.JsonIgnoreProperties; import com.xiaojukeji.know.streaming.km.common.bean.dto.partition.PartitionOffsetDTO; import com.xiaojukeji.know.streaming.km.common.bean.dto.topic.ClusterTopicDTO; +import com.xiaojukeji.know.streaming.km.common.enums.OffsetTypeEnum; import io.swagger.annotations.ApiModelProperty; import lombok.Data; @@ -23,7 +24,7 @@ public class GroupOffsetResetDTO extends ClusterTopicDTO { private String groupName; /** - * @see com.xiaojukeji.know.streaming.km.common.enums.GroupOffsetResetEnum + * @see OffsetTypeEnum */ @NotNull(message = "resetType不允许为空") @ApiModelProperty(value = "重置方式", example = "1") diff --git a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/dto/metrices/MetricDetailDTO.java b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/dto/metrices/MetricDetailDTO.java new file mode 100644 index 00000000..0d51d2c6 --- /dev/null +++ b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/dto/metrices/MetricDetailDTO.java @@ -0,0 +1,28 @@ +package com.xiaojukeji.know.streaming.km.common.bean.dto.metrices; + +import com.xiaojukeji.know.streaming.km.common.bean.dto.BaseDTO; +import io.swagger.annotations.ApiModel; +import io.swagger.annotations.ApiModelProperty; +import lombok.AllArgsConstructor; +import lombok.Data; +import lombok.NoArgsConstructor; + +/** + * @author didi + */ +@Data +@NoArgsConstructor +@AllArgsConstructor +@ApiModel(description = "指标详细属性信息") +public class MetricDetailDTO extends BaseDTO { + + @ApiModelProperty("指标名称") + private String metric; + + @ApiModelProperty("指标是否显示") + private Boolean set; + + @ApiModelProperty("指标优先级") + private Integer rank; + +} diff --git a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/dto/metrices/UserMetricConfigDTO.java b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/dto/metrices/UserMetricConfigDTO.java index 02bb1d2a..bf71b663 100644 --- a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/dto/metrices/UserMetricConfigDTO.java +++ b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/dto/metrices/UserMetricConfigDTO.java @@ -7,6 +7,7 @@ import lombok.AllArgsConstructor; import lombok.Data; import lombok.NoArgsConstructor; +import java.util.List; import java.util.Map; @@ -17,4 +18,7 @@ import java.util.Map; public class UserMetricConfigDTO extends BaseDTO { @ApiModelProperty("指标展示设置项，key：指标名；value：是否展现(true展现/false不展现)") private Map metricsSet; + + @ApiModelProperty("指标自定义属性列表") + private List metricDetailDTOList; } diff --git a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/dto/topic/TopicRecordDTO.java b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/dto/topic/TopicRecordDTO.java index b16f2e52..ca625f01 100644 --- a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/dto/topic/TopicRecordDTO.java +++ b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/dto/topic/TopicRecordDTO.java @@ -1,7 +1,8 @@ package com.xiaojukeji.know.streaming.km.common.bean.dto.topic; import com.fasterxml.jackson.annotation.JsonIgnoreProperties; -import com.xiaojukeji.know.streaming.km.common.bean.dto.BaseDTO; +import com.xiaojukeji.know.streaming.km.common.bean.dto.pagination.PaginationSortDTO; +import com.xiaojukeji.know.streaming.km.common.enums.OffsetTypeEnum; import io.swagger.annotations.ApiModel; import io.swagger.annotations.ApiModelProperty; import lombok.Data; @@ -15,7 +16,7 @@ import javax.validation.constraints.NotNull; @Data @JsonIgnoreProperties(ignoreUnknown = true) @ApiModel(description = "Topic记录") -public class TopicRecordDTO extends BaseDTO { +public class TopicRecordDTO extends PaginationSortDTO { @NotNull(message = "truncate不允许为空") @ApiModelProperty(value = "是否截断", example = "true") private Boolean truncate; @@ -34,4 +35,13 @@ public class TopicRecordDTO extends BaseDTO { @ApiModelProperty(value = "预览超时时间", example = "10000") private Long pullTimeoutUnitMs = 8000L; + + /** + * @see OffsetTypeEnum + */ + @ApiModelProperty(value = "offset", example = "") + private Integer filterOffsetReset = 0; + + @ApiModelProperty(value = "开始日期时间戳", example = "") + private Long startTimestampUnitMs; } diff --git a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/entity/broker/Broker.java b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/entity/broker/Broker.java index fa67cac5..752aade0 100644 --- a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/entity/broker/Broker.java +++ b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/entity/broker/Broker.java @@ -5,7 +5,6 @@ import com.alibaba.fastjson.TypeReference; import com.xiaojukeji.know.streaming.km.common.bean.entity.common.IpPortData; import com.xiaojukeji.know.streaming.km.common.bean.po.broker.BrokerPO; import com.xiaojukeji.know.streaming.km.common.utils.ConvertUtil; -import com.xiaojukeji.know.streaming.km.common.zookeeper.znode.brokers.BrokerMetadata; import lombok.AllArgsConstructor; import lombok.Data; import lombok.NoArgsConstructor; @@ -79,20 +78,6 @@ public class Broker implements Serializable { return metadata; } - public static Broker buildFrom(Long clusterPhyId, Integer brokerId, BrokerMetadata brokerMetadata) { - Broker metadata = new Broker(); - metadata.setClusterPhyId(clusterPhyId); - metadata.setBrokerId(brokerId); - metadata.setHost(brokerMetadata.getHost()); - metadata.setPort(brokerMetadata.getPort()); - metadata.setJmxPort(brokerMetadata.getJmxPort()); - metadata.setStartTimestamp(brokerMetadata.getTimestamp()); - metadata.setRack(brokerMetadata.getRack()); - metadata.setStatus(1); - metadata.setEndpointMap(brokerMetadata.getEndpointMap()); - return metadata; - } - public static Broker buildFrom(BrokerPO brokerPO) { Broker broker = ConvertUtil.obj2Obj(brokerPO, Broker.class); String endpointMapStr = brokerPO.getEndpointMap(); diff --git a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/entity/config/metric/UserMetricConfig.java b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/entity/config/metric/UserMetricConfig.java index 6895fb40..e244181a 100644 --- a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/entity/config/metric/UserMetricConfig.java +++ b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/entity/config/metric/UserMetricConfig.java @@ -1,12 +1,12 @@ package com.xiaojukeji.know.streaming.km.common.bean.entity.config.metric; +import com.xiaojukeji.know.streaming.km.common.constant.Constant; import lombok.AllArgsConstructor; import lombok.Data; import lombok.NoArgsConstructor; @Data @NoArgsConstructor -@AllArgsConstructor public class UserMetricConfig { private int type; @@ -15,6 +15,22 @@ public class UserMetricConfig { private boolean set; + private Integer rank; + + public UserMetricConfig(int type, String metric, boolean set, Integer rank) { + this.type = type; + this.metric = metric; + this.set = set; + this.rank = rank; + } + + public UserMetricConfig(int type, String metric, boolean set) { + this.type = type; + this.metric = metric; + this.set = set; + this.rank = null; + } + @Override public int hashCode(){ return metric.hashCode() << 1 + type; diff --git a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/vo/config/metric/UserMetricConfigVO.java b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/vo/config/metric/UserMetricConfigVO.java index e50fc6e7..2b4e76b3 100644 --- a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/vo/config/metric/UserMetricConfigVO.java +++ b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/vo/config/metric/UserMetricConfigVO.java @@ -14,4 +14,7 @@ import lombok.NoArgsConstructor; public class UserMetricConfigVO extends VersionItemVO { @ApiModelProperty(value = "该指标用户是否设置展现", example = "true") private Boolean set; + + @ApiModelProperty(value = "该指标展示优先级", example = "1") + private Integer rank; } diff --git a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/constant/Constant.java b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/constant/Constant.java index fae5db21..edd897ff 100644 --- a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/constant/Constant.java +++ b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/constant/Constant.java @@ -42,6 +42,7 @@ public class Constant { */ public static final Integer DEFAULT_CLUSTER_HEALTH_SCORE = 90; + public static final String DEFAULT_USER_NAME = "know-streaming-app"; public static final int INVALID_CODE = -1; @@ -64,4 +65,5 @@ public class Constant { public static final Float COLLECT_METRICS_ERROR_COST_TIME = -1.0F; public static final Integer DEFAULT_RETRY_TIME = 3; + } diff --git a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/constant/MsgConstant.java b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/constant/MsgConstant.java index 3d0b6a5c..1be8dadf 100644 --- a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/constant/MsgConstant.java +++ b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/constant/MsgConstant.java @@ -52,6 +52,10 @@ public class MsgConstant { /**************************************************** Partition ****************************************************/ + public static String getPartitionNoLeader(Long clusterPhyId, String topicName) { + return String.format("集群ID:[%d] Topic名称:[%s] 所有分区NoLeader", clusterPhyId, topicName); + } + public static String getPartitionNotExist(Long clusterPhyId, String topicName) { return String.format("集群ID:[%d] Topic名称:[%s] 存在非法的分区ID", clusterPhyId, topicName); } diff --git a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/enums/GroupOffsetResetEnum.java b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/enums/OffsetTypeEnum.java similarity index 50% rename from km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/enums/GroupOffsetResetEnum.java rename to km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/enums/OffsetTypeEnum.java index 178fb90e..6bdc8b80 100644 --- a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/enums/GroupOffsetResetEnum.java +++ b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/enums/OffsetTypeEnum.java @@ -3,19 +3,19 @@ package com.xiaojukeji.know.streaming.km.common.enums; import lombok.Getter; /** - * 重置offset + * offset类型 * @author zengqiao * @date 19/4/8 */ @Getter -public enum GroupOffsetResetEnum { - LATEST(0, "重置到最新"), +public enum OffsetTypeEnum { + LATEST(0, "最新"), - EARLIEST(1, "重置到最旧"), + EARLIEST(1, "最旧"), - PRECISE_TIMESTAMP(2, "按时间进行重置"), + PRECISE_TIMESTAMP(2, "指定时间"), - PRECISE_OFFSET(3, "重置到指定位置"), + PRECISE_OFFSET(3, "指定位置"), ; @@ -23,7 +23,7 @@ public enum GroupOffsetResetEnum { private final String message; - GroupOffsetResetEnum(int resetType, String message) { + OffsetTypeEnum(int resetType, String message) { this.resetType = resetType; this.message = message; } diff --git a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/jmx/JmxConnectorWrap.java b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/jmx/JmxConnectorWrap.java index 0fb65589..ca7c01c4 100644 --- a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/jmx/JmxConnectorWrap.java +++ b/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/jmx/JmxConnectorWrap.java @@ -90,6 +90,8 @@ public class JmxConnectorWrap { } try { jmxConnector.close(); + + jmxConnector = null; } catch (IOException e) { LOGGER.warn("close JmxConnector exception, physicalClusterId:{} brokerId:{} host:{} port:{}.", physicalClusterId, brokerId, host, port, e); } @@ -105,6 +107,11 @@ public class JmxConnectorWrap { acquire(); MBeanServerConnection mBeanServerConnection = jmxConnector.getMBeanServerConnection(); return mBeanServerConnection.getAttribute(name, attribute); + } catch (IOException ioe) { + // 如果是因为连接断开，则进行重新连接，并抛出异常 + reInitDueIOException(); + + throw ioe; } finally { atomicInteger.incrementAndGet(); } @@ -120,6 +127,11 @@ public class JmxConnectorWrap { acquire(); MBeanServerConnection mBeanServerConnection = jmxConnector.getMBeanServerConnection(); return mBeanServerConnection.getAttributes(name, attributes); + } catch (IOException ioe) { + // 如果是因为连接断开，则进行重新连接，并抛出异常 + reInitDueIOException(); + + throw ioe; } finally { atomicInteger.incrementAndGet(); } @@ -131,6 +143,11 @@ public class JmxConnectorWrap { acquire(); MBeanServerConnection mBeanServerConnection = jmxConnector.getMBeanServerConnection(); return mBeanServerConnection.queryNames(name, query); + } catch (IOException ioe) { + // 如果是因为连接断开，则进行重新连接，并抛出异常 + reInitDueIOException(); + + throw ioe; } finally { atomicInteger.incrementAndGet(); } @@ -186,4 +203,26 @@ public class JmxConnectorWrap { } } } + + private synchronized void reInitDueIOException() { + try { + if (jmxConnector == null) { + return; + } + + // 检查是否正常 + jmxConnector.getConnectionId(); + + // 如果正常则直接返回 + return; + } catch (Exception e) { + // ignore + } + + // 关闭旧的 + this.close(); + + // 重新创建 + this.checkJmxConnectionAndInitIfNeed(); + } } diff --git a/km-console/package-lock.json b/km-console/package-lock.json index e120e9eb..cb07da96 100644 --- a/km-console/package-lock.json +++ b/km-console/package-lock.json @@ -5100,9 +5100,9 @@ } }, "is-callable": { - "version": "1.2.5", - "resolved": "https://registry.npmmirror.com/is-callable/-/is-callable-1.2.5.tgz", - "integrity": "sha512-ZIWRujF6MvYGkEuHMYtFRkL2wAtFw89EHfKlXrkPkjQZZRWeh9L1q3SV13NIfHnqxugjLvAOkEHx9mb1zcMnEw==", + "version": "1.2.6", + "resolved": "https://registry.npmmirror.com/is-callable/-/is-callable-1.2.6.tgz", + "integrity": "sha512-krO72EO2NptOGAX2KYyqbP9vYMlNAXdB53rq6f8LXY6RY7JdSR/3BD6wLUlPHSAesmY9vstNrjvqGaCiRK/91Q==", "dev": true }, "is-ci": { diff --git a/km-console/packages/config-manager-fe/config/d1-webpack.base.js b/km-console/packages/config-manager-fe/config/d1-webpack.base.js deleted file mode 100644 index 95f0bc70..00000000 --- a/km-console/packages/config-manager-fe/config/d1-webpack.base.js +++ /dev/null @@ -1,205 +0,0 @@ -/* eslint-disable */ -const MiniCssExtractPlugin = require('mini-css-extract-plugin'); -const ProgressBarPlugin = require('progress-bar-webpack-plugin'); -const CaseSensitivePathsPlugin = require('case-sensitive-paths-webpack-plugin'); -const StatsPlugin = require('stats-webpack-plugin'); -const { CleanWebpackPlugin } = require('clean-webpack-plugin'); -const TerserJSPlugin = require('terser-webpack-plugin'); -const OptimizeCSSAssetsPlugin = require('optimize-css-assets-webpack-plugin'); -const HappyPack = require('happypack'); -const os = require('os'); -const happyThreadPool = HappyPack.ThreadPool({ size: os.cpus().length }); -const ReactRefreshWebpackPlugin = require('@pmmmwh/react-refresh-webpack-plugin'); -const theme = require('./theme'); -var cwd = process.cwd(); - -const path = require('path'); -const isProd = process.env.NODE_ENV === 'production'; -const babelOptions = { - cacheDirectory: true, - babelrc: false, - presets: [require.resolve('@babel/preset-env'), require.resolve('@babel/preset-typescript'), require.resolve('@babel/preset-react')], - plugins: [ - [require.resolve('@babel/plugin-proposal-decorators'), { legacy: true }], - [require.resolve('@babel/plugin-proposal-class-properties'), { loose: true }], - [require.resolve('@babel/plugin-proposal-private-methods'), { loose: true }], - require.resolve('@babel/plugin-proposal-export-default-from'), - require.resolve('@babel/plugin-proposal-export-namespace-from'), - require.resolve('@babel/plugin-proposal-object-rest-spread'), - require.resolve('@babel/plugin-transform-runtime'), - require.resolve('@babel/plugin-proposal-optional-chaining'), // - require.resolve('@babel/plugin-proposal-nullish-coalescing-operator'), // 解决 ?? 无法转义问题 - require.resolve('@babel/plugin-proposal-numeric-separator'), // 转义 1_000_000 - !isProd && require.resolve('react-refresh/babel'), - ] - .filter(Boolean) - .concat([ - [ - 'babel-plugin-import', - { - libraryName: 'antd', - style: true, - }, - ], - '@babel/plugin-transform-object-assign', - ]), -}; -module.exports = () => { - const manifestName = `manifest.json`; - const cssFileName = isProd ? '[name]-[chunkhash].css' : '[name].css'; - - const plugins = [ - new ProgressBarPlugin(), - new CaseSensitivePathsPlugin(), - new MiniCssExtractPlugin({ - filename: cssFileName, - }), - new StatsPlugin(manifestName, { - chunkModules: false, - source: true, - chunks: false, - modules: false, - assets: true, - children: false, - exclude: [/node_modules/], - }), - new HappyPack({ - id: 'babel', - loaders: [ - 'cache-loader', - { - loader: 'babel-loader', - options: babelOptions, - }, - ], - threadPool: happyThreadPool, - }), - !isProd && - new ReactRefreshWebpackPlugin({ - overlay: false, - }), - // new BundleAnalyzerPlugin({ - // analyzerPort: 8889 - // }), - ].filter(Boolean); - if (isProd) { - plugins.push(new CleanWebpackPlugin()); - } - return { - externals: isProd - ? [ - /^react$/, - /^react\/lib.*/, - /^react-dom$/, - /.*react-dom.*/, - /^single-spa$/, - /^single-spa-react$/, - /^moment$/, - /^antd$/, - /^lodash$/, - /^react-router$/, - /^react-router-dom$/, - ] - : [], - resolve: { - symlinks: false, - extensions: ['.web.jsx', '.web.js', '.ts', '.tsx', '.js', '.jsx', '.json'], - alias: { - // '@pkgs': path.resolve(cwd, 'src/packages'), - '@pkgs': path.resolve(cwd, './node_modules/@didi/d1-packages'), - '@cpts': path.resolve(cwd, 'src/components'), - '@interface': path.resolve(cwd, 'src/interface'), - '@apis': path.resolve(cwd, 'src/api'), - react: path.resolve('./node_modules/react'), - actions: path.resolve(cwd, 'src/actions'), - lib: path.resolve(cwd, 'src/lib'), - constants: path.resolve(cwd, 'src/constants'), - components: path.resolve(cwd, 'src/components'), - container: path.resolve(cwd, 'src/container'), - api: path.resolve(cwd, 'src/api'), - assets: path.resolve(cwd, 'src/assets'), - mobxStore: path.resolve(cwd, 'src/mobxStore'), - }, - }, - plugins, - module: { - rules: [ - { - parser: { system: false }, - }, - { - test: /\.(js|jsx|ts|tsx)$/, - exclude: /node_modules\/(?!react-intl|@didi\/dcloud-design)/, - use: [ - { - loader: 'happypack/loader?id=babel', - }, - ], - }, - { - test: /\.(png|svg|jpeg|jpg|gif|ttf|woff|woff2|eot|pdf)$/, - use: [ - { - loader: 'file-loader', - options: { - name: '[name].[ext]', - outputPath: './assets/image/', - esModule: false, - }, - }, - ], - }, - { - test: /\.(css|less)$/, - use: [ - { - loader: MiniCssExtractPlugin.loader, - }, - 'css-loader', - { - loader: 'less-loader', - options: { - javascriptEnabled: true, - modifyVars: theme, - }, - }, - ], - }, - ], - }, - optimization: Object.assign( - { - splitChunks: { - cacheGroups: { - vendor: { - test: /[\\/]node_modules[\\/]/, - chunks: 'all', - name: 'vendor', - priority: 10, - enforce: true, - minChunks: 1, - maxSize: 3500000, - }, - }, - }, - }, - isProd - ? { - minimizer: [ - new TerserJSPlugin({ - cache: true, - sourceMap: true, - }), - new OptimizeCSSAssetsPlugin({}), - ], - } - : {} - ), - devtool: isProd ? 'cheap-module-source-map' : 'source-map', - node: { - fs: 'empty', - net: 'empty', - tls: 'empty', - }, - }; -}; diff --git a/km-console/packages/config-manager-fe/config/webpack.common.js b/km-console/packages/config-manager-fe/config/webpack.common.js new file mode 100644 index 00000000..cef88e61 --- /dev/null +++ b/km-console/packages/config-manager-fe/config/webpack.common.js @@ -0,0 +1,132 @@ +const path = require('path'); +const webpack = require('webpack'); +const HtmlWebpackPlugin = require('html-webpack-plugin'); +const MiniCssExtractPlugin = require('mini-css-extract-plugin'); +const ProgressBarPlugin = require('progress-bar-webpack-plugin'); +const CaseSensitivePathsPlugin = require('case-sensitive-paths-webpack-plugin'); +const StatsPlugin = require('stats-webpack-plugin'); +const HappyPack = require('happypack'); +const os = require('os'); +const happyThreadPool = HappyPack.ThreadPool({ size: os.cpus().length }); +const theme = require('./theme'); +const pkgJson = require('../package'); + +const devMode = process.env.NODE_ENV === 'development'; +const babelOptions = { + cacheDirectory: true, + babelrc: false, + presets: [require.resolve('@babel/preset-env'), require.resolve('@babel/preset-typescript'), require.resolve('@babel/preset-react')], + plugins: [ + [require.resolve('@babel/plugin-proposal-decorators'), { legacy: true }], + [require.resolve('@babel/plugin-proposal-class-properties'), { loose: true }], + [require.resolve('@babel/plugin-proposal-private-methods'), { loose: true }], + [require.resolve('@babel/plugin-proposal-private-property-in-object'), { loose: true }], + require.resolve('@babel/plugin-proposal-export-default-from'), + require.resolve('@babel/plugin-proposal-export-namespace-from'), + require.resolve('@babel/plugin-proposal-object-rest-spread'), + require.resolve('@babel/plugin-transform-runtime'), + require.resolve('@babel/plugin-proposal-optional-chaining'), // + require.resolve('@babel/plugin-proposal-nullish-coalescing-operator'), // 解决 ?? 无法转义问题 + require.resolve('@babel/plugin-proposal-numeric-separator'), // 转义 1_000_000 + devMode && require.resolve('react-refresh/babel'), + ].filter(Boolean), +}; + +module.exports = { + entry: { + [pkgJson.ident]: ['./src/index.tsx'], + }, + resolve: { + symlinks: false, + extensions: ['.web.jsx', '.web.js', '.ts', '.tsx', '.js', '.jsx', '.json'], + alias: { + '@src': path.resolve(process.cwd(), 'src'), + }, + }, + plugins: [ + new ProgressBarPlugin(), + new CaseSensitivePathsPlugin(), + new StatsPlugin('manifest.json', { + chunkModules: false, + source: true, + chunks: false, + modules: false, + assets: true, + children: false, + exclude: [/node_modules/], + }), + new HappyPack({ + id: 'babel', + loaders: [ + 'cache-loader', + { + loader: 'babel-loader', + options: babelOptions, + }, + ], + threadPool: happyThreadPool, + }), + new webpack.DefinePlugin({ + 'process.env': { + NODE_ENV: JSON.stringify(process.env.NODE_ENV), + RUN_ENV: JSON.stringify(process.env.RUN_ENV), + }, + }), + new HtmlWebpackPlugin({ + meta: { + manifest: 'manifest.json', + }, + template: './src/index.html', + inject: 'body', + }), + ].filter(Boolean), + module: { + rules: [ + { + parser: { system: false }, + }, + { + test: /\.(js|jsx|ts|tsx)$/, + exclude: /node_modules\/(?!react-intl|@didi\/dcloud-design)/, + use: [ + { + loader: 'happypack/loader?id=babel', + }, + ], + }, + { + test: /\.(png|svg|jpeg|jpg|gif|ttf|woff|woff2|eot|pdf)$/, + use: [ + { + loader: 'file-loader', + options: { + name: '[name].[ext]', + outputPath: './assets/image/', + esModule: false, + }, + }, + ], + }, + { + test: /\.(css|less)$/, + use: [ + MiniCssExtractPlugin.loader, + 'css-loader', + { + loader: 'less-loader', + options: { + javascriptEnabled: true, + modifyVars: theme, + }, + }, + ], + }, + ], + }, + node: { + fs: 'empty', + net: 'empty', + tls: 'empty', + }, + stats: 'errors-warnings', +}; diff --git a/km-console/packages/config-manager-fe/config/webpack.dev.js b/km-console/packages/config-manager-fe/config/webpack.dev.js new file mode 100644 index 00000000..1b60f703 --- /dev/null +++ b/km-console/packages/config-manager-fe/config/webpack.dev.js @@ -0,0 +1,35 @@ +const MiniCssExtractPlugin = require('mini-css-extract-plugin'); +const ReactRefreshWebpackPlugin = require('@pmmmwh/react-refresh-webpack-plugin'); +const pkgJson = require('../package'); + +module.exports = { + mode: 'development', + plugins: [ + new MiniCssExtractPlugin(), + new ReactRefreshWebpackPlugin({ + overlay: false, + }), + ], + devServer: { + host: '127.0.0.1', + port: pkgJson.port, + hot: true, + open: false, + publicPath: `http://localhost:${pkgJson.port}/${pkgJson.ident}/`, + inline: true, + disableHostCheck: true, + historyApiFallback: true, + headers: { + 'Access-Control-Allow-Origin': '*', + }, + }, + output: { + path: '/', + publicPath: `http://localhost:${pkgJson.port}/${pkgJson.ident}/`, + library: pkgJson.ident, + libraryTarget: 'amd', + filename: '[name].js', + chunkFilename: '[name].js', + }, + devtool: 'cheap-module-eval-source-map', +}; diff --git a/km-console/packages/config-manager-fe/config/webpack.prod.js b/km-console/packages/config-manager-fe/config/webpack.prod.js new file mode 100644 index 00000000..9dd9ee66 --- /dev/null +++ b/km-console/packages/config-manager-fe/config/webpack.prod.js @@ -0,0 +1,59 @@ +const path = require('path'); +const MiniCssExtractPlugin = require('mini-css-extract-plugin'); +const { CleanWebpackPlugin } = require('clean-webpack-plugin'); +const TerserJSPlugin = require('terser-webpack-plugin'); +const OptimizeCSSAssetsPlugin = require('optimize-css-assets-webpack-plugin'); +const pkgJson = require('../package'); + +module.exports = { + mode: 'production', + externals: [ + /^react$/, + /^react\/lib.*/, + /^react-dom$/, + /.*react-dom.*/, + /^single-spa$/, + /^single-spa-react$/, + /^moment$/, + /^lodash$/, + /^react-router$/, + /^react-router-dom$/, + ], + plugins: [ + new CleanWebpackPlugin(), + new MiniCssExtractPlugin({ + filename: '[name]-[chunkhash].css', + }), + ], + output: { + path: path.resolve(process.cwd(), `../../../km-rest/src/main/resources/templates/${pkgJson.ident}`), + publicPath: `${process.env.PUBLIC_PATH}/${pkgJson.ident}/`, + library: pkgJson.ident, + libraryTarget: 'amd', + filename: '[name]-[chunkhash].js', + chunkFilename: '[name]-[chunkhash].js', + }, + optimization: { + splitChunks: { + cacheGroups: { + vendor: { + test: /[\\/]node_modules[\\/]/, + chunks: 'all', + name: 'vendor', + priority: 10, + enforce: true, + minChunks: 1, + maxSize: 3500000, + }, + }, + }, + minimizer: [ + new TerserJSPlugin({ + cache: true, + sourceMap: true, + }), + new OptimizeCSSAssetsPlugin({}), + ], + }, + devtool: 'none', +}; diff --git a/km-console/packages/config-manager-fe/package-lock.json b/km-console/packages/config-manager-fe/package-lock.json index 6c32859e..bf0a4f18 100644 --- a/km-console/packages/config-manager-fe/package-lock.json +++ b/km-console/packages/config-manager-fe/package-lock.json @@ -1344,6 +1344,16 @@ "@jridgewell/sourcemap-codec": "^1.4.10" } }, + "@knowdesign/icons": { + "version": "1.0.0", + "resolved": "https://registry.npmmirror.com/@knowdesign/icons/-/icons-1.0.0.tgz", + "integrity": "sha512-7c+h2TSbh2ihTkXIivuO+DddNC5wG7hVv9SS4ccmkvTKls2ZTLitPu+U0wpufDxPhkPMaKEQfsECsVJ+7jLMiw==", + "requires": { + "@ant-design/colors": "^6.0.0", + "@ant-design/icons": "^4.7.0", + "react": "16.12.0" + } + }, "@nodelib/fs.scandir": { "version": "2.1.5", "resolved": "https://registry.npmmirror.com/@nodelib/fs.scandir/-/fs.scandir-2.1.5.tgz", @@ -6815,9 +6825,9 @@ "dev": true }, "is-callable": { - "version": "1.2.5", - "resolved": "https://registry.npmmirror.com/is-callable/-/is-callable-1.2.5.tgz", - "integrity": "sha512-ZIWRujF6MvYGkEuHMYtFRkL2wAtFw89EHfKlXrkPkjQZZRWeh9L1q3SV13NIfHnqxugjLvAOkEHx9mb1zcMnEw==", + "version": "1.2.6", + "resolved": "https://registry.npmmirror.com/is-callable/-/is-callable-1.2.6.tgz", + "integrity": "sha512-krO72EO2NptOGAX2KYyqbP9vYMlNAXdB53rq6f8LXY6RY7JdSR/3BD6wLUlPHSAesmY9vstNrjvqGaCiRK/91Q==", "dev": true }, "is-color-stop": { diff --git a/km-console/packages/config-manager-fe/package.json b/km-console/packages/config-manager-fe/package.json index 2b820594..945e8b5c 100644 --- a/km-console/packages/config-manager-fe/package.json +++ b/km-console/packages/config-manager-fe/package.json @@ -21,9 +21,11 @@ "build": "cross-env NODE_ENV=production webpack --max_old_space_size=8000" }, "dependencies": { + "@knowdesign/icons": "^1.0.0", "babel-preset-react-app": "^10.0.0", "classnames": "^2.2.6", "dotenv": "^16.0.1", + "knowdesign": "1.3.7", "less": "^3.9.0", "lodash": "^4.17.11", "mobx": "4.15.7", @@ -36,8 +38,7 @@ "react-intl": "^3.2.1", "react-router-cache-route": "^1.11.1", "single-spa": "^5.8.0", - "single-spa-react": "^2.14.0", - "knowdesign": "1.3.7" + "single-spa-react": "^2.14.0" }, "devDependencies": { "@ant-design/icons": "^4.6.2", diff --git a/km-console/packages/config-manager-fe/src/components/TypicalListCard/index.less b/km-console/packages/config-manager-fe/src/components/TypicalListCard/index.less index 06516afa..9a48d134 100644 --- a/km-console/packages/config-manager-fe/src/components/TypicalListCard/index.less +++ b/km-console/packages/config-manager-fe/src/components/TypicalListCard/index.less @@ -22,6 +22,20 @@ display: flex; justify-content: space-between; margin-bottom: 12px; + .left, + .right { + display: flex; + align-items: center; + } + .left .refresh-icon { + font-size: 20px; + color: #74788d; + cursor: pointer; + } + .right .search-input { + width: 248px; + margin-right: 8px; + } } } } diff --git a/km-console/packages/config-manager-fe/src/constants/axiosConfig.ts b/km-console/packages/config-manager-fe/src/constants/axiosConfig.ts index f08ebc77..3e05e105 100644 --- a/km-console/packages/config-manager-fe/src/constants/axiosConfig.ts +++ b/km-console/packages/config-manager-fe/src/constants/axiosConfig.ts @@ -47,8 +47,8 @@ serviceInstance.interceptors.response.use( return res; }, (err: any) => { - const config = err.config; - if (!config || !config.retryTimes) return dealResponse(err, config.customNotification); + const config = err?.config; + if (!config || !config.retryTimes) return dealResponse(err); const { __retryCount = 0, retryDelay = 300, retryTimes } = config; config.__retryCount = __retryCount; if (__retryCount >= retryTimes) { diff --git a/km-console/packages/config-manager-fe/src/pages/CommonConfig.tsx b/km-console/packages/config-manager-fe/src/pages/CommonConfig.tsx index 2a9a9a74..835abf51 100644 --- a/km-console/packages/config-manager-fe/src/pages/CommonConfig.tsx +++ b/km-console/packages/config-manager-fe/src/pages/CommonConfig.tsx @@ -1,6 +1,6 @@ import React, { useLayoutEffect } from 'react'; import { Utils, AppContainer } from 'knowdesign'; -import { goLogin } from 'constants/axiosConfig'; +import { goLogin } from '@src/constants/axiosConfig'; // 权限对应表 export enum ConfigPermissionMap { diff --git a/km-console/packages/config-manager-fe/src/pages/ConfigManage/index.tsx b/km-console/packages/config-manager-fe/src/pages/ConfigManage/index.tsx index 84b5d2f6..1430fbb6 100644 --- a/km-console/packages/config-manager-fe/src/pages/ConfigManage/index.tsx +++ b/km-console/packages/config-manager-fe/src/pages/ConfigManage/index.tsx @@ -15,6 +15,7 @@ import { AppContainer, Utils, } from 'knowdesign'; +import { IconFont } from '@knowdesign/icons'; import { PlusOutlined } from '@ant-design/icons'; import moment from 'moment'; // 引入代码编辑器 @@ -26,8 +27,8 @@ import 'codemirror/addon/selection/active-line'; import 'codemirror/addon/edit/closebrackets'; require('codemirror/mode/xml/xml'); require('codemirror/mode/javascript/javascript'); -import api from 'api'; -import { defaultPagination } from 'constants/common'; +import api from '@src/api'; +import { defaultPagination } from '@src/constants/common'; import TypicalListCard from '../../components/TypicalListCard'; import { ConfigPermissionMap } from '../CommonConfig'; import { ConfigOperate, ConfigProps } from './config'; @@ -384,7 +385,7 @@ export default () => { const onDelete = (record: ConfigProps) => { confirm({ title: '确定删除配置吗?', - content: `配置⌈${record.valueName}⌋${record.status === 1 ? '为启用状态，无法删除' : ''}`, + content: `配置 [${record.valueName}] ${record.status === 1 ? '为启用状态，无法删除' : ''}`, centered: true, okText: '删除', okType: 'primary', @@ -398,9 +399,11 @@ export default () => { }, maskClosable: true, onOk() { - return request(api.editConfig, { - method: 'POST', - data: record.id, + return request(api.delConfig, { + method: 'DELETE', + params: { + id: record.id, + }, }).then((_) => { message.success('删除成功'); getConfigList(); @@ -431,22 +434,28 @@ export default () => {
-
getConfigList({ page: 1 })}> - - - - - - - - - -
+
+
getConfigList()}> + +
+ +
getConfigList({ page: 1 })}> + + + + + + + + + +
+
{global.hasPermission && global.hasPermission(ConfigPermissionMap.CONFIG_ADD) ? ( - - +
+
getData()}> + +
+ + +
getData({ page: 1 })}> + + + + + + + + + + + + +
+
{ return ( <> -
- { - setSearchKeywords(searchKeywordsInput); - }} - style={{ fontSize: '16px' }} - /> - } - placeholder="请输入角色名称" - value={searchKeywordsInput} - onPressEnter={(_) => { - setSearchKeywords(searchKeywordsInput); - }} - onChange={(e) => { - setSearchKeywordsInput(e.target.value); - }} - /> - {global.hasPermission && global.hasPermission(ConfigPermissionMap.ROLE_ADD) ? ( - - ) : ( - <> - )} +
+
+
getRoleList()}> + +
+
+
+ { + setSearchKeywords(searchKeywordsInput); + }} + style={{ fontSize: '16px' }} + /> + } + placeholder="请输入角色名称" + value={searchKeywordsInput} + onPressEnter={(_) => { + setSearchKeywords(searchKeywordsInput); + }} + onChange={(e) => { + setSearchKeywordsInput(e.target.value); + }} + /> + {global.hasPermission && global.hasPermission(ConfigPermissionMap.ROLE_ADD) ? ( + + ) : ( + <> + )} +
{ return ( <>
-
getUserList({ page: 1 })}> - - - - - - - - + + + + + + { }} onPressEnter={searchFn} /> - - { }} onPressEnter={searchFn} /> - +
{/* */}
-
+ {/*
*/}
)} {/* diff --git a/km-console/packages/layout-clusters-fe/src/pages/Jobs/RebalancePlan.tsx b/km-console/packages/layout-clusters-fe/src/pages/Jobs/RebalancePlan.tsx index 4b11b2e7..024fced3 100644 --- a/km-console/packages/layout-clusters-fe/src/pages/Jobs/RebalancePlan.tsx +++ b/km-console/packages/layout-clusters-fe/src/pages/Jobs/RebalancePlan.tsx @@ -13,8 +13,8 @@ interface PropsType { } const typeObj: any = { - 1: '周期均衡', - 2: '立即均衡', + 1: '立即均衡', + 2: '周期均衡', }; const { request, post } = Utils; diff --git a/km-console/packages/layout-clusters-fe/src/pages/Jobs/index.tsx b/km-console/packages/layout-clusters-fe/src/pages/Jobs/index.tsx index 84e70298..5432b434 100644 --- a/km-console/packages/layout-clusters-fe/src/pages/Jobs/index.tsx +++ b/km-console/packages/layout-clusters-fe/src/pages/Jobs/index.tsx @@ -1,6 +1,6 @@ import React, { useState, useEffect, memo } from 'react'; import { useParams, useHistory, useLocation } from 'react-router-dom'; -import { ProTable, Drawer, Utils, AppContainer, Form, Select, Input, Button, message, Modal } from 'knowdesign'; +import { ProTable, Drawer, Utils, AppContainer, Form, Select, Input, Button, message, Modal, IconFont, Divider } from 'knowdesign'; import API from '../../api'; import { getJobsListColumns, defaultPagination, runningStatus, jobType } from './config'; import JobsCheck from '@src/components/CardBar/JobsCheck'; @@ -10,6 +10,7 @@ import './index.less'; import ReplicaChange from '@src/components/TopicJob/ReplicaChange'; import ReplicaMove from '@src/components/TopicJob/ReplicaMove'; import BalanceDrawer from '../LoadRebalance/BalanceDrawer'; +import { tableHeaderPrefix } from '@src/constants/common'; const { request } = Utils; const JobsList: React.FC = (props: any) => { @@ -171,35 +172,44 @@ const JobsList: React.FC = (props: any) => { {/* */}
-
- - - - - - + + + + + + @@ -354,8 +365,7 @@ const LoadBalance: React.FC = (props: any) => { */} - -
+
{ const intl = useIntl(); const [form] = Form.useForm(); const [loading, setLoading] = React.useState(false); + const [confirmLoading, setConfirmLoading] = React.useState(false); const [curClusterInfo, setCurClusterInfo] = React.useState({}); - const [security, setSecurity] = React.useState(curClusterInfo?.security || 'None'); const [extra, setExtra] = React.useState({ versionExtra: '', zooKeeperExtra: '', bootstrapExtra: '', jmxExtra: '', }); - const [isLowVersion, setIsLowVersion] = React.useState(false); - const [zookeeperErrorStatus, setZookeeperErrorStatus] = React.useState(false); const lastFormItemValue = React.useRef({ - bootstrap: curClusterInfo?.bootstrapServers || '', + bootstrapServers: curClusterInfo?.bootstrapServers || '', zookeeper: curClusterInfo?.zookeeper || '', clientProperties: curClusterInfo?.clientProperties || {}, }); - const onHandleValuesChange = (value: any, allValues: any) => { - Object.keys(value).forEach((key) => { + const onHandleValuesChange = (changedValue: string[]) => { + Object.keys(changedValue).forEach((key) => { switch (key) { - case 'security': - setSecurity(value.security); - break; case 'zookeeper': - setExtra({ - ...extra, - zooKeeperExtra: '', - bootstrapExtra: '', - jmxExtra: '', - }); - break; case 'bootstrapServers': setExtra({ ...extra, @@ -78,21 +65,19 @@ const AccessClusters = (props: any): JSX.Element => { const onCancel = () => { form.resetFields(); setLoading(false); - setZookeeperErrorStatus(false); - setIsLowVersion(false); - setSecurity('None'); setExtra({ versionExtra: '', zooKeeperExtra: '', bootstrapExtra: '', jmxExtra: '', }); - lastFormItemValue.current = { bootstrap: '', zookeeper: '', clientProperties: {} }; + lastFormItemValue.current = { bootstrapServers: '', zookeeper: '', clientProperties: {} }; props.setVisible && props.setVisible(false); }; const onSubmit = () => { form.validateFields().then((res) => { + setConfirmLoading(true); let clientProperties = null; try { clientProperties = res.clientProperties && JSON.parse(res.clientProperties); @@ -107,7 +92,7 @@ const AccessClusters = (props: any): JSX.Element => { jmxProperties: { jmxPort: res.jmxPort, maxConn: res.maxConn, - openSSL: res.security === 'Password', + openSSL: res.openSSL || false, token: res.token, username: res.username, }, @@ -115,7 +100,7 @@ const AccessClusters = (props: any): JSX.Element => { name: res.name, zookeeper: res.zookeeper || '', }; - setLoading(true); + if (!isNaN(curClusterInfo?.id)) { Utils.put(api.phyCluster, { ...params, @@ -127,7 +112,7 @@ const AccessClusters = (props: any): JSX.Element => { onCancel(); }) .finally(() => { - setLoading(false); + setConfirmLoading(false); }); } else { Utils.post(api.phyCluster, params) @@ -137,7 +122,7 @@ const AccessClusters = (props: any): JSX.Element => { onCancel(); }) .finally(() => { - setLoading(false); + setConfirmLoading(false); }); } }); @@ -154,125 +139,224 @@ const AccessClusters = (props: any): JSX.Element => { } setLoading(true); - setIsLowVersion(false); - setZookeeperErrorStatus(false); return Utils.post(api.kafkaValidator, { bootstrapServers: bootstrapServers || '', zookeeper: zookeeper || '', clientProperties, }) - .then((res: any) => { - form.setFieldsValue({ - jmxPort: res.jmxPort, - }); + .then( + (res: { + errList: { code: number; message: string; data: any }[]; + jmxPort: number | null; + kafkaVersion: string | null; + zookeeper: string | null; + }) => { + const changedValue: { jmxPort?: number; kafkaVersion?: string; zookeeper: string } = { + zookeeper: zookeeper || res.zookeeper, + }; + if (res.kafkaVersion && props.kafkaVersion.includes(res.kafkaVersion)) { + changedValue.kafkaVersion = res.kafkaVersion; + } + if (res.jmxPort) { + changedValue.jmxPort = res.jmxPort; + } + form.setFieldsValue(changedValue); - if (props.kafkaVersion.indexOf(res.kafkaVersion) > -1) { - form.setFieldsValue({ - kafkaVersion: res.kafkaVersion, - }); - } else { - form.setFieldsValue({ - kafkaVersion: undefined, + const extraMsg = { + ...extra, + // 重置默认信息为连接成功 + bootstrapExtra: bootstrapServers ? '连接成功' : '', + zooKeeperExtra: zookeeper ? '连接成功' : '', + }; + + const errList = res.errList || []; + // 处理错误信息 + errList.forEach((item: any) => { + const { code, message } = item; + let modifyKey: 'bootstrapExtra' | 'zooKeeperExtra' | 'jmxExtra' | undefined; + if (bootstrapServersErrCodes.includes(code)) { + modifyKey = 'bootstrapExtra'; + } else if (zkErrCodes.includes(code)) { + modifyKey = 'zooKeeperExtra'; + } else if (jmxErrCodes.includes(code)) { + modifyKey = 'jmxExtra'; + } + + if (modifyKey) { + extraMsg[modifyKey] = message; + } }); + + setExtra(extraMsg); + return res; } - - form.setFieldsValue({ - zookeeper: zookeeper || res.zookeeper, - }); - - const errList = res.errList || []; - - const extraMsg = extra; - - // 初始化信息为连接成功 - extraMsg.bootstrapExtra = bootstrapServers ? '连接成功' : ''; - extraMsg.zooKeeperExtra = zookeeper ? '连接成功' : ''; - - // 处理错误信息 - errList.forEach((item: any) => { - const { code, message } = item; - let modifyKey: 'bootstrapExtra' | 'zooKeeperExtra' | 'jmxExtra' | undefined; - if (bootstrapServersErrCodes.includes(code)) { - modifyKey = 'bootstrapExtra'; - } else if (zkErrCodes.includes(code)) { - modifyKey = 'zooKeeperExtra'; - } else if (jmxErrCodes.includes(code)) { - modifyKey = 'jmxExtra'; - } - - if (modifyKey) { - extraMsg[modifyKey] = `连接失败。${message}`; - } - }); - - // 如果kafkaVersion小于最低版本则提示 - const showLowVersion = !( - curClusterInfo?.zookeeper || - !curClusterInfo?.kafkaVersion || - curClusterInfo?.kafkaVersion >= lowKafkaVersion - ); - setIsLowVersion(showLowVersion); - setExtra({ - ...extraMsg, - versionExtra: showLowVersion ? intl.formatMessage({ id: 'access.cluster.low.version.tip' }) : '', - }); - return res; - }) + ) .finally(() => { setLoading(false); }); }; + // 更新表单状态 React.useEffect(() => { - const showLowVersion = !(curClusterInfo?.zookeeper || !curClusterInfo?.kafkaVersion || curClusterInfo?.kafkaVersion >= lowKafkaVersion); lastFormItemValue.current = { - bootstrap: curClusterInfo?.bootstrapServers || '', + bootstrapServers: curClusterInfo?.bootstrapServers || '', zookeeper: curClusterInfo?.zookeeper || '', clientProperties: curClusterInfo?.clientProperties || {}, }; - setIsLowVersion(showLowVersion); - setExtra({ - ...extra, - versionExtra: showLowVersion ? intl.formatMessage({ id: 'access.cluster.low.version.tip' }) : '', - }); form.setFieldsValue({ ...curClusterInfo }); + if (curClusterInfo?.kafkaVersion) { + form.validateFields(['kafkaVersion']); + } }, [curClusterInfo]); + // 获取集群详情数据 React.useEffect(() => { if (visible) { if (clusterInfo?.id) { setLoading(true); + + const resolveJmxProperties = (obj: any) => { + const res = { ...obj }; + try { + const originValue = obj?.jmxProperties; + if (originValue) { + const jmxProperties = JSON.parse(originValue); + typeof jmxProperties === 'object' && jmxProperties !== null && Object.assign(res, jmxProperties); + } + } catch (err) { + console.error('jmxProperties not JSON: ', err); + } + return res; + }; + Utils.request(api.getPhyClusterBasic(clusterInfo.id)) .then((res: any) => { - let jmxProperties = null; - try { - jmxProperties = JSON.parse(res?.jmxProperties); - } catch (err) { - console.error(err); - } - - // 转化值对应成表单值 - if (jmxProperties?.openSSL) { - jmxProperties.security = 'Password'; - } - - if (jmxProperties) { - res = Object.assign({}, res || {}, jmxProperties); - } - setCurClusterInfo(res); - setLoading(false); + setCurClusterInfo(resolveJmxProperties(res)); }) .catch((err) => { - setCurClusterInfo(clusterInfo); + setCurClusterInfo(resolveJmxProperties(clusterInfo)); + }) + .finally(() => { setLoading(false); }); } else { - setCurClusterInfo(clusterInfo); + setCurClusterInfo({}); } } }, [visible, clusterInfo]); + const validators = { + name: async (_: any, value: string) => { + if (!value) { + return Promise.reject('集群名称不能为空'); + } + if (value === curClusterInfo?.name) { + return Promise.resolve(); + } + if (value?.length > 128) { + return Promise.reject('集群名称长度限制在1～128字符'); + } + if (!new RegExp(regClusterName).test(value)) { + return Promise.reject('集群名称支持中英文、数字、特殊字符 ! " # $ % & \' ( ) * + , - . / : ; < = > ? @ [ ] ^ _ ` { | } ~'); + } + return Utils.request(api.getClusterBasicExit(value)) + .then((res: any) => { + const data = res || {}; + return data?.exist ? Promise.reject('集群名称重复') : Promise.resolve(); + }) + .catch(() => Promise.reject('连接超时! 请重试或检查服务')); + }, + bootstrapServers: async (_: any, value: string) => { + if (!value) { + return Promise.reject('Bootstrap Servers不能为空'); + } + if (value.length > 2000) { + return Promise.reject('Bootstrap Servers长度限制在2000字符'); + } + if (value && value !== lastFormItemValue.current.bootstrapServers) { + lastFormItemValue.current.bootstrapServers = value; + return connectTest().catch(() => (lastFormItemValue.current.bootstrapServers = '')); + } + return Promise.resolve(''); + }, + zookeeper: async (_: any, value: string) => { + if (!value) { + return Promise.resolve(''); + } + + if (value.length > 2000) { + return Promise.reject('Zookeeper长度限制在2000字符'); + } + + if (value && value !== lastFormItemValue.current.zookeeper) { + lastFormItemValue.current.zookeeper = value; + return connectTest().catch(() => (lastFormItemValue.current.zookeeper = '')); + } + return Promise.resolve(''); + }, + securityUserName: async (_: any, value: string) => { + if (!value) { + return Promise.reject('用户名不能为空'); + } + if (!new RegExp(regUsername).test(value)) { + return Promise.reject('仅支持大小写、下划线、短划线（-）'); + } + if (value.length > 128) { + return Promise.reject('用户名长度限制在1～128字符'); + } + return Promise.resolve(); + }, + securityToken: async (_: any, value: string) => { + if (!value) { + return Promise.reject('密码不能为空'); + } + if (!new RegExp(regUsername).test(value)) { + return Promise.reject('密码只能由大小写、下划线、短划线（-）组成'); + } + if (value.length < 6 || value.length > 32) { + return Promise.reject('密码长度限制在6～32字符'); + } + return Promise.resolve(); + }, + kafkaVersion: async (_: any, value: any) => { + if (!value) { + return Promise.reject('版本号不能为空'); + } + // 检测版本号小于2.8.0，如果没有填zookeeper信息，才会提示 + const zookeeper = form.getFieldValue('zookeeper'); + let versionExtra = ''; + if (value < LOW_KAFKA_VERSION && !zookeeper) { + versionExtra = intl.formatMessage({ id: 'access.cluster.low.version.tip' }); + } + setExtra({ + ...extra, + versionExtra, + }); + return Promise.resolve(); + }, + clientProperties: async (_: any, value: string) => { + try { + if (value) { + JSON.parse(value); + } + + return Promise.resolve(); + } catch (e) { + return Promise.reject(new Error('输入内容必须为 JSON')); + } + }, + description: async (_: any, value: string) => { + if (!value) { + return Promise.resolve(''); + } + if (value && value.length > 200) { + return Promise.reject('集群描述长度限制在200字符'); + } + return Promise.resolve(); + }, + }; + return ( <> { -
} - title={intl.formatMessage({ id: props.title || 'access.cluster' })} + title={intl.formatMessage({ id: props.title || clusterInfo?.id ? 'edit.cluster' : 'access.cluster' })} visible={props.visible} placement="right" width={480} @@ -306,30 +390,7 @@ const AccessClusters = (props: any): JSX.Element => { rules={[ { required: true, - validator: async (rule: any, value: string) => { - if (!value) { - return Promise.reject('集群名称不能为空'); - } - if (value === curClusterInfo?.name) { - return Promise.resolve(); - } - if (value?.length > 128) { - return Promise.reject('集群名称长度限制在1～128字符'); - } - if (!new RegExp(regClusterName).test(value)) { - return Promise.reject( - '集群名称支持中英文、数字、特殊字符 ! " # $ % & \' ( ) * + , - . / : ; < = > ? @ [ ] ^ _ ` { | } ~' - ); - } - return Utils.request(api.getClusterBasicExit(value)).then((res: any) => { - const data = res || {}; - if (data?.exist) { - return Promise.reject('集群名称重复'); - } else { - return Promise.resolve(); - } - }); - }, + validator: validators.name, }, ]} > @@ -338,31 +399,12 @@ const AccessClusters = (props: any): JSX.Element => { {extra.bootstrapExtra}} + extra={{extra.bootstrapExtra}} validateTrigger={'onBlur'} rules={[ { required: true, - validator: async (rule: any, value: string) => { - if (!value) { - return Promise.reject('Bootstrap Servers不能为空'); - } - if (value.length > 2000) { - return Promise.reject('Bootstrap Servers长度限制在2000字符'); - } - if (value && value !== lastFormItemValue.current.bootstrap) { - return connectTest() - .then((res: any) => { - lastFormItemValue.current.bootstrap = value; - - return Promise.resolve(''); - }) - .catch((err) => { - return Promise.reject('连接失败'); - }); - } - return Promise.resolve(''); - }, + validator: validators.bootstrapServers, }, ]} > @@ -374,36 +416,11 @@ const AccessClusters = (props: any): JSX.Element => { {extra.zooKeeperExtra}} - validateStatus={zookeeperErrorStatus ? 'error' : 'success'} + extra={{extra.zooKeeperExtra}} validateTrigger={'onBlur'} rules={[ { - required: false, - validator: async (rule: any, value: string) => { - if (!value) { - setZookeeperErrorStatus(false); - return Promise.resolve(''); - } - - if (value.length > 2000) { - return Promise.reject('Zookeeper长度限制在2000字符'); - } - - if (value && value !== lastFormItemValue.current.zookeeper) { - return connectTest() - .then((res: any) => { - lastFormItemValue.current.zookeeper = value; - setZookeeperErrorStatus(false); - return Promise.resolve(''); - }) - .catch((err) => { - setZookeeperErrorStatus(true); - return Promise.reject('连接失败'); - }); - } - return Promise.resolve(''); - }, + validator: validators.zookeeper, }, ]} > @@ -412,142 +429,65 @@ const AccessClusters = (props: any): JSX.Element => { placeholder="请输入Zookeeper地址，例如：192.168.0.1:2181,192.168.0.2:2181,192.168.0.2:2181/ks-kafka" /> - - <> - - - - - - - - - - None - Password Authentication - - - {security === 'Password' ? ( - <> - { - if (!value) { - return Promise.reject('用户名不能为空'); - } - if (!new RegExp(regUsername).test(value)) { - return Promise.reject('仅支持大小写、下划线、短划线（-）'); - } - if (value.length > 128) { - return Promise.reject('用户名长度限制在1～128字符'); - } - return Promise.resolve(); - }, - }, - ]} - > - + +
+
+ + + + + + +
+ + + None + Password Authentication + - { - if (!value) { - return Promise.reject('密码不能为空'); - } - if (!new RegExp(regUsername).test(value)) { - return Promise.reject('密码只能由大小写、下划线、短划线（-）组成'); - } - if (value.length < 6 || value.length > 32) { - return Promise.reject('密码长度限制在6～32字符'); - } - return Promise.resolve(); - }, - }, - ]} - > - + + {({ getFieldValue }) => { + return getFieldValue('openSSL') ? ( +
+ +
+ + + + + + +
+
+ ) : null; + }} - - ) : null} +
+ {extra.versionExtra}} - validateStatus={isLowVersion ? 'error' : 'success'} rules={[ { required: true, - validator: async (rule: any, value: any) => { - if (!value) { - setIsLowVersion(true); - return Promise.reject('版本号不能为空'); - } - // 检测版本号小于2.8.0，如果没有填zookeeper信息，才会提示 - const zookeeper = form.getFieldValue('zookeeper'); - if (value < lowKafkaVersion && !zookeeper) { - setIsLowVersion(true); - setExtra({ - ...extra, - versionExtra: intl.formatMessage({ id: 'access.cluster.low.version.tip' }), - }); - return Promise.resolve(); - } - setIsLowVersion(false); - return Promise.resolve(); - }, + validator: validators.kafkaVersion, }, ]} > @@ -565,29 +505,15 @@ const AccessClusters = (props: any): JSX.Element => { label="集群配置" rules={[ { - required: false, - message: '请输入集群配置', + validator: validators.clientProperties, }, - () => ({ - validator(_, value) { - try { - if (value) { - JSON.parse(value); - } - - return Promise.resolve(); - } catch (e) { - return Promise.reject(new Error('输入内容必须为 JSON')); - } - }, - }), ]} >
{ form.setFieldsValue({ clientProperties }); form.validateFields(['clientProperties']); @@ -621,20 +547,11 @@ const AccessClusters = (props: any): JSX.Element => { label="集群描述" rules={[ { - required: false, - validator: async (rule: any, value: string) => { - if (!value) { - return Promise.resolve(''); - } - if (value && value.length > 200) { - return Promise.reject('集群描述长度限制在200字符'); - } - return Promise.resolve(); - }, + validator: validators.description, }, ]} > - + diff --git a/km-console/packages/layout-clusters-fe/src/pages/MutliClusterPage/index.less b/km-console/packages/layout-clusters-fe/src/pages/MutliClusterPage/index.less index 3108c4c0..f459fd1c 100644 --- a/km-console/packages/layout-clusters-fe/src/pages/MutliClusterPage/index.less +++ b/km-console/packages/layout-clusters-fe/src/pages/MutliClusterPage/index.less @@ -656,43 +656,37 @@ color: @error-color; } } - .inline-item.dcloud-form-item { - display: -webkit-inline-box; - margin-right: 16px; - - &.adjust-height-style { - .dcloud-form-item-label { - padding: 0; - label { - height: 36px; - } - } - .dcloud-form-item-control { - &-input { - height: 36px; - } - } + .horizontal-form-container { + padding-left: 16px; + .inline-items { + display: flex; + justify-content: space-between; } - - &.max-width-66 { - .dcloud-form-item-control { - max-width: 66%; - } - } - - .dcloud-form-item-label { - margin-right: 12px; - - label { + .dcloud-form-item { + flex-direction: row; + align-items: center; + &-label { + padding: 0 12px 0 0; + font-size: 13px; font-family: @font-family; + color: #74788d; } } - } - - .no-item-control { - margin-bottom: 8px !important; - .dcloud-form-item-control { - display: none; + .metrics-form-item { + margin-top: 8px; + } + .user-info-form-items { + display: flex; + align-items: flex-start; + .user-info-label { + padding-top: 4px; + } + .inline-items { + flex: 0 0 80%; + .token-form-item { + margin-left: 16px; + } + } } } } diff --git a/km-console/packages/layout-clusters-fe/src/pages/SecurityACLs/index.less b/km-console/packages/layout-clusters-fe/src/pages/SecurityACLs/index.less index 54f77499..1d962f93 100644 --- a/km-console/packages/layout-clusters-fe/src/pages/SecurityACLs/index.less +++ b/km-console/packages/layout-clusters-fe/src/pages/SecurityACLs/index.less @@ -10,11 +10,6 @@ box-shadow: 0 2px 4px 0 rgba(0, 0, 0, 0.01), 0 3px 6px 3px rgba(0, 0, 0, 0.01), 0 2px 6px 0 rgba(0, 0, 0, 0.03); // border-radius: 12px; } - .operate-bar { - display: flex; - justify-content: space-between; - margin-bottom: 12px; - } } .acls-edit-drawer { diff --git a/km-console/packages/layout-clusters-fe/src/pages/SecurityACLs/index.tsx b/km-console/packages/layout-clusters-fe/src/pages/SecurityACLs/index.tsx index 3f51bc55..7d0c9446 100644 --- a/km-console/packages/layout-clusters-fe/src/pages/SecurityACLs/index.tsx +++ b/km-console/packages/layout-clusters-fe/src/pages/SecurityACLs/index.tsx @@ -1,7 +1,8 @@ import React, { useEffect, useRef, useState } from 'react'; -import { Button, Form, Input, Select, Modal, message, ProTable, AppContainer, DKSBreadcrumb, Utils } from 'knowdesign'; +import { Button, Form, Input, Select, Modal, message, ProTable, AppContainer, DKSBreadcrumb, Utils, IconFont, Divider } from 'knowdesign'; import ACLsCardBar from '@src/components/CardBar/ACLsCardBar'; import api from '@src/api'; +import { tableHeaderPrefix } from '@src/constants/common'; import { useParams } from 'react-router-dom'; import AddACLDrawer, { ACL_OPERATION, @@ -205,37 +206,45 @@ const SecurityACLs = (): JSX.Element => {
-
-
getACLs({ page: 1 })}> - - - - - - - - - - - +
+
+
getACLs()}> + +
+ +
getACLs({ page: 1 })}> + + + + + + + + + + +
+
+ +
{ const maxPos = chars.length; @@ -426,34 +427,41 @@ const SecurityUsers = (): JSX.Element => { ]} />
-
- { - setSearchKeywords(searchKeywordsInput); - }} - style={{ fontSize: '16px' }} - /> - } - placeholder="请输入 Kafka User" - value={searchKeywordsInput} - onPressEnter={(_) => { - setSearchKeywords(searchKeywordsInput); - }} - onChange={(e) => { - setSearchKeywordsInput(e.target.value); - }} - /> - +
+
+
getKafkaUserList()}> + +
+
+
+ { + setSearchKeywords(searchKeywordsInput); + }} + style={{ fontSize: '16px' }} + /> + } + placeholder="请输入 Kafka User" + value={searchKeywordsInput} + onPressEnter={(_) => { + setSearchKeywords(searchKeywordsInput); + }} + onChange={(e) => { + setSearchKeywordsInput(e.target.value); + }} + /> + +
div { width: 100%; height: 100%; @@ -73,65 +40,64 @@ justify-content: center; align-items: center; } - - .chart-box { - position: relative; - width: 100%; - height: 244px; - background: #f8f9fa; - border-radius: 8px; - - .expand-icon-box { - position: absolute; - z-index: 1000; - top: 14px; - right: 16px; - width: 24px; - height: 24px; - cursor: pointer; - font-size: 16px; - text-align: center; - border-radius: 50%; - transition: background-color 0.3s ease; - - .expand-icon { - color: #adb5bc; - line-height: 24px; - } - - &:hover { - background: rgba(33, 37, 41, 0.06); - .expand-icon { - color: #74788d; - } - } - } - } } .config-change-records-container { width: 240px; height: 100%; margin-left: 12px; - .cluster-container-border(); + .cluster-detail-container-border(); } } } +} - .chart-box-title { - padding: 18px 0 0 20px; - font-family: @font-family-bold; - line-height: 16px; - .name { - font-size: 14px; - color: #212529; +.cluster-detail-chart-box { + position: relative; + width: 100%; + height: 244px; + background: #f8f9fa; + border-radius: 8px; + + .expand-icon-box { + position: absolute; + z-index: 1000; + top: 14px; + right: 16px; + width: 24px; + height: 24px; + cursor: pointer; + font-size: 16px; + text-align: center; + border-radius: 50%; + transition: background-color 0.3s ease; + + .expand-icon { + color: #adb5bc; + line-height: 24px; } - .unit { - font-size: 12px; - color: #495057; - } - > span { - cursor: pointer; + + &:hover { + background: rgba(33, 37, 41, 0.06); + .expand-icon { + color: #74788d; + } } } } +.cluster-detail-chart-box-title { + padding: 18px 0 0 20px; + font-family: @font-family-bold; + line-height: 16px; + .name { + font-size: 14px; + color: #212529; + } + .unit { + font-size: 12px; + color: #495057; + } + > span { + cursor: pointer; + } +} diff --git a/km-console/packages/layout-clusters-fe/src/pages/SingleClusterDetail/DetailChart/index.tsx b/km-console/packages/layout-clusters-fe/src/pages/SingleClusterDetail/DetailChart/index.tsx index 6ecb7604..db241e4e 100644 --- a/km-console/packages/layout-clusters-fe/src/pages/SingleClusterDetail/DetailChart/index.tsx +++ b/km-console/packages/layout-clusters-fe/src/pages/SingleClusterDetail/DetailChart/index.tsx @@ -15,6 +15,7 @@ import { getDataNumberUnit, getUnit } from '@src/constants/chartConfig'; import SingleChartHeader, { KsHeaderOptions } from '@src/components/SingleChartHeader'; import { MAX_TIME_RANGE_WITH_SMALL_POINT_INTERVAL } from '@src/constants/common'; import RenderEmpty from '@src/components/RenderEmpty'; +import DragGroup from '@src/components/DragGroup'; type ChartFilterOptions = Omit; interface MetricInfo { @@ -279,7 +280,7 @@ const DetailChart = (props: { children: JSX.Element }): JSX.Element => { }, []); return ( -
+
{ {messagesInMetricData.data && ( <> -
+
{ @@ -354,14 +355,25 @@ const DetailChart = (props: { children: JSX.Element }): JSX.Element => {
- - {metricDataList.length ? ( - metricDataList.map((data: any, i: number) => { - const { metricName, metricUnit, metricLines } = data; - return ( -
-
-
+ {metricDataList.length ? ( +
+ 0, + onSortEnd: () => 0, + axis: 'xy', + useDragHandle: false, + }} + gridProps={{ + span: 12, + gutter: [16, 16], + }} + > + {metricDataList.map((data: any, i: number) => { + const { metricName, metricUnit, metricLines } = data; + return ( +
+
{ @@ -379,15 +391,6 @@ const DetailChart = (props: { children: JSX.Element }): JSX.Element => {
-
{ - setChartDetail(data); - setShowChartDetailModal(true); - }} - > - -
{ })} />
- - ); - }) - ) : chartLoading ? ( - <> - ) : ( - - )} - + ); + })} + +
+ ) : chartLoading ? ( + <> + ) : ( + + )}
@@ -421,35 +424,6 @@ const DetailChart = (props: { children: JSX.Element }): JSX.Element => {
{props.children}
- - {/* 图表详情 */} - setShowChartDetailModal(false)} - > -
-
setShowChartDetailModal(false)}> - -
- {chartDetail && ( - - )} -
- ); }; diff --git a/km-console/packages/layout-clusters-fe/src/pages/SingleClusterDetail/index.less b/km-console/packages/layout-clusters-fe/src/pages/SingleClusterDetail/index.less index 7baae7b8..3fa013de 100644 --- a/km-console/packages/layout-clusters-fe/src/pages/SingleClusterDetail/index.less +++ b/km-console/packages/layout-clusters-fe/src/pages/SingleClusterDetail/index.less @@ -231,9 +231,10 @@ } .chart-panel { - flex: 1; + flex: auto; margin-left: 12px; margin-right: 10px; + overflow: hidden; } .change-log-panel { diff --git a/km-console/packages/layout-clusters-fe/src/pages/SingleClusterDetail/index.tsx b/km-console/packages/layout-clusters-fe/src/pages/SingleClusterDetail/index.tsx index e4b4b2c6..59e679b3 100644 --- a/km-console/packages/layout-clusters-fe/src/pages/SingleClusterDetail/index.tsx +++ b/km-console/packages/layout-clusters-fe/src/pages/SingleClusterDetail/index.tsx @@ -21,11 +21,9 @@ const SingleClusterDetail = (): JSX.Element => {
-
- - - -
+ + +
diff --git a/km-console/packages/layout-clusters-fe/src/pages/TestingConsumer/config.tsx b/km-console/packages/layout-clusters-fe/src/pages/TestingConsumer/config.tsx index d27f8d4e..fc529e9d 100644 --- a/km-console/packages/layout-clusters-fe/src/pages/TestingConsumer/config.tsx +++ b/km-console/packages/layout-clusters-fe/src/pages/TestingConsumer/config.tsx @@ -268,6 +268,7 @@ export const getFormConfig = (topicMetaData: any, info = {} as any, partitionLis type: FormItemType.inputNumber, attrs: { min: 1, + max: 1000, }, invisible: !info?.needMsgNum, rules: [ diff --git a/km-console/packages/layout-clusters-fe/src/pages/TestingProduce/config.tsx b/km-console/packages/layout-clusters-fe/src/pages/TestingProduce/config.tsx index 81503271..97589f32 100644 --- a/km-console/packages/layout-clusters-fe/src/pages/TestingProduce/config.tsx +++ b/km-console/packages/layout-clusters-fe/src/pages/TestingProduce/config.tsx @@ -152,6 +152,7 @@ export const getFormConfig = (params: any) => { rules: [{ required: true, message: '请输入' }], attrs: { min: 0, + max: 1000, style: { width: 232 }, }, }, @@ -391,7 +392,7 @@ export const getTableColumns = () => { { title: 'time', dataIndex: 'costTimeUnitMs', - width: 60, + width: 100, }, ]; }; diff --git a/km-console/packages/layout-clusters-fe/src/pages/TopicDetail/Messages.tsx b/km-console/packages/layout-clusters-fe/src/pages/TopicDetail/Messages.tsx index 4771e0c7..1e1f6582 100644 --- a/km-console/packages/layout-clusters-fe/src/pages/TopicDetail/Messages.tsx +++ b/km-console/packages/layout-clusters-fe/src/pages/TopicDetail/Messages.tsx @@ -10,6 +10,7 @@ const defaultParams: any = { maxRecords: 100, pullTimeoutUnitMs: 5000, // filterPartitionId: 1, + filterOffsetReset: 0, }; const defaultpaPagination = { current: 1, @@ -29,12 +30,20 @@ const TopicMessages = (props: any) => { const [pagination, setPagination] = useState(defaultpaPagination); const [form] = Form.useForm(); + // 获取消息开始位置 + const offsetResetList = [ + { label: 'latest', value: 0 }, + { label: 'earliest', value: 1 }, + ]; + // 默认排序 const defaultSorter = { sortField: 'timestampUnitMs', sortType: 'desc', }; + const [sorter, setSorter] = useState(defaultSorter); + // 请求接口获取数据 const genData = async () => { if (urlParams?.clusterId === undefined || hashData?.topicName === undefined) return; @@ -49,7 +58,7 @@ const TopicMessages = (props: any) => { }); setPartitionIdList(newPartitionIdList || []); }); - request(Api.getTopicMessagesList(hashData?.topicName, urlParams?.clusterId), { data: { ...params, ...defaultSorter }, method: 'POST' }) + request(Api.getTopicMessagesList(hashData?.topicName, urlParams?.clusterId), { data: { ...params, ...sorter }, method: 'POST' }) .then((res: any) => { // setPagination({ // current: res.pagination?.pageNo, @@ -87,8 +96,15 @@ const TopicMessages = (props: any) => { history.push(`/cluster/${urlParams?.clusterId}/testing/consumer`); }; - const onTableChange = (pagination: any, filters: any, sorter: any) => { + const onTableChange = (pagination: any, filters: any, sorter: any, extra: any) => { setPagination(pagination); + // 只有排序事件时，触发重新请求后端数据 + if (extra.action === 'sort') { + setSorter({ + sortField: sorter.field || '', + sortType: sorter.order ? sorter.order.substring(0, sorter.order.indexOf('end')) : '', + }); + } // const asc = sorter?.order && sorter?.order === 'ascend' ? true : false; // const sortColumn = sorter.field && toLine(sorter.field); // genData({ pageNo: pagination.current, pageSize: pagination.pageSize, filters, asc, sortColumn, queryTerm: searchResult, ...allParams }); @@ -96,7 +112,7 @@ const TopicMessages = (props: any) => { useEffect(() => { props.positionType === 'Messages' && genData(); - }, [props, params]); + }, [props, params, sorter]); return ( <> @@ -119,6 +135,15 @@ const TopicMessages = (props: any) => {
+ + { style={{ margin: '12px 0 4px', padding: '7px 12px', background: '#FFF9E6' }} message={
- 此处展示Topic最近的100条messages，若想获取其他messages，可前往Produce&Consume进行操作 + 此处展示 Topic 最近的 100 条 messages。 + {process.env.BUSINESS_VERSION ? ( + + 若想获取其他 messages，可前往 Produce&Consume 进行操作 + + ) : ( + '' + )}
} type="warning" @@ -158,7 +190,7 @@ const TopicMessages = (props: any) => { showQueryForm={false} tableProps={{ showHeader: false, - rowKey: 'path', + rowKey: 'offset', loading: loading, columns: getTopicMessagesColmns(), dataSource: data, @@ -169,6 +201,7 @@ const TopicMessages = (props: any) => { bordered: false, onChange: onTableChange, scroll: { x: 'max-content' }, + sortDirections: ['descend', 'ascend', 'default'], }, }} /> diff --git a/km-console/packages/layout-clusters-fe/src/pages/TopicDetail/config.tsx b/km-console/packages/layout-clusters-fe/src/pages/TopicDetail/config.tsx index 5ed6cc73..03a93181 100644 --- a/km-console/packages/layout-clusters-fe/src/pages/TopicDetail/config.tsx +++ b/km-console/packages/layout-clusters-fe/src/pages/TopicDetail/config.tsx @@ -85,7 +85,8 @@ export const getTopicMessagesColmns = () => { title: 'Timestamp', dataIndex: 'timestampUnitMs', key: 'timestampUnitMs', - render: (t: number) => (t ? moment(t).format(timeFormat) : '-'), + sorter: true, + render: (t: number) => (t ? moment(t).format(timeFormat) + '.' + moment(t).millisecond() : '-'), }, { title: 'Key', diff --git a/km-console/packages/layout-clusters-fe/src/pages/TopicList/Create.tsx b/km-console/packages/layout-clusters-fe/src/pages/TopicList/Create.tsx index 218b0754..288b99c7 100644 --- a/km-console/packages/layout-clusters-fe/src/pages/TopicList/Create.tsx +++ b/km-console/packages/layout-clusters-fe/src/pages/TopicList/Create.tsx @@ -218,10 +218,10 @@ export default (props: any) => {
- + - +
diff --git a/km-console/packages/layout-clusters-fe/src/pages/TopicList/index.less b/km-console/packages/layout-clusters-fe/src/pages/TopicList/index.less index 669e85fb..453363c6 100644 --- a/km-console/packages/layout-clusters-fe/src/pages/TopicList/index.less +++ b/km-console/packages/layout-clusters-fe/src/pages/TopicList/index.less @@ -1,48 +1,9 @@ -.operation-bar { - // height: 60px; +.internal-switch { display: flex; - justify-content: space-between; align-items: center; - margin-bottom: 10px; - .left { - display: flex; - align-items: center; - .divider { - margin-left: 8px; - margin-right: 8px; - width: 1px; - height: 20px; - background-color: #ced4da; - } - .internal-switch { - display: flex; - align-items: center; - > span { - margin-left: 4px; - color: #74788d; - } - } - } - .right { - display: flex; - align-items: center; - .dcloud-form-item { - margin-bottom: 0; - } - .search-input { - width: 248px; - margin-right: 8px; - } - .search-input-short { - width: 120px; - margin-right: 8px; - } - .batch-btn { - margin-right: 8px; - } - .add-btn { - width: 117px; - } + > span { + margin-left: 4px; + color: #74788d; } } .operation-list { diff --git a/km-console/packages/layout-clusters-fe/src/pages/TopicList/index.tsx b/km-console/packages/layout-clusters-fe/src/pages/TopicList/index.tsx index 5bce3d6c..692da9c2 100644 --- a/km-console/packages/layout-clusters-fe/src/pages/TopicList/index.tsx +++ b/km-console/packages/layout-clusters-fe/src/pages/TopicList/index.tsx @@ -1,7 +1,7 @@ /* eslint-disable react/display-name */ import React, { useState, useEffect } from 'react'; import { useHistory, useParams } from 'react-router-dom'; -import { AppContainer, IconFont, Input, ProTable, Select, Switch, Tooltip, Utils, Dropdown, Menu, Button } from 'knowdesign'; +import { AppContainer, IconFont, Input, ProTable, Select, Switch, Tooltip, Utils, Dropdown, Menu, Button, Divider } from 'knowdesign'; import Create from './Create'; import './index.less'; import Api from '@src/api/index'; @@ -16,6 +16,7 @@ import SmallChart from '@src/components/SmallChart'; import ReplicaMove from '@src/components/TopicJob/ReplicaMove'; import { formatAssignSize } from '../Jobs/config'; import { DownOutlined } from '@ant-design/icons'; +import { tableHeaderPrefix } from '@src/constants/common'; const { Option } = Select; @@ -66,6 +67,7 @@ const AutoPage = (props: any) => { // params.sortField = sortObj.sortField; // params.sortType = sortObj.sortType || 'desc'; // } + setTopicListLoading(true); Utils.post(Api.getTopicsList(Number(routeParams.clusterId)), params) .then((data: any) => { setTopicListLoading(false); @@ -79,7 +81,6 @@ const AutoPage = (props: any) => { }); }; useEffect(() => { - setTopicListLoading(true); getTopicsList(); }, [sortObj, showInternalTopics, searchKeywords, pageIndex, pageSize]); @@ -285,26 +286,17 @@ const AutoPage = (props: any) => {
-
-
+
+
{/* 批量扩缩副本 */} {/* 批量迁移 */} - {/* */} - {/* - - */} - {/*
*/} + +
getTopicsList()}> + +
+
{ 展示系统Topic
-
+
p), - output: { - path: outPath, - publicPath: isProd ? process.env.PUBLIC_PATH + '/layout/' : '/', - filename: jsFileName, - chunkFilename: jsFileName, - library: 'layout', - libraryTarget: 'amd', - }, - devServer: { - host: 'localhost', - port: 8000, - hot: true, - open: true, - openPage: 'http://localhost:8000/', - inline: true, - historyApiFallback: true, - publicPath: `http://localhost:8000/`, - headers: { - 'cache-control': 'no-cache', - pragma: 'no-cache', - 'Access-Control-Allow-Origin': '*', - }, - proxy: { - '/ks-km/api/v3': { - changeOrigin: true, - target: 'http://localhost:8080/', - }, - '/logi-security/api/v1': { - changeOrigin: true, - target: 'http://localhost:8080/', - }, - }, - }, +const devMode = process.env.NODE_ENV === 'development'; +const commonConfig = require('./config/webpack.common'); +const devConfig = require('./config/webpack.dev'); +const prodConfig = require('./config/webpack.prod'); - resolve: { - alias: { - '@src': path.resolve(__dirname, 'src'), - }, - }, -}); +module.exports = merge(commonConfig, devMode ? devConfig : prodConfig); diff --git a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/AbstractZKWatcher.java b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/AbstractZKWatcher.java index 261aff0a..e43f1b40 100644 --- a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/AbstractZKWatcher.java +++ b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/AbstractZKWatcher.java @@ -5,7 +5,7 @@ import com.didiglobal.logi.log.LogFactory; import com.xiaojukeji.know.streaming.km.common.bean.entity.cluster.ClusterPhy; import com.xiaojukeji.know.streaming.km.common.exception.NotExistException; import com.xiaojukeji.know.streaming.km.persistence.kafka.KafkaAdminZKClient; -import com.xiaojukeji.know.streaming.km.persistence.zk.KafkaZKDAO; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.service.KafkaZKDAO; import kafka.zk.KafkaZkClient; import org.springframework.beans.factory.annotation.Autowired; diff --git a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/handler/AbstractZKHandler.java b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/handler/AbstractZKHandler.java index e2ed09d1..04a14e87 100644 --- a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/handler/AbstractZKHandler.java +++ b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/handler/AbstractZKHandler.java @@ -7,7 +7,7 @@ import com.xiaojukeji.know.streaming.km.common.utils.BackoffUtils; import com.xiaojukeji.know.streaming.km.core.service.change.record.KafkaChangeRecordService; import com.xiaojukeji.know.streaming.km.persistence.cache.LoadedClusterPhyCache; import com.xiaojukeji.know.streaming.km.persistence.kafka.KafkaAdminZKClient; -import com.xiaojukeji.know.streaming.km.persistence.zk.KafkaZKDAO; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.service.KafkaZKDAO; public abstract class AbstractZKHandler { diff --git a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/handler/BrokersNodeChangeHandler.java b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/handler/BrokersNodeChangeHandler.java index b7c93c2f..314195af 100644 --- a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/handler/BrokersNodeChangeHandler.java +++ b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/handler/BrokersNodeChangeHandler.java @@ -9,7 +9,7 @@ import com.xiaojukeji.know.streaming.km.common.enums.operaterecord.OperationEnum import com.xiaojukeji.know.streaming.km.common.utils.FutureUtil; import com.xiaojukeji.know.streaming.km.core.service.broker.BrokerService; import com.xiaojukeji.know.streaming.km.core.service.change.record.KafkaChangeRecordService; -import com.xiaojukeji.know.streaming.km.persistence.zk.KafkaZKDAO; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.service.KafkaZKDAO; import kafka.zk.BrokerIdsZNode; import kafka.zookeeper.ZNodeChildChangeHandler; diff --git a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/handler/ConfigNotificationNodeChangeHandler.java b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/handler/ConfigNotificationNodeChangeHandler.java index 91d91571..1e626632 100644 --- a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/handler/ConfigNotificationNodeChangeHandler.java +++ b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/handler/ConfigNotificationNodeChangeHandler.java @@ -8,11 +8,11 @@ import com.xiaojukeji.know.streaming.km.common.enums.KafkaConfigTypeEnum; import com.xiaojukeji.know.streaming.km.common.enums.operaterecord.OperationEnum; import com.xiaojukeji.know.streaming.km.common.utils.FutureUtil; import com.xiaojukeji.know.streaming.km.common.utils.Tuple; -import com.xiaojukeji.know.streaming.km.common.zookeeper.znode.config.ConfigChangeNotificationBaseData; -import com.xiaojukeji.know.streaming.km.common.zookeeper.znode.config.ConfigChangeNotificationDataV1; -import com.xiaojukeji.know.streaming.km.common.zookeeper.znode.config.ConfigChangeNotificationDataV2; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.znode.config.ConfigChangeNotificationBaseData; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.znode.config.ConfigChangeNotificationDataV1; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.znode.config.ConfigChangeNotificationDataV2; import com.xiaojukeji.know.streaming.km.core.service.change.record.KafkaChangeRecordService; -import com.xiaojukeji.know.streaming.km.persistence.zk.KafkaZKDAO; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.service.KafkaZKDAO; import kafka.zk.ConfigEntityChangeNotificationZNode; import kafka.zookeeper.ZNodeChildChangeHandler; import org.apache.zookeeper.data.Stat; diff --git a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/handler/ControllerNodeChangeHandler.java b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/handler/ControllerNodeChangeHandler.java index 904b7d72..b671c4a3 100644 --- a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/handler/ControllerNodeChangeHandler.java +++ b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/handler/ControllerNodeChangeHandler.java @@ -11,7 +11,7 @@ import com.xiaojukeji.know.streaming.km.common.utils.BackoffUtils; import com.xiaojukeji.know.streaming.km.common.utils.FutureUtil; import com.xiaojukeji.know.streaming.km.core.service.change.record.KafkaChangeRecordService; import com.xiaojukeji.know.streaming.km.core.service.kafkacontroller.KafkaControllerService; -import com.xiaojukeji.know.streaming.km.persistence.zk.KafkaZKDAO; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.service.KafkaZKDAO; import kafka.zk.ControllerZNode; import kafka.zookeeper.ZNodeChangeHandler; diff --git a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/handler/TopicsNodeChangeHandler.java b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/handler/TopicsNodeChangeHandler.java index 31602632..88c01281 100644 --- a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/handler/TopicsNodeChangeHandler.java +++ b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/flusher/zk/handler/TopicsNodeChangeHandler.java @@ -9,7 +9,7 @@ import com.xiaojukeji.know.streaming.km.common.enums.operaterecord.OperationEnum import com.xiaojukeji.know.streaming.km.common.utils.FutureUtil; import com.xiaojukeji.know.streaming.km.core.service.change.record.KafkaChangeRecordService; import com.xiaojukeji.know.streaming.km.core.service.topic.TopicService; -import com.xiaojukeji.know.streaming.km.persistence.zk.KafkaZKDAO; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.service.KafkaZKDAO; import kafka.zk.TopicsZNode; import kafka.zookeeper.ZNodeChildChangeHandler; diff --git a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/broker/impl/BrokerServiceImpl.java b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/broker/impl/BrokerServiceImpl.java index dc702388..fbede23c 100644 --- a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/broker/impl/BrokerServiceImpl.java +++ b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/broker/impl/BrokerServiceImpl.java @@ -24,7 +24,6 @@ import com.xiaojukeji.know.streaming.km.common.exception.VCHandlerNotExistExcept import com.xiaojukeji.know.streaming.km.common.jmx.JmxConnectorWrap; import com.xiaojukeji.know.streaming.km.common.utils.ConvertUtil; import com.xiaojukeji.know.streaming.km.common.utils.ValidateUtils; -import com.xiaojukeji.know.streaming.km.common.zookeeper.znode.brokers.BrokerMetadata; import com.xiaojukeji.know.streaming.km.core.service.broker.BrokerService; import com.xiaojukeji.know.streaming.km.core.service.topic.TopicService; import com.xiaojukeji.know.streaming.km.core.service.version.BaseVersionControlService; @@ -32,8 +31,7 @@ import com.xiaojukeji.know.streaming.km.persistence.jmx.JmxDAO; import com.xiaojukeji.know.streaming.km.persistence.kafka.KafkaAdminClient; import com.xiaojukeji.know.streaming.km.persistence.kafka.KafkaJMXClient; import com.xiaojukeji.know.streaming.km.persistence.mysql.broker.BrokerDAO; -import com.xiaojukeji.know.streaming.km.persistence.zk.KafkaZKDAO; -import kafka.zk.BrokerIdZNode; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.service.KafkaZKDAO; import kafka.zk.BrokerIdsZNode; import org.apache.kafka.clients.admin.*; import org.apache.kafka.common.Node; @@ -310,9 +308,7 @@ public class BrokerServiceImpl extends BaseVersionControlService implements Brok List brokerIdList = kafkaZKDAO.getChildren(clusterPhy.getId(), BrokerIdsZNode.path(), false); for (String brokerId: brokerIdList) { - BrokerMetadata metadata = kafkaZKDAO.getData(clusterPhy.getId(), BrokerIdZNode.path(Integer.valueOf(brokerId)), BrokerMetadata.class); - BrokerMetadata.parseAndUpdateBrokerMetadata(metadata); - brokerList.add(Broker.buildFrom(clusterPhy.getId(), Integer.valueOf(brokerId), metadata)); + brokerList.add(kafkaZKDAO.getBrokerMetadata(clusterPhy.getId(), Integer.valueOf(brokerId))); } return Result.buildSuc(brokerList); diff --git a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/cluster/impl/ClusterMetricServiceImpl.java b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/cluster/impl/ClusterMetricServiceImpl.java index 075c53c2..9fdd9ec0 100644 --- a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/cluster/impl/ClusterMetricServiceImpl.java +++ b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/cluster/impl/ClusterMetricServiceImpl.java @@ -751,8 +751,8 @@ public class ClusterMetricServiceImpl extends BaseMetricService implements Clust private Result getMetricFromKafkaByTotalTopics(Long clusterId, String metric, String topicMetric){ List topics = topicService.listTopicsFromCacheFirst(clusterId); - float metricsSum = 0f; - for(Topic topic : topics){ + float sumMetricValue = 0f; + for(Topic topic : topics) { Result> ret = topicMetricService.collectTopicMetricsFromKafkaWithCacheFirst( clusterId, topic.getTopicName(), @@ -763,14 +763,15 @@ public class ClusterMetricServiceImpl extends BaseMetricService implements Clust continue; } - List topicMetrics = ret.getData(); - for (TopicMetrics metrics : topicMetrics) { - if(metrics.isBBrokerAgg()){ - metricsSum += Double.valueOf(metrics.getMetrics().get(topicMetric)); + for (TopicMetrics metrics : ret.getData()) { + if(metrics.isBBrokerAgg()) { + Float metricValue = metrics.getMetric(topicMetric); + sumMetricValue += (metricValue == null? 0f: metricValue); + break; } } } - return Result.buildSuc(initWithMetrics(clusterId, metric, metricsSum)); + return Result.buildSuc(initWithMetrics(clusterId, metric, sumMetricValue)); } } diff --git a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/cluster/impl/ClusterValidateServiceImpl.java b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/cluster/impl/ClusterValidateServiceImpl.java index 6dcd858e..ba72d2fe 100644 --- a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/cluster/impl/ClusterValidateServiceImpl.java +++ b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/cluster/impl/ClusterValidateServiceImpl.java @@ -13,8 +13,8 @@ import com.xiaojukeji.know.streaming.km.common.enums.valid.ValidateKafkaAddressE import com.xiaojukeji.know.streaming.km.common.utils.ValidateUtils; import com.xiaojukeji.know.streaming.km.core.service.cluster.ClusterValidateService; import com.xiaojukeji.know.streaming.km.persistence.jmx.JmxDAO; -import com.xiaojukeji.know.streaming.km.persistence.zk.KafkaZKDAO; -import com.xiaojukeji.know.streaming.km.persistence.zk.impl.KafkaZKDAOImpl; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.service.KafkaZKDAO; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.service.impl.KafkaZKDAOImpl; import kafka.server.KafkaConfig; import lombok.extern.slf4j.Slf4j; import org.apache.kafka.clients.admin.*; diff --git a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/kafkacontroller/impl/KafkaControllerServiceImpl.java b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/kafkacontroller/impl/KafkaControllerServiceImpl.java index 1fb3f488..8048eabe 100644 --- a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/kafkacontroller/impl/KafkaControllerServiceImpl.java +++ b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/kafkacontroller/impl/KafkaControllerServiceImpl.java @@ -19,7 +19,7 @@ import com.xiaojukeji.know.streaming.km.core.service.broker.BrokerService; import com.xiaojukeji.know.streaming.km.core.service.kafkacontroller.KafkaControllerService; import com.xiaojukeji.know.streaming.km.persistence.kafka.KafkaAdminClient; import com.xiaojukeji.know.streaming.km.persistence.mysql.kafkacontroller.KafkaControllerDAO; -import com.xiaojukeji.know.streaming.km.persistence.zk.KafkaZKDAO; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.service.KafkaZKDAO; import org.apache.kafka.clients.admin.*; import org.apache.kafka.common.Node; import org.springframework.beans.factory.annotation.Autowired; diff --git a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/partition/impl/PartitionServiceImpl.java b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/partition/impl/PartitionServiceImpl.java index 13eedb41..1795e4d4 100644 --- a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/partition/impl/PartitionServiceImpl.java +++ b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/partition/impl/PartitionServiceImpl.java @@ -21,14 +21,14 @@ import com.xiaojukeji.know.streaming.km.common.exception.NotExistException; import com.xiaojukeji.know.streaming.km.common.exception.VCHandlerNotExistException; import com.xiaojukeji.know.streaming.km.common.utils.CommonUtils; import com.xiaojukeji.know.streaming.km.common.utils.ValidateUtils; -import com.xiaojukeji.know.streaming.km.common.zookeeper.znode.brokers.PartitionMap; -import com.xiaojukeji.know.streaming.km.common.zookeeper.znode.brokers.PartitionState; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.znode.brokers.PartitionMap; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.znode.brokers.PartitionState; import com.xiaojukeji.know.streaming.km.core.service.partition.PartitionService; import com.xiaojukeji.know.streaming.km.core.service.version.BaseVersionControlService; import com.xiaojukeji.know.streaming.km.persistence.kafka.KafkaAdminClient; import com.xiaojukeji.know.streaming.km.persistence.kafka.KafkaConsumerClient; import com.xiaojukeji.know.streaming.km.persistence.mysql.partition.PartitionDAO; -import com.xiaojukeji.know.streaming.km.persistence.zk.KafkaZKDAO; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.service.KafkaZKDAO; import kafka.zk.TopicPartitionStateZNode; import kafka.zk.TopicPartitionsZNode; import kafka.zk.TopicZNode; @@ -202,10 +202,22 @@ public class PartitionServiceImpl extends BaseVersionControlService implements P @Override public Result> getPartitionOffsetFromKafka(Long clusterPhyId, String topicName, OffsetSpec offsetSpec, Long timestamp) { Map topicPartitionOffsets = new HashMap<>(); - this.listPartitionByTopic(clusterPhyId, topicName) - .stream() + + List partitionList = this.listPartitionByTopic(clusterPhyId, topicName); + if (partitionList == null || partitionList.isEmpty()) { + // Topic不存在 + return Result.buildFromRSAndMsg(ResultStatus.NOT_EXIST, MsgConstant.getTopicNotExist(clusterPhyId, topicName)); + } + + partitionList.stream() + .filter(item -> !item.getLeaderBrokerId().equals(KafkaConstant.NO_LEADER)) .forEach(elem -> topicPartitionOffsets.put(new TopicPartition(topicName, elem.getPartitionId()), offsetSpec)); + if (topicPartitionOffsets.isEmpty()) { + // 所有分区no-leader + return Result.buildFromRSAndMsg(ResultStatus.OPERATION_FAILED, MsgConstant.getPartitionNoLeader(clusterPhyId, topicName)); + } + try { return (Result>) doVCHandler(clusterPhyId, PARTITION_OFFSET_GET, new PartitionOffsetParam(clusterPhyId, topicName, topicPartitionOffsets, timestamp)); } catch (VCHandlerNotExistException e) { diff --git a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/topic/impl/OpTopicServiceImpl.java b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/topic/impl/OpTopicServiceImpl.java index 7f289c88..7cd017f4 100644 --- a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/topic/impl/OpTopicServiceImpl.java +++ b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/topic/impl/OpTopicServiceImpl.java @@ -23,7 +23,7 @@ import com.xiaojukeji.know.streaming.km.core.service.topic.TopicService; import com.xiaojukeji.know.streaming.km.core.service.version.BaseVersionControlService; import com.xiaojukeji.know.streaming.km.persistence.kafka.KafkaAdminClient; import com.xiaojukeji.know.streaming.km.persistence.kafka.KafkaAdminZKClient; -import com.xiaojukeji.know.streaming.km.persistence.zk.KafkaZKDAO; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.service.KafkaZKDAO; import kafka.controller.ReplicaAssignment; import kafka.server.ConfigType; import kafka.zk.AdminZkClient; diff --git a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/topic/impl/TopicConfigServiceImpl.java b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/topic/impl/TopicConfigServiceImpl.java index 9aaadee5..09be0d43 100644 --- a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/topic/impl/TopicConfigServiceImpl.java +++ b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/topic/impl/TopicConfigServiceImpl.java @@ -30,7 +30,7 @@ import com.xiaojukeji.know.streaming.km.core.service.topic.TopicService; import com.xiaojukeji.know.streaming.km.core.service.version.BaseVersionControlService; import com.xiaojukeji.know.streaming.km.persistence.kafka.KafkaAdminClient; import com.xiaojukeji.know.streaming.km.persistence.kafka.KafkaAdminZKClient; -import com.xiaojukeji.know.streaming.km.persistence.zk.KafkaZKDAO; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.service.KafkaZKDAO; import kafka.server.ConfigType; import kafka.zk.AdminZkClient; import kafka.zk.KafkaZkClient; diff --git a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/topic/impl/TopicServiceImpl.java b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/topic/impl/TopicServiceImpl.java index bffabec8..e2870d9d 100644 --- a/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/topic/impl/TopicServiceImpl.java +++ b/km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/topic/impl/TopicServiceImpl.java @@ -23,7 +23,7 @@ import com.xiaojukeji.know.streaming.km.common.utils.ValidateUtils; import com.xiaojukeji.know.streaming.km.core.service.topic.TopicService; import com.xiaojukeji.know.streaming.km.persistence.kafka.KafkaAdminClient; import com.xiaojukeji.know.streaming.km.persistence.mysql.topic.TopicDAO; -import com.xiaojukeji.know.streaming.km.persistence.zk.KafkaZKDAO; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.service.KafkaZKDAO; import kafka.zk.TopicsZNode; import org.apache.kafka.clients.admin.*; import org.apache.kafka.common.TopicPartitionInfo; diff --git a/km-dist/helm/Chart.yaml b/km-dist/helm/Chart.yaml index bd9b7d98..4b764af4 100644 --- a/km-dist/helm/Chart.yaml +++ b/km-dist/helm/Chart.yaml @@ -4,13 +4,13 @@ description: knowstreaming-manager Helm chart type: application -version: 0.1.3 +version: 0.1.4 maintainers: - email: didicloud@didiglobal.com name: didicloud -appVersion: "3.0.0-beta.1" +appVersion: "3.0.0-beta.2" dependencies: - name: knowstreaming-web diff --git a/km-dist/helm/charts/elasticsearch/values.yaml b/km-dist/helm/charts/elasticsearch/values.yaml index fbbc599d..e325ec68 100644 --- a/km-dist/helm/charts/elasticsearch/values.yaml +++ b/km-dist/helm/charts/elasticsearch/values.yaml @@ -173,8 +173,8 @@ antiAffinityTopologyKey: "kubernetes.io/hostname" # Hard means that by default pods will only be scheduled if there are enough nodes for them # and that they will never end up on the same node. Setting this to soft will do this "best effort" -antiAffinity: "hard" - +antiAffinity: "" +#antiAffinity: "hard" # This is the node affinity settings as defined in # https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#node-affinity-beta-feature nodeAffinity: {} diff --git a/km-dist/helm/charts/ksmysql/templates/statefulset.yaml b/km-dist/helm/charts/ksmysql/templates/statefulset.yaml index 15ab2858..a536f2f8 100644 --- a/km-dist/helm/charts/ksmysql/templates/statefulset.yaml +++ b/km-dist/helm/charts/ksmysql/templates/statefulset.yaml @@ -21,7 +21,7 @@ spec: {{- include "ksmysql.selectorLabels" . | nindent 8 }} spec: containers: - - image: knowstreaming/knowstreaming-mysql:0.1.0 + - image: knowstreaming/knowstreaming-mysql:0.2.0 name: {{ .Chart.Name }} env: - name: MYSQL_DATABASE diff --git a/km-dist/helm/templates/configmap.yaml b/km-dist/helm/templates/configmap.yaml index ae09f6b9..6395aff1 100644 --- a/km-dist/helm/templates/configmap.yaml +++ b/km-dist/helm/templates/configmap.yaml @@ -71,6 +71,7 @@ data: driver-class-name: org.mariadb.jdbc.Driver app-name: know-streaming resource-extend-bean-name: myResourceExtendImpl + login-extend-bean-name: logiSecurityDefaultLoginExtendImpl logging: config: classpath:logback-spring.xml @@ -85,11 +86,16 @@ data: queue-size: 10000 # 每个线程池队列大小 select-suitable-enable: true # 任务是否自动选择合适的线程池，非主要，可不修改 suitable-queue-size: 1000 # 线程池理想的队列大小，非主要，可不修改 - task: # 任务模块的配置 - heaven: # 采集任务配置 - thread-num: 20 # 采集任务线程池核心线程数 - queue-size: 1000 # 采集任务线程池队列大小 - + task: # 任务模块的配置 + metrics: # metrics采集任务配置 + thread-num: 18 # metrics采集任务线程池核心线程数 + queue-size: 180 # metrics采集任务线程池队列大小 + metadata: # metadata同步任务配置 + thread-num: 27 # metadata同步任务线程池核心线程数 + queue-size: 270 # metadata同步任务线程池队列大小 + common: # 剩余其他任务配置 + thread-num: 15 # 剩余其他任务线程池核心线程数 + queue-size: 150 # 剩余其他任务线程池队列大小 client-pool: @@ -99,17 +105,16 @@ data: max-total-client-num: 20 # 最大客户端数 borrow-timeout-unit-ms: 5000 # 租借超时时间，单位秒 + es: + client: {{ if .Values.elasticsearch.enabled }} - es.client.address: elasticsearch-master:9200 - #es.client.address: {{ .Release.Name }}-elasticsearch:9200 + address: elasticsearch-master:9200 {{- else }} - es.client.address: {{ .Values.elasticsearch.esClientAddress }}:{{ .Values.elasticsearch.esProt }} + address: {{ .Values.elasticsearch.esClientAddress }}:{{ .Values.elasticsearch.esProt }} {{- end }} - # es.client.pass: knowstreaming-manager - # 集群自动均衡相关配置 - cluster-balance: - ignored-topics: - time-second: 300 + client-cnt: 10 + io-thread-cnt: 2 + max-retry-cnt: 5 # 普罗米修斯指标导出相关配置 management: @@ -158,4 +163,3 @@ data: curl -s -o /dev/null -X PUT http://${esaddr}:${port}/ks_kafka_topic_metric${logdate} || \ exit 2 done - diff --git a/km-dist/helm/values.yaml b/km-dist/helm/values.yaml index 036f5cc8..2c422f61 100644 --- a/km-dist/helm/values.yaml +++ b/km-dist/helm/values.yaml @@ -3,7 +3,7 @@ replicaCount: 2 image: repository: knowstreaming/knowstreaming-manager pullPolicy: IfNotPresent - tag: "0.1.0" + tag: "0.2.0" imagePullSecrets: [] nameOverride: "" @@ -73,7 +73,7 @@ knowstreaming-web: image: repository: knowstreaming/knowstreaming-ui pullPolicy: IfNotPresent - tag: "0.1.0" + tag: "0.2.0" service: type: NodePort diff --git a/km-dist/init/sql/dml-logi.sql b/km-dist/init/sql/dml-logi.sql index 86eb9a69..6d6e8159 100644 --- a/km-dist/init/sql/dml-logi.sql +++ b/km-dist/init/sql/dml-logi.sql @@ -48,7 +48,7 @@ INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `l -- 初始化用户 ---INSERT INTO `logi_security_user` (`id`, `user_name`, `pw`, `real_name`, `is_delete`, `app_name`) VALUES ('1', 'admin', 'V1ZkU2RHRlhOSGxOUkVsNVdETjBRVlp0Y0V0T1IwWnlaVEZ6YWxGRVJrRkpNVEU1VTJwYVUySkhlRzlSU0RBOWUwQldha28wWVd0N1d5TkFNa0FqWFgxS05sSnNiR2hBZlE9PXtAVmpKNGFre1sjQDNAI119SjZSbGxoQH0=Mv{#cdRgJ45Lqx}3IubEW87!==', '系统管理员', '0', 'know-streaming'); +-- INSERT INTO `logi_security_user` (`id`, `user_name`, `pw`, `real_name`, `is_delete`, `app_name`) VALUES ('1', 'admin', 'V1ZkU2RHRlhOSGxOUkVsNVdETjBRVlp0Y0V0T1IwWnlaVEZ6YWxGRVJrRkpNVEU1VTJwYVUySkhlRzlSU0RBOWUwQldha28wWVd0N1d5TkFNa0FqWFgxS05sSnNiR2hBZlE9PXtAVmpKNGFre1sjQDNAI119SjZSbGxoQH0=Mv{#cdRgJ45Lqx}3IubEW87!==', '系统管理员', '0', 'know-streaming'); INSERT INTO `logi_security_user` (`id`, `user_name`, `pw`, `real_name`, `is_delete`, `app_name`) VALUES ('1', 'admin', 'V1ZkU2RHRlhOVGRSUmxweFUycFNhR0V6ZEdKSk1FRjRVVU5PWkdaVmJ6SlZiWGh6WVVWQ09YdEFWbXBLTkdGcmUxc2pRREpBSTExOVNqWlNiR3hvUUgwPXtAVmpKNGFre1sjQDNAI119SjZSbGxoQH0=Mv{#cdRgJ45Lqx}3IubEW87!==', '系统管理员', '0', 'know-streaming'); -- 初始化角色 @@ -96,4 +96,4 @@ INSERT INTO `logi_security_user_role` (`id`, `user_id`, `role_id`, `is_delete`, INSERT INTO `logi_security_config` (`value_group`,`value_name`,`value`,`edit`,`status`,`memo`,`is_delete`,`app_name`,`operator`) VALUES -('SECURITY.LOGIN','SECURITY.TRICK_USERS','[\n \"admin\"\n]',1,1,'允许跳过登录的用户',0,'know-streaming','admin'); \ No newline at end of file +('SECURITY.LOGIN','SECURITY.TRICK_USERS','[\n \"admin\"\n]',1,1,'允许跳过登录的用户',0,'know-streaming','admin'); diff --git a/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/KafkaJMXClient.java b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/KafkaJMXClient.java index 68d1011e..39ae1ebe 100644 --- a/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/KafkaJMXClient.java +++ b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/KafkaJMXClient.java @@ -191,6 +191,10 @@ public class KafkaJMXClient extends AbstractClusterLoadedChangedHandler { lambdaQueryWrapper.eq(BrokerPO::getStatus, Constant.ALIVE); BrokerPO brokerPO = brokerDAO.selectOne(lambdaQueryWrapper); + if (brokerPO == null) { + return null; + } + return Broker.buildFrom(brokerPO); } } diff --git a/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/package-info.java b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/package-info.java new file mode 100644 index 00000000..88139db3 --- /dev/null +++ b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/package-info.java @@ -0,0 +1,4 @@ +/** + * 读取Kafka在ZK中存储的数据的包 + */ +package com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper; \ No newline at end of file diff --git a/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/zk/KafkaZKDAO.java b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/service/KafkaZKDAO.java similarity index 97% rename from km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/zk/KafkaZKDAO.java rename to km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/service/KafkaZKDAO.java index 3e00e558..7a7d4b76 100644 --- a/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/zk/KafkaZKDAO.java +++ b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/service/KafkaZKDAO.java @@ -1,4 +1,4 @@ -package com.xiaojukeji.know.streaming.km.persistence.zk; +package com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.service; import com.xiaojukeji.know.streaming.km.common.bean.entity.broker.Broker; import com.xiaojukeji.know.streaming.km.common.bean.entity.kafkacontroller.KafkaController; diff --git a/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/zk/impl/KafkaZKDAOImpl.java b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/service/impl/KafkaZKDAOImpl.java similarity index 90% rename from km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/zk/impl/KafkaZKDAOImpl.java rename to km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/service/impl/KafkaZKDAOImpl.java index 61a7bad0..82cb8130 100644 --- a/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/zk/impl/KafkaZKDAOImpl.java +++ b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/service/impl/KafkaZKDAOImpl.java @@ -1,4 +1,4 @@ -package com.xiaojukeji.know.streaming.km.persistence.zk.impl; +package com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.service.impl; import com.alibaba.fastjson.JSON; import com.didiglobal.logi.log.ILog; @@ -11,11 +11,11 @@ import com.xiaojukeji.know.streaming.km.common.enums.topic.TopicTypeEnum; import com.xiaojukeji.know.streaming.km.common.exception.AdminOperateException; import com.xiaojukeji.know.streaming.km.common.exception.NotExistException; import com.xiaojukeji.know.streaming.km.common.utils.Tuple; -import com.xiaojukeji.know.streaming.km.common.zookeeper.znode.ControllerData; -import com.xiaojukeji.know.streaming.km.common.zookeeper.znode.brokers.BrokerMetadata; -import com.xiaojukeji.know.streaming.km.common.zookeeper.znode.brokers.PartitionMap; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.znode.ControllerData; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.znode.brokers.BrokerMetadata; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.znode.brokers.PartitionMap; import com.xiaojukeji.know.streaming.km.persistence.kafka.KafkaAdminZKClient; -import com.xiaojukeji.know.streaming.km.persistence.zk.KafkaZKDAO; +import com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.service.KafkaZKDAO; import kafka.utils.Json; import kafka.zk.*; import kafka.zookeeper.AsyncResponse; @@ -46,14 +46,14 @@ public class KafkaZKDAOImpl implements KafkaZKDAO { public Broker getBrokerMetadata(String zkAddress) throws KeeperException.NoNodeException, AdminOperateException { ZooKeeper zooKeeper = null; try { - zooKeeper = new ZooKeeper(zkAddress, 1000, watchedEvent -> logger.info(" receive event : " + watchedEvent.getType().name())); + zooKeeper = new ZooKeeper(zkAddress, 3000, watchedEvent -> logger.info(" receive event : " + watchedEvent.getType().name())); List brokerIdList = this.getChildren(zooKeeper, BrokerIdsZNode.path()); if (brokerIdList == null || brokerIdList.isEmpty()) { return null; } BrokerMetadata brokerMetadata = this.getData(zooKeeper, BrokerIdZNode.path(Integer.parseInt(brokerIdList.get(0))), false, BrokerMetadata.class); - return Broker.buildFrom(null, Integer.valueOf(brokerIdList.get(0)), brokerMetadata); + return this.convert2Broker(null, Integer.valueOf(brokerIdList.get(0)), brokerMetadata); } catch (KeeperException.NoNodeException nne) { logger.warn("method=getBrokerMetadata||zkAddress={}||errMsg=exception", zkAddress, nne); throw nne; @@ -79,7 +79,7 @@ public class KafkaZKDAOImpl implements KafkaZKDAO { try { BrokerMetadata metadata = this.getData(kafkaZkClient.currentZooKeeper(), BrokerIdZNode.path(brokerId), false, BrokerMetadata.class); BrokerMetadata.parseAndUpdateBrokerMetadata(metadata); - return Broker.buildFrom(clusterPhyId, brokerId, metadata); + return this.convert2Broker(clusterPhyId, brokerId, metadata); } catch (KeeperException ke) { logger.error("method=getBrokerMetadata||clusterPhyId={}||brokerId={}||errMsg=exception", clusterPhyId, brokerId, ke); throw ke; @@ -269,4 +269,18 @@ public class KafkaZKDAOImpl implements KafkaZKDAO { byte[] bytes = zooKeeper.getData(path, addWatch, null); return JSON.parseObject(bytes, clazz); } + + private Broker convert2Broker(Long clusterPhyId, Integer brokerId, BrokerMetadata brokerMetadata) { + Broker metadata = new Broker(); + metadata.setClusterPhyId(clusterPhyId); + metadata.setBrokerId(brokerId); + metadata.setHost(brokerMetadata.getHost()); + metadata.setPort(brokerMetadata.getPort()); + metadata.setJmxPort(brokerMetadata.getJmxPort()); + metadata.setStartTimestamp(brokerMetadata.getTimestamp()); + metadata.setRack(brokerMetadata.getRack()); + metadata.setStatus(1); + metadata.setEndpointMap(brokerMetadata.getEndpointMap()); + return metadata; + } } diff --git a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/ControllerData.java b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/ControllerData.java similarity index 81% rename from km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/ControllerData.java rename to km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/ControllerData.java index f69c6862..afc7f55b 100644 --- a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/ControllerData.java +++ b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/ControllerData.java @@ -1,4 +1,4 @@ -package com.xiaojukeji.know.streaming.km.common.zookeeper.znode; +package com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.znode; import lombok.AllArgsConstructor; import lombok.Data; diff --git a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/brokers/BrokerMetadata.java b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/brokers/BrokerMetadata.java similarity index 97% rename from km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/brokers/BrokerMetadata.java rename to km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/brokers/BrokerMetadata.java index 480867af..3b252c5f 100644 --- a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/brokers/BrokerMetadata.java +++ b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/brokers/BrokerMetadata.java @@ -1,4 +1,4 @@ -package com.xiaojukeji.know.streaming.km.common.zookeeper.znode.brokers; +package com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.znode.brokers; import com.fasterxml.jackson.annotation.JsonIgnore; import com.fasterxml.jackson.annotation.JsonIgnoreProperties; diff --git a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/brokers/PartitionMap.java b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/brokers/PartitionMap.java similarity index 91% rename from km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/brokers/PartitionMap.java rename to km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/brokers/PartitionMap.java index bf1fbd1a..4bc36cac 100644 --- a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/brokers/PartitionMap.java +++ b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/brokers/PartitionMap.java @@ -1,4 +1,4 @@ -package com.xiaojukeji.know.streaming.km.common.zookeeper.znode.brokers; +package com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.znode.brokers; import lombok.AllArgsConstructor; import lombok.Data; diff --git a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/brokers/PartitionState.java b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/brokers/PartitionState.java similarity index 93% rename from km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/brokers/PartitionState.java rename to km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/brokers/PartitionState.java index 60ae4307..47be5cb9 100644 --- a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/brokers/PartitionState.java +++ b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/brokers/PartitionState.java @@ -1,4 +1,4 @@ -package com.xiaojukeji.know.streaming.km.common.zookeeper.znode.brokers; +package com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.znode.brokers; import com.fasterxml.jackson.annotation.JsonProperty; import lombok.AllArgsConstructor; diff --git a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/brokers/TopicMetadata.java b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/brokers/TopicMetadata.java similarity index 91% rename from km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/brokers/TopicMetadata.java rename to km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/brokers/TopicMetadata.java index 803a5e29..f84c8fcf 100644 --- a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/brokers/TopicMetadata.java +++ b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/brokers/TopicMetadata.java @@ -1,4 +1,4 @@ -package com.xiaojukeji.know.streaming.km.common.zookeeper.znode.brokers; +package com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.znode.brokers; import lombok.AllArgsConstructor; import lombok.Data; diff --git a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/config/ConfigChangeNotificationBaseData.java b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/config/ConfigChangeNotificationBaseData.java similarity index 77% rename from km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/config/ConfigChangeNotificationBaseData.java rename to km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/config/ConfigChangeNotificationBaseData.java index 86a3abe9..09ffee10 100644 --- a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/config/ConfigChangeNotificationBaseData.java +++ b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/config/ConfigChangeNotificationBaseData.java @@ -1,4 +1,4 @@ -package com.xiaojukeji.know.streaming.km.common.zookeeper.znode.config; +package com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.znode.config; import lombok.AllArgsConstructor; import lombok.Data; diff --git a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/config/ConfigChangeNotificationDataV1.java b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/config/ConfigChangeNotificationDataV1.java similarity index 86% rename from km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/config/ConfigChangeNotificationDataV1.java rename to km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/config/ConfigChangeNotificationDataV1.java index 75598e65..1853b940 100644 --- a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/config/ConfigChangeNotificationDataV1.java +++ b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/config/ConfigChangeNotificationDataV1.java @@ -1,4 +1,4 @@ -package com.xiaojukeji.know.streaming.km.common.zookeeper.znode.config; +package com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.znode.config; import com.fasterxml.jackson.annotation.JsonProperty; import lombok.AllArgsConstructor; diff --git a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/config/ConfigChangeNotificationDataV2.java b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/config/ConfigChangeNotificationDataV2.java similarity index 90% rename from km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/config/ConfigChangeNotificationDataV2.java rename to km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/config/ConfigChangeNotificationDataV2.java index 6b0d8806..5e6024fa 100644 --- a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/config/ConfigChangeNotificationDataV2.java +++ b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/config/ConfigChangeNotificationDataV2.java @@ -1,4 +1,4 @@ -package com.xiaojukeji.know.streaming.km.common.zookeeper.znode.config; +package com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.znode.config; import com.fasterxml.jackson.annotation.JsonProperty; import lombok.AllArgsConstructor; diff --git a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/config/ConfigNodeData.java b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/config/ConfigNodeData.java similarity index 81% rename from km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/config/ConfigNodeData.java rename to km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/config/ConfigNodeData.java index 13132b4f..287912dc 100644 --- a/km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/zookeeper/znode/config/ConfigNodeData.java +++ b/km-persistence/src/main/java/com/xiaojukeji/know/streaming/km/persistence/kafka/zookeeper/znode/config/ConfigNodeData.java @@ -1,4 +1,4 @@ -package com.xiaojukeji.know.streaming.km.common.zookeeper.znode.config; +package com.xiaojukeji.know.streaming.km.persistence.kafka.zookeeper.znode.config; import lombok.AllArgsConstructor; import lombok.Data; diff --git a/km-rest/src/main/java/com/xiaojukeji/know/streaming/km/rest/api/v3/topic/TopicStateController.java b/km-rest/src/main/java/com/xiaojukeji/know/streaming/km/rest/api/v3/topic/TopicStateController.java index 52a279e9..d1e09e66 100644 --- a/km-rest/src/main/java/com/xiaojukeji/know/streaming/km/rest/api/v3/topic/TopicStateController.java +++ b/km-rest/src/main/java/com/xiaojukeji/know/streaming/km/rest/api/v3/topic/TopicStateController.java @@ -3,6 +3,7 @@ package com.xiaojukeji.know.streaming.km.rest.api.v3.topic; import com.xiaojukeji.know.streaming.km.biz.topic.TopicStateManager; import com.xiaojukeji.know.streaming.km.common.bean.dto.metrices.MetricDTO; import com.xiaojukeji.know.streaming.km.common.bean.dto.pagination.PaginationBaseDTO; +import com.xiaojukeji.know.streaming.km.common.bean.dto.pagination.PaginationSortDTO; import com.xiaojukeji.know.streaming.km.common.bean.dto.topic.TopicRecordDTO; import com.xiaojukeji.know.streaming.km.common.bean.entity.metrics.BaseMetrics; import com.xiaojukeji.know.streaming.km.common.bean.entity.result.PaginationResult; diff --git a/km-task/pom.xml b/km-task/pom.xml index 502d806d..d07b37b3 100644 --- a/km-task/pom.xml +++ b/km-task/pom.xml @@ -43,7 +43,21 @@ io.github.zqrferrari logi-job-spring-boot-starter + + + oshi-core + com.github.oshi + + + + + + com.github.oshi + oshi-core + 5.6.1 + + io.github.zqrferrari logi-security-spring-boot-starter diff --git a/pom.xml b/pom.xml index 0a21cf83..619f24c7 100644 --- a/pom.xml +++ b/pom.xml @@ -15,7 +15,7 @@ - 3.0.0-beta.2 + 3.0.0-beta.3 8 8 @@ -230,6 +230,19 @@ io.github.zqrferrari logi-job-spring-boot-starter 1.0.23 + + + oshi-core + com.github.oshi + + + + + + + com.github.oshi + oshi-core + 5.6.1