8x48/KnowStreaming
1
0
Fork 0
mirror of https://github.com/didi/KnowStreaming.git synced 2025-12-30 08:02:11 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

Merge pull request #560 from didi/dev

Browse Source 
合并开发分支
This commit is contained in:
EricZeng
2022-09-07 14:20:04 +08:00
committed by GitHub
parent 1aa759e5be 7a3d15525c
commit be60ae8399
15 changed files with 178 additions and 86 deletions
Download Patch File Download Diff File Expand all files Collapse all files

61
docs/dev_guide/解决连接JMX失败.md
View File

@@ -1,19 +1,14 @@
![Logo](https://user-images.githubusercontent.com/71620349/185368586-aed82d30-1534-453d-86ff-ecfa9d0f35bd.png)
## JMX-连接失败问题解决
- [JMX-连接失败问题解决](#jmx-连接失败问题解决)
- [1、问题&说明](#1问题说明)
- [2、解决方法](#2解决方法)
- [3、解决方法 —— 认证的JMX](#3解决方法--认证的jmx)
集群正常接入Logi-KafkaManager之后，即可以看到集群的Broker列表，此时如果查看不了Topic的实时流量，或者是Broker的实时流量信息时，那么大概率就是JMX连接的问题了。
集群正常接入`KnowStreaming`之后，即可以看到集群的Broker列表，此时如果查看不了Topic的实时流量，或者是Broker的实时流量信息时，那么大概率就是`JMX`连接的问题了。
下面我们按照步骤来一步一步的检查。
### 1、问题&说明
### 1、问题说明
**类型一：JMX配置未开启**
@@ -43,6 +38,26 @@ java.rmi.ConnectException: Connection refused to host: 192.168.0.1; nested excep
java.rmi.ConnectException: Connection refused to host: 127.0.0.1;; nested exception is:
```
**类型三：连接特定IP**
Broker 配置了内外网，而JMX在配置时，可能配置了内网IP或者外网IP，此时 `KnowStreaming` 需要连接到特定网络的IP才可以进行访问。
比如：
Broker在ZK的存储结构如下所示，我们期望连接到 `endpoints` 中标记为 `INTERNAL` 的地址，但是 `KnowStreaming` 却连接了 `EXTERNAL` 的地址，此时可以看 `4、解决方法 —— JMX连接特定网络` 进行解决。
```json
{
"listener_security_protocol_map": {"EXTERNAL":"SASL_PLAINTEXT","INTERNAL":"SASL_PLAINTEXT"},
"endpoints": ["EXTERNAL://192.168.0.1:7092","INTERNAL://192.168.0.2:7093"],
"jmx_port": 8099,
"host": "192.168.0.1",
"timestamp": "1627289710439",
"port": -1,
"version": 4
}
```
### 2、解决方法
这里仅介绍一下比较通用的解决方式，如若有更好的方式，欢迎大家指导告知一下。
@@ -76,26 +91,36 @@ fi
如果您是直接看的这个部分，建议先看一下上一节：`2、解决方法`以确保`JMX`的配置没有问题了。
在JMX的配置等都没有问题的情况下，如果是因为认证的原因导致连接不了的，此时可以使用下面介绍的方法进行解决。
在`JMX`的配置等都没有问题的情况下，如果是因为认证的原因导致连接不了的，可以在集群接入界面配置你的`JMX`认证信息。
**当前这块后端刚刚开发完成，可能还不够完善，有问题随时沟通。**
<img src='http://img-ys011.didistatic.com/static/dc2img/do1_EUU352qMEX1Jdp7pxizp' width=350>
`Logi-KafkaManager 2.2.0+`之后的版本后端已经支持`JMX`认证方式的连接，但是还没有界面，此时我们可以往`cluster`表的`jmx_properties`字段写入`JMX`的认证信息。
这个数据是`json`格式的字符串，例子如下所示：
### 4、解决方法 —— JMX连接特定网络
可以手动往`ks_km_physical_cluster`表的`jmx_properties`字段增加一个`useWhichEndpoint`字段，从而控制 `KnowStreaming` 连接到特定的JMX IP及PORT。
`jmx_properties`格式：
```json
{
"maxConn": 10, # KM对单台Broker的最大JMX连接数
"username": "xxxxx", # 用户名
"password": "xxxx", # 密码
"maxConn": 100, # KM对单台Broker的最大JMX连接数
"username": "xxxxx", # 用户名，可以不填写
"password": "xxxx", # 密码，可以不填写
"openSSL": true, # 开启SSL, true表示开启ssl, false表示关闭
"useWhichEndpoint": "EXTERNAL" #指定要连接的网络名称，填写EXTERNAL就是连接endpoints里面的EXTERNAL地址
}
```
&nbsp;
SQL的例子：
SQL例子：
```sql
UPDATE cluster SET jmx_properties='{ "maxConn": 10, "username": "xxxxx", "password": "xxxx", "openSSL": false }' where id={xxx};
```
UPDATE ks_km_physical_cluster SET jmx_properties='{ "maxConn": 10, "username": "xxxxx", "password": "xxxx", "openSSL": false , "useWhichEndpoint": "xxx"}' where id={xxx};
```
注意：
+ 目前此功能只支持采用 `ZK` 做分布式协调的kafka集群。

26
docs/install_guide/版本升级手册.md
View File

@@ -4,7 +4,31 @@
### 6.2.0、升级至 `master` 版本
暂无
**SQL变更**
```sql
-- 多集群管理权限2022-09-06新增
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('2000', '多集群管理查看', '1593', '1', '2', '多集群管理查看', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('2002', 'Topic-迁移副本', '1593', '1', '2', 'Topic-迁移副本', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('2004', 'Topic-扩缩副本', '1593', '1', '2', 'Topic-扩缩副本', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('2006', 'Cluster-LoadReBalance-周期均衡', '1593', '1', '2', 'Cluster-LoadReBalance-周期均衡', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('2008', 'Cluster-LoadReBalance-立即均衡', '1593', '1', '2', 'Cluster-LoadReBalance-立即均衡', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('2010', 'Cluster-LoadReBalance-设置集群规格', '1593', '1', '2', 'Cluster-LoadReBalance-设置集群规格', '0', 'know-streaming');
-- 系统管理权限2022-09-06新增
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('3000', '系统管理查看', '1595', '1', '2', '系统管理查看', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '2000', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '2002', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '2004', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '2006', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '2008', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '2010', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '3000', '0', 'know-streaming');
```
---

17
km-biz/src/main/java/com/xiaojukeji/know/streaming/km/biz/cluster/impl/ClusterBrokersManagerImpl.java
View File

@@ -14,6 +14,7 @@ import com.xiaojukeji.know.streaming.km.common.bean.entity.topic.Topic;
import com.xiaojukeji.know.streaming.km.common.bean.vo.cluster.res.ClusterBrokersOverviewVO;
import com.xiaojukeji.know.streaming.km.common.bean.vo.cluster.res.ClusterBrokersStateVO;
import com.xiaojukeji.know.streaming.km.common.bean.vo.kafkacontroller.KafkaControllerVO;
import com.xiaojukeji.know.streaming.km.common.constant.KafkaConstant;
import com.xiaojukeji.know.streaming.km.common.enums.SortTypeEnum;
import com.xiaojukeji.know.streaming.km.common.utils.PaginationMetricsUtil;
import com.xiaojukeji.know.streaming.km.common.utils.PaginationUtil;
@@ -71,6 +72,9 @@ public class ClusterBrokersManagerImpl implements ClusterBrokersManager {
Topic groupTopic = topicService.getTopic(clusterPhyId, org.apache.kafka.common.internals.Topic.GROUP_METADATA_TOPIC_NAME);
Topic transactionTopic = topicService.getTopic(clusterPhyId, org.apache.kafka.common.internals.Topic.TRANSACTION_STATE_TOPIC_NAME);
//获取controller信息
KafkaController kafkaController = kafkaControllerService.getKafkaControllerFromDB(clusterPhyId);
// 格式转换
return PaginationResult.buildSuc(
this.convert2ClusterBrokersOverviewVOList(
@@ -78,7 +82,8 @@ public class ClusterBrokersManagerImpl implements ClusterBrokersManager {
brokerList,
metricsResult.getData(),
groupTopic,
transactionTopic
transactionTopic,
kafkaController
),
paginationResult
);
@@ -159,7 +164,8 @@ public class ClusterBrokersManagerImpl implements ClusterBrokersManager {
List<Broker> brokerList,
List<BrokerMetrics> metricsList,
Topic groupTopic,
Topic transactionTopic) {
Topic transactionTopic,
KafkaController kafkaController) {
Map<Integer, BrokerMetrics> metricsMap = metricsList == null? new HashMap<>(): metricsList.stream().collect(Collectors.toMap(BrokerMetrics::getBrokerId, Function.identity()));
Map<Integer, Broker> brokerMap = brokerList == null? new HashMap<>(): brokerList.stream().collect(Collectors.toMap(Broker::getBrokerId, Function.identity()));
@@ -169,12 +175,12 @@ public class ClusterBrokersManagerImpl implements ClusterBrokersManager {
Broker broker = brokerMap.get(brokerId);
BrokerMetrics brokerMetrics = metricsMap.get(brokerId);
voList.add(this.convert2ClusterBrokersOverviewVO(brokerId, broker, brokerMetrics, groupTopic, transactionTopic));
voList.add(this.convert2ClusterBrokersOverviewVO(brokerId, broker, brokerMetrics, groupTopic, transactionTopic, kafkaController));
}
return voList;
}
private ClusterBrokersOverviewVO convert2ClusterBrokersOverviewVO(Integer brokerId, Broker broker, BrokerMetrics brokerMetrics, Topic groupTopic, Topic transactionTopic) {
private ClusterBrokersOverviewVO convert2ClusterBrokersOverviewVO(Integer brokerId, Broker broker, BrokerMetrics brokerMetrics, Topic groupTopic, Topic transactionTopic, KafkaController kafkaController) {
ClusterBrokersOverviewVO clusterBrokersOverviewVO = new ClusterBrokersOverviewVO();
clusterBrokersOverviewVO.setBrokerId(brokerId);
if (broker != null) {
@@ -192,6 +198,9 @@ public class ClusterBrokersManagerImpl implements ClusterBrokersManager {
if (transactionTopic != null && transactionTopic.getBrokerIdSet().contains(brokerId)) {
clusterBrokersOverviewVO.getKafkaRoleList().add(transactionTopic.getTopicName());
}
if (kafkaController != null && kafkaController.getBrokerId().equals(brokerId)) {
clusterBrokersOverviewVO.getKafkaRoleList().add(KafkaConstant.CONTROLLER_ROLE);
}
clusterBrokersOverviewVO.setLatestMetrics(brokerMetrics);
return clusterBrokersOverviewVO;

2
km-biz/src/main/java/com/xiaojukeji/know/streaming/km/biz/group/impl/GroupManagerImpl.java
View File

@@ -75,7 +75,7 @@ public class GroupManagerImpl implements GroupManager {
}
if (!paginationResult.hasData()) {
return PaginationResult.buildSuc(dto);
return PaginationResult.buildSuc(new ArrayList<>(), paginationResult);
}
// 获取指标

2
km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/constant/KafkaConstant.java
View File

@@ -41,6 +41,8 @@ public class KafkaConstant {
public static final Long POLL_ONCE_TIMEOUT_UNIT_MS = 2000L;
public static final String CONTROLLER_ROLE = "controller";
public static final Map<String, ConfigDef.ConfigKey> KAFKA_ALL_CONFIG_DEF_MAP = new ConcurrentHashMap<>();
static {

9
km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/topic/impl/TopicServiceImpl.java
View File

@@ -27,11 +27,13 @@ import com.xiaojukeji.know.streaming.km.persistence.zk.KafkaZKDAO;
import kafka.zk.TopicsZNode;
import org.apache.kafka.clients.admin.*;
import org.apache.kafka.common.TopicPartitionInfo;
import org.apache.kafka.common.errors.UnknownTopicOrPartitionException;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.dao.DuplicateKeyException;
import org.springframework.stereotype.Service;
import java.util.*;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.TimeUnit;
import java.util.function.Function;
import java.util.stream.Collectors;
@@ -84,6 +86,13 @@ public class TopicServiceImpl implements TopicService {
}
return partitionMap;
} catch (ExecutionException e) {
log.error("method=getTopicPartitionMapFromKafka||clusterPhyId={}||topicName={}||errMsg=exception", clusterPhyId, topicName, e);
if (e.getCause() instanceof UnknownTopicOrPartitionException) {
throw new AdminOperateException(String.format("Kafka does not host Topic:[%s]", topicName), e.getCause(), ResultStatus.KAFKA_OPERATE_FAILED);
}
throw new AdminOperateException("get topic info from kafka failed", e.getCause(), ResultStatus.KAFKA_OPERATE_FAILED);
} catch (Exception e) {
log.error("method=getTopicPartitionMapFromKafka||clusterPhyId={}||topicName={}||errMsg=exception", clusterPhyId, topicName, e);
throw new AdminOperateException("get topic info from kafka failed", e, ResultStatus.KAFKA_OPERATE_FAILED);

4
km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/version/metrics/PartitionMetricVersionItems.java
View File

@@ -37,12 +37,12 @@ public class PartitionMetricVersionItems extends BaseMetricVersionMetric {
// LogEndOffset 指标
itemList.add( buildAllVersionsItem()
.name(PARTITION_METRIC_LOG_END_OFFSET).unit("条").desc("Partition中Leader副本的LogEndOffset")
.name(PARTITION_METRIC_LOG_END_OFFSET).unit("").desc("Partition中Leader副本的LogEndOffset")
.extendMethod(PARTITION_METHOD_GET_OFFSET_RELEVANT_METRICS));
// LogStartOffset 指标
itemList.add( buildAllVersionsItem()
.name(PARTITION_METRIC_LOG_START_OFFSET).unit("条").desc("Partition中Leader副本的LogStartOffset")
.name(PARTITION_METRIC_LOG_START_OFFSET).unit("").desc("Partition中Leader副本的LogStartOffset")
.extendMethod(PARTITION_METHOD_GET_OFFSET_RELEVANT_METRICS));
// Messages

4
km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/version/metrics/ReplicaMetricVersionItems.java
View File

@@ -36,13 +36,13 @@ public class ReplicaMetricVersionItems extends BaseMetricVersionMetric {
// LogEndOffset 指标
itemList.add(buildAllVersionsItem()
.name(REPLICATION_METRIC_LOG_END_OFFSET).unit("条").desc("副本的LogEndOffset")
.name(REPLICATION_METRIC_LOG_END_OFFSET).unit("").desc("副本的LogEndOffset")
.extend(buildJMXMethodExtend(REPLICATION_METHOD_GET_METRIC_FROM_JMX )
.jmxObjectName( JMX_LOG_LOG_END_OFFSET ).jmxAttribute(VALUE)));
// LogStartOffset 指标
itemList.add(buildAllVersionsItem()
.name( REPLICATION_METRIC_LOG_START_OFFSET ).unit("条").desc("副本的LogStartOffset")
.name( REPLICATION_METRIC_LOG_START_OFFSET ).unit("").desc("副本的LogStartOffset")
.extend(buildJMXMethodExtend(REPLICATION_METHOD_GET_METRIC_FROM_JMX )
.jmxObjectName( JMX_LOG_LOG_START_OFFSET ).jmxAttribute(VALUE)));

81
km-dist/init/sql/dml-logi.sql
View File

@@ -1,6 +1,8 @@
-- 初始化权限
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('1593', '多集群管理', '0', '0', '1', '多集群管理', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('1595', '系统管理', '0', '0', '1', '系统管理', '0', 'know-streaming');
-- 多集群管理权限
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('1597', '接入集群', '1593', '1', '2', '接入集群', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('1599', '删除集群', '1593', '1', '2', '删除集群', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('1601', 'Cluster-修改集群信息', '1593', '1', '2', 'Cluster-修改集群信息', '0', 'know-streaming');
@@ -14,6 +16,8 @@ INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `l
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('1617', 'Consumers-重置Offset', '1593', '1', '2', 'Consumers-重置Offset', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('1619', 'Test-Producer', '1593', '1', '2', 'Test-Producer', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('1621', 'Test-Consumer', '1593', '1', '2', 'Test-Consumer', '0', 'know-streaming');
-- 系统管理权限
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('1623', '配置管理-新增配置', '1595', '1', '2', '配置管理-新增配置', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('1625', '配置管理-编辑配置', '1595', '1', '2', '配置管理-编辑配置', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('1627', '配置管理-删除配置', '1595', '1', '2', '配置管理-删除配置', '0', 'know-streaming');
@@ -26,6 +30,23 @@ INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `l
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('1641', '用户管理-分配用户角色', '1595', '1', '2', '用户管理-分配用户角色', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('1643', '用户管理-删除角色', '1595', '1', '2', '用户管理-删除角色', '0', 'know-streaming');
-- 多集群管理权限2022-09-06新增
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('2000', '多集群管理查看', '1593', '1', '2', '多集群管理查看', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('2002', 'Topic-迁移副本', '1593', '1', '2', 'Topic-迁移副本', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('2004', 'Topic-扩缩副本', '1593', '1', '2', 'Topic-扩缩副本', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('2006', 'Cluster-LoadReBalance-周期均衡', '1593', '1', '2', 'Cluster-LoadReBalance-周期均衡', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('2008', 'Cluster-LoadReBalance-立即均衡', '1593', '1', '2', 'Cluster-LoadReBalance-立即均衡', '0', 'know-streaming');
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('2010', 'Cluster-LoadReBalance-设置集群规格', '1593', '1', '2', 'Cluster-LoadReBalance-设置集群规格', '0', 'know-streaming');
-- 系统管理权限2022-09-06新增
INSERT INTO `logi_security_permission` (`id`, `permission_name`, `parent_id`, `leaf`, `level`, `description`, `is_delete`, `app_name`) VALUES ('3000', '系统管理查看', '1595', '1', '2', '系统管理查看', '0', 'know-streaming');
-- 初始化用户
INSERT INTO `logi_security_user` (`id`, `user_name`, `pw`, `real_name`, `is_delete`, `app_name`) VALUES ('1', 'admin', 'V1ZkU2RHRlhOSGxOUkVsNVdETjBRVlp0Y0V0T1IwWnlaVEZ6YWxGRVJrRkpNVEU1VTJwYVUySkhlRzlSU0RBOWUwQldha28wWVd0N1d5TkFNa0FqWFgxS05sSnNiR2hBZlE9PXtAVmpKNGFre1sjQDNAI119SjZSbGxoQH0=Mv{#cdRgJ45Lqx}3IubEW87!==', '系统管理员', '0', 'know-streaming');
@@ -33,32 +54,40 @@ INSERT INTO `logi_security_user` (`id`, `user_name`, `pw`, `real_name`, `is_dele
INSERT INTO `logi_security_role` (`id`, `role_code`, `role_name`, `description`, `last_reviser`, `is_delete`, `app_name`) VALUES ('1677', 'r15477137', '管理员角色', '包含系统所有权限', 'admin', '0', 'know-streaming');
-- 初始化角色权限关系
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2053', '1677', '1597', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2055', '1677', '1599', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2057', '1677', '1601', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2059', '1677', '1603', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2061', '1677', '1605', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2063', '1677', '1607', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2065', '1677', '1609', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2067', '1677', '1611', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2069', '1677', '1613', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2071', '1677', '1615', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2073', '1677', '1617', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2075', '1677', '1619', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2077', '1677', '1621', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2079', '1677', '1593', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2081', '1677', '1623', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2083', '1677', '1625', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2085', '1677', '1627', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2087', '1677', '1629', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2089', '1677', '1631', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2091', '1677', '1633', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2093', '1677', '1635', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2095', '1677', '1637', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2097', '1677', '1639', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2099', '1677', '1641', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2101', '1677', '1643', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`id`, `role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('2103', '1677', '1595', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1597', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1599', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1601', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1603', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1605', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1607', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1609', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1611', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1613', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1615', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1617', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1619', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1621', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1593', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1623', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1625', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1627', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1629', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1631', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1633', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1635', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1637', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1639', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1641', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1643', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '1595', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '2000', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '2002', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '2004', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '2006', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '2008', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '2010', '0', 'know-streaming');
INSERT INTO `logi_security_role_permission` (`role_id`, `permission_id`, `is_delete`, `app_name`) VALUES ('1677', '3000', '0', 'know-streaming');
-- 初始化 用户角色关系
INSERT INTO `logi_security_user_role` (`id`, `user_id`, `role_id`, `is_delete`, `app_name`) VALUES ('1', '1', '1677', '0', 'know-streaming');

7
km-extends/km-account/src/main/java/com/xiaojukeji/know/streaming/km/account/KmAccountConfig.java
View File

@@ -7,13 +7,6 @@ import org.springframework.stereotype.Service;
@Data
@Service
public class KmAccountConfig {
/**
* LoginService的默认配置
*/
@Value(value = "${account.login.service.name:loginService}")
private String loginServiceName;
/**************************************************** Ldap 登录相关配置 ****************************************************/
@Value(value = "${account.ldap.url:}")

8
km-extends/km-account/src/main/java/com/xiaojukeji/know/streaming/km/account/common/bizenum/LoginServiceNameEnum.java
View File

@@ -1,5 +1,6 @@
package com.xiaojukeji.know.streaming.km.account.common.bizenum;
import com.didiglobal.logi.security.extend.LoginExtendBeanTool;
import lombok.Getter;
@Getter
@@ -10,9 +11,12 @@ public enum LoginServiceNameEnum {
;
public static final String DEFAULT_LOGIN_NAME = "loginService";
/**
* @see LoginExtendBeanTool.DEFAULT_BEAN_NAME
*/
public static final String DEFAULT_LOGIN_NAME = "logiSecurityDefaultLoginExtendImpl";
public static final String LDAP_LOGIN_NAME = "ldapLoginService";
public static final String LDAP_LOGIN_NAME = "ksLdapLoginService";
private final String name;

15
km-extends/km-account/src/main/java/com/xiaojukeji/know/streaming/km/account/login/ldap/LdapLoginServiceImpl.java
View File

@@ -7,7 +7,7 @@ import com.didiglobal.logi.security.common.entity.user.User;
import com.didiglobal.logi.security.common.enums.ResultCode;
import com.didiglobal.logi.security.common.vo.user.UserBriefVO;
import com.didiglobal.logi.security.exception.LogiSecurityException;
import com.didiglobal.logi.security.service.LoginService;
import com.didiglobal.logi.security.extend.LoginExtend;
import com.didiglobal.logi.security.service.UserService;
import com.didiglobal.logi.security.util.AESUtils;
import com.didiglobal.logi.security.util.CopyBeanUtil;
@@ -39,8 +39,8 @@ import static com.didiglobal.logi.security.util.HttpRequestUtil.COOKIE_OR_SESSIO
* @author Hu.Yue
* @date 2021/8/4
*/
//@Service(LoginServiceNameEnum.LDAP_LOGIN_NAME)
public class LdapLoginServiceImpl implements LoginService {
@Service(LoginServiceNameEnum.LDAP_LOGIN_NAME)
public class LdapLoginServiceImpl implements LoginExtend {
private static final Logger LOGGER = LoggerFactory.getLogger(LdapLoginServiceImpl.class);
@Autowired
@@ -89,8 +89,17 @@ public class LdapLoginServiceImpl implements LoginService {
@Override
public Result<Boolean> logout(HttpServletRequest request, HttpServletResponse response){
// 清理session
request.getSession().invalidate();
response.setStatus(REDIRECT_CODE);
// 清理cookies
for (Cookie cookie: request.getCookies()) {
cookie.setMaxAge(0);
cookie.setPath("/");
response.addCookie(cookie);
}
return Result.buildSucc(Boolean.TRUE);
}

20
km-rest/src/main/java/com/xiaojukeji/know/streaming/km/rest/interceptor/PermissionInterceptor.java
View File

@@ -4,10 +4,7 @@ import com.didiglobal.logi.log.ILog;
import com.didiglobal.logi.log.LogFactory;
import com.didiglobal.logi.security.common.constant.Constants;
import com.didiglobal.logi.security.service.LoginService;
import com.xiaojukeji.know.streaming.km.account.KmAccountConfig;
import com.xiaojukeji.know.streaming.km.account.common.bizenum.LoginServiceNameEnum;
import com.xiaojukeji.know.streaming.km.account.login.trick.TrickJumpLoginService;
import com.xiaojukeji.know.streaming.km.common.component.HandleFactory;
import com.xiaojukeji.know.streaming.km.common.constant.ApiPrefix;
import com.xiaojukeji.know.streaming.km.common.constant.Constant;
import org.springframework.beans.factory.annotation.Autowired;
@@ -36,10 +33,7 @@ public class PermissionInterceptor implements HandlerInterceptor {
private static final String OPEN_URL_PREFIX = ApiPrefix.API_V3_OPEN_PREFIX;
@Autowired
private HandleFactory handleFactory;
@Autowired
private KmAccountConfig kmAccountConfig;
private LoginService loginService;
@Autowired
private TrickJumpLoginService trickJumpLoginService;
@@ -77,21 +71,11 @@ public class PermissionInterceptor implements HandlerInterceptor {
whiteMappingValues.add(LOGIN_URL);
whiteMappingValues.add(OPEN_URL_PREFIX);
return this.getLoginService().interceptorCheck(request, response, classRequestMappingValue, whiteMappingValues);
return loginService.interceptorCheck(request, response, classRequestMappingValue, whiteMappingValues);
}
/**************************************************** private method ****************************************************/
private LoginService getLoginService() {
LoginService loginService = handleFactory.getByClassNamePer(kmAccountConfig.getLoginServiceName(), LoginService.class);
if (loginService == null) {
LOGGER.error("method=getLoginService||specifiedLoginServiceName={}||msg=specified login service not exist and use default", kmAccountConfig.getLoginServiceName());
return handleFactory.getByClassNamePer(LoginServiceNameEnum.DEFAULT_LOGIN_NAME, LoginService.class);
}
return loginService;
}
/**
* 通过反射获取带有@RequestMapping的Controller
* @param handler 请求处理器

1
km-rest/src/main/resources/application.yml
View File

@@ -41,6 +41,7 @@ spring:
driver-class-name: org.mariadb.jdbc.Driver
app-name: know-streaming
resource-extend-bean-name: myResourceExtendImpl
login-extend-bean-name: logiSecurityDefaultLoginExtendImpl
logging:
config: classpath:logback-spring.xml

7
pom.xml
View File

@@ -40,6 +40,9 @@
<log4j2.version>2.16.0</log4j2.version>
<logback.version>1.2.8</logback.version>
<!--LogICommon-->
<logi-security.version>2.10.13</logi-security.version>
</properties>
<modules>
@@ -232,7 +235,7 @@
<dependency>
<groupId>io.github.zqrferrari</groupId>
<artifactId>logi-elasticsearch-client</artifactId>
<version>1.0.16</version>
<version>1.0.24</version>
<exclusions>
<exclusion>
<artifactId>jna</artifactId>
@@ -266,7 +269,7 @@
<dependency>
<groupId>io.github.zqrferrari</groupId>
<artifactId>logi-security-spring-boot-starter</artifactId>
<version>2.10.4</version>
<version>${logi-security.version}</version>
</dependency>
<!-- logi end -->