Code Monkey home page Code Monkey logo

xaecbd / kcenter Goto Github PK

View Code? Open in Web Editor NEW
1.2K 37.0 196.0 9.28 MB

KCenter(KafkaCenter) is a unified platform for kafka cluster management and maintenance, producer / consumer monitoring, and use of ecological components(ksql/kafka connect).it's kafkacenter(kafka center).

Home Page: https://xaecbd.github.io/KCenter

License: Apache License 2.0

Java 53.38% HTML 0.99% JavaScript 43.43% Dockerfile 0.02% Shell 0.05% SCSS 2.13% Less 0.01%
kafka kafkacenter kafkamanager kafka-center kafka-manager kmanager kafkaoffsetmonitor kafka-monitor kafka-eagle ksql

kcenter's People

Contributors

dengliming avatar dr-kyle avatar hulva avatar loernl avatar matthiaspfenninger avatar trumandu avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

coolboydan sourcelliu gdzy1987 dbnaxlc zou-weidong tonytonyy nanofabricfx yubobo zcj-1 sc750531323 hulva fucfwangnet gztueday osmondgao zhangalex zhangduoli keevol feimin neil-huang wsf1990 qt-pay lcp5674 lsr911 joe2hpimn itxiaobao zhaochangyou dependencyinjection liuxingrong0 jackwiy macheal gitteng kndarp laashub-soa switchii junler thesky0108 bjcawanglu cmgjsw xiashuijun morningglow huanzui humorbin yoster0520 tianqiongenze 353355756 jiajie999 dosealy xiesen310 bug-builder jollyzhe hujiexuan maomao00700701 monsdio jackqii trehack beelcoder hadesrookie sofire matthiaspfenninger hadoop-devops zxs94 wbing520 liyongios wangenlu lovecoati damozhiying duhongle gallopyd jianjunliu119 scwanglijun melodyjerry shihua001 iewgnid farukonder xinzhuxiansheng manasys zhang-shoukang alone95 junneyang demogorgon314 hejiajie2008 godchou tongzhongkai liuboflyon145 zhangneng junjiem nathanxyz hujiang75277381 hkyouwen huangshengyi recentprogress evenchan86 baobin81 aapiro hadesfeng a524631266 jianoll nandakishoremutyala skymysky cghtom

kcenter's Issues

支持Kafka认证

请问是否支持Kafka SASL/PLAIN，SASL/SCRAM 或Kerberos 认证。

ACL版本Kafka

对于多个kafka集群，其中有部分是ACL版本，是否支持管理？

zk连接timeout

【环境】
zk版本 3.5.8
kafka版本 2.6.0

【异常】
2020-09-12 17:37:25,347 WARN [http-nio-8081-exec-3] ZKService: checkZKAddressHealth result is inact
ive
java.util.concurrent.TimeoutException: null
at java.util.concurrent.FutureTask.get(FutureTask.java:205)
at org.nesc.ec.bigdata.service.ZKService.checkZKAddressHealth(ZKService.java:77)
at org.nesc.ec.bigdata.controller.ClusterController.validateZKAddress(ClusterController.java:
39)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMet
hod.java:190)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHa
ndlerMethod.java:138)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invoke
AndHandle(ServletInvocableHandlerMethod.java:106)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeH
andlerMethod(RequestMappingHandlerAdapter.java:879)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleI
nternal(RequestMappingHandlerAdapter.java:793)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHan
dlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006
)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.ja
va:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.ja
va:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:92)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119
)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.ja
va:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:320)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterS
ecurityInterceptor.java:126)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(Filte
rSecurityInterceptor.java:90)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:334)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTrans
lationFilter.java:118)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:334)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagemen
tFilter.java:137)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:334)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(Ano
nymousAuthenticationFilter.java:111)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:334)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilt
er(SecurityContextHolderAwareRequestFilter.java:158)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:334)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCach
eAwareFilter.java:63)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:334)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.
java:116)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:334)
at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilt
er.java:92)
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFi
lter.java:77)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119
)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:334)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(Securit
yContextPersistenceFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:334)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.do
FilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119
)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:334)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:2
15)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.
java:358)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:2
71)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.ja
va:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.
java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119
)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.ja
va:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:9
3)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119
)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.ja
va:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(
WebMvcMetricsFilter.java:109)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119
)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.ja
va:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingF
ilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119
)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.ja
va:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
2020-09-12 17:37:25,347 WARN [http-nio-8081-exec-3] ZKService: checkZKAddressHealth result is inact
ive
java.util.concurrent.TimeoutException: null
at java.util.concurrent.FutureTask.get(FutureTask.java:205)
at org.nesc.ec.bigdata.service.ZKService.checkZKAddressHealth(ZKService.java:77)
at org.nesc.ec.bigdata.controller.ClusterController.validateZKAddress(ClusterController.java:
39)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMet
hod.java:190)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHa
ndlerMethod.java:138)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invoke
AndHandle(ServletInvocableHandlerMethod.java:106)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeH
andlerMethod(RequestMappingHandlerAdapter.java:879)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleI
nternal(RequestMappingHandlerAdapter.java:793)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHan
dlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006
)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.ja
va:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.ja
va:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:92)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119
)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.ja
va:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:320)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterS
ecurityInterceptor.java:126)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(Filte
rSecurityInterceptor.java:90)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:334)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTrans
lationFilter.java:118)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:334)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagemen
tFilter.java:137)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:334)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(Ano
nymousAuthenticationFilter.java:111)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:334)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilt
er(SecurityContextHolderAwareRequestFilter.java:158)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:334)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCach
eAwareFilter.java:63)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:334)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.
java:116)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:334)
at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilt
er.java:92)
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFi
lter.java:77)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119
)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:334)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(Securit
yContextPersistenceFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:334)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.do
FilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119
)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainP
roxy.java:334)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:2
15)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.
java:358)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:2
71)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.ja
va:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.
java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119
)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.ja
va:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:9
3)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119
)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.ja
va:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(
WebMvcMetricsFilter.java:109)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119
)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.ja
va:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingF
ilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119
)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.ja
va:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)

[Feature request] Pagination on consumer group detail page

In my case, I have a common consumer group that consumes all topics.
And I have hundreds of topics in my Kafka cluster.
Checking consumer group details takes a minute or more.
I am not sure if this is a common issue but we appreciate it if KafkaCenter can support pagination in consumer group detail screen.

请问使用Kafka connect功能还需要单独安装kafka-connect-ui吗？

在知乎上了解到了我们这个项目，很看好项目中的connect管理功能，今天clone了一份KafkaCenter代码到我的IDEA里面可以运行起来，但是Kafka connect界面是空白的，请问这个功能的使用时需要单独再安装kafka-connect-ui吗？（我只是在IDEA里面启动了一下项目，然后UI上面看下功能，并没有对Kafka进行配置，不知道是不是这个原因导致Kafka connect界面空白），附截图：
TIM图片20200420181956
最后，非常感谢大佬们提供的项目！

入门失败

1、Local run 是否需要搭建es？目前看不到相关指标数据（Monitor）
2、Docker run 启动失败
3、官网中文显示报错

KafkaConsumerServer.java 中的 getLogSize() 使用subscribe() 会产生Consumer Rebalance

Master分支：
KafkaConsumer<String, String> kafkaConsumer = consumer.subscribe(topicName);
调用
public KafkaConsumer<K, V> subscribe(String toppicName) {
this.consumer.subscribe(Collections.singleton(toppicName));
return this.consumer;
}

这个会引起其他当前groupid下的members Consumer 进行Rebalance，这样做不合理。

你可以在 KafkaConsumer实例化后，直接调用partitionsFor(topicName)，拿到Metadata的EndOffset , committedOffset

开关配置优化建议

有个小建议这边注意到很多地方使用到开关属性，可以放到一个公用配置开关类里面判断即可，不需要每个使用类都用@Value读取，如：mail.enable

kafkaAddress Inactive

kafkaAddress Inactive when i create a cluster, but my kafka is normal， kafka version is 1.0.0

KafkaCenter-Core-2.1.0.jar can not use ksql tables

KafkaCenter-Core-2.1.0.jar can not use ksql tables

image

2020-06-08 13:23:52,234 WARN [http-nio-8080-exec-4] KsqlService: ksql execute error.
java.io.IOException: Server returned HTTP response code: 400 for URL: http://10.5.3.125:8088/ksql
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1876)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
at org.nesc.ec.bigdata.service.KsqlService.executeKsql(KsqlService.java:243)
at org.nesc.ec.bigdata.service.KsqlService.describeTable(KsqlService.java:420)
at org.nesc.ec.bigdata.service.KsqlService.showStreamOrTables(KsqlService.java:138)
at org.nesc.ec.bigdata.controller.KsqlController.showTables(KsqlController.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:92)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:158)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:92)
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:77)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:109)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
2020-06-08 13:23:52,235 ERROR [http-nio-8080-exec-4] KsqlController: Ksql showTables Error!Please check,
com.alibaba.fastjson.JSONException: syntax error, expect [, actual {, pos 0, fieldName null
at com.alibaba.fastjson.parser.DefaultJSONParser.parseArray(DefaultJSONParser.java:1165)
at com.alibaba.fastjson.parser.DefaultJSONParser.parseArray(DefaultJSONParser.java:1152)
at com.alibaba.fastjson.JSON.parseArray(JSON.java:593)
at org.nesc.ec.bigdata.service.KsqlService.describeTable(KsqlService.java:420)
at org.nesc.ec.bigdata.service.KsqlService.showStreamOrTables(KsqlService.java:138)
at org.nesc.ec.bigdata.controller.KsqlController.showTables(KsqlController.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:92)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:158)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:92)
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:77)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:109)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)

项目运行起来了，但是 jdk1.8 混搭 jdk11 总会出各种问题

启动时的疑惑：
主配置文件里面的 connect.url=http://localhost:8000/#/ ，我在项目里面全局搜索只发现两个地方用到该参数，docker 启动时。然后我找到了 KafkaConnectUi 官网，我猜想这个模块不是必须的吧，所以就没管它。

运行环境：
centos7 + ES7.4 + 默认全局JDK1.8 + mysql-5.7.26(win10)
KafkaCenter-2.0.0 使用 jdk11 启动（/opt/module/jdk-11.0.5/bin/java -jar KafkaCenter-2.0.0.jar）

修改的配置：
spring.datasource.url=jdbc:mysql://lx:3306/kafka_center?useUnicode=true&characterEncoding=utf-8
monitor.elasticsearch.hosts=cdh231:9200,cdh232:9200,cdh233:9200

es 状态正常：
image
image

然后打开网页 http://cdh231:8080：
添加了kafka 集群，生产和消费功能正常，但是所有的图表监控都看不到曲线。
image
image
image
image

报错日志：
2020-03-29 13:27:45,438 ERROR [pool-2-thread-2] CollectMetricsJob: monitor metric error.
java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: cdh232; nested exception is:
java.net.ConnectException: 拒绝连接 (Connection refused)]
at java.management.rmi/javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:370)
at java.management/javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270)
at org.nesc.ec.bigdata.common.util.JmxCollector.connect(JmxCollector.java:51)
at org.nesc.ec.bigdata.common.util.JmxCollector.getJmxConnector(JmxCollector.java:37)
at org.nesc.ec.bigdata.common.util.JmxCollector.metricEveryBrokerTopic(JmxCollector.java:65)
at org.nesc.ec.bigdata.common.util.JmxCollector.metricEveryBroker(JmxCollector.java:54)
at org.nesc.ec.bigdata.service.HomeService.brokerMetric(HomeService.java:127)
at org.nesc.ec.bigdata.job.CollectMetricsJob.collectMetric(CollectMetricsJob.java:53)
at org.nesc.ec.bigdata.job.CollectorAndAlertJob.lambda$runCollectMetricsJob$1(CollectorAndAlertJob.java:91)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: javax.naming.ServiceUnavailableException: null
at jdk.naming.rmi/com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:137)
at java.naming/com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:207)
at java.naming/javax.naming.InitialContext.lookup(InitialContext.java:409)
at java.management.rmi/javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1839)
at java.management.rmi/javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1813)
at java.management.rmi/javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:302)
... 14 common frames omitted
Caused by: java.rmi.ConnectException: Connection refused to host: cdh232; nested exception is:
java.net.ConnectException: 拒绝连接 (Connection refused)
at java.rmi/sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
at java.rmi/sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:209)
at java.rmi/sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:196)
at java.rmi/sun.rmi.server.UnicastRef.newCall(UnicastRef.java:338)
at java.rmi/sun.rmi.registry.RegistryImpl_Stub.lookup(RegistryImpl_Stub.java:116)
at jdk.naming.rmi/com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:133)
... 19 common frames omitted
Caused by: java.net.ConnectException: 拒绝连接 (Connection refused)
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:403)
at java.base/java.net.Socket.connect(Socket.java:608)
at java.base/java.net.Socket.connect(Socket.java:557)
at java.base/java.net.Socket.(Socket.java:453)
at java.base/java.net.Socket.(Socket.java:230)
at java.rmi/sun.rmi.transport.tcp.TCPDirectSocketFactory.createSocket(TCPDirectSocketFactory.java:40)
at java.rmi/sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
... 24 common frames omitted

补充说明：
我的 cdh6.2 集群部署了es7.0，java环境全是1.8，但是 KafkaCenter-2.0.0 这个项目要求使用jdk11，然后我就使用jdk11 启动该项目，但是宿主机全局的环境变量是jdk1.8，上面部署了es 集群，使用过程，发现es服务无法连接，我猜想是否因为jdk不一致导致？

疑问：
1-是否有计划兼容jdk1.8？
2-kafka 消费进度的曲线怎么展示出来？以及兼容版本列表？
3-es 服务提示拒绝连接是不是jdk不兼容？应该升级全局jdk 到11吗？
4-KafkaConnectUi 模块有什么特殊作用吗？这是可选的吗？

kafka cluster health check

  • 实时监控: 在集群上建立一个专门的topic, 监控程序实时的写入数据, 但无法写入或写入耗时达到阈值时报警;

  • 基础监控: cpu, 磁盘IO, 网卡流量, FD, 连接数等;

  • Topic流量监控: 监控topic的生产和消费流量, 特别是流量突增的情况, 快速找出害群之马, 可以通过kafka的jmx来获取相关的数据, 使用Grafana来显示和报警;

Add possibility to enable/disable Connect & KSQL

Since not everyone uses Kafka Connect & KSQL in their setup, I suggest to:

  • Add parameters connect.enable and ksql.enable to the application.properties config (default:true for both)
  • Hide/show these entries in the UI based on the configuration above

与Kafka Schema Registry对接

你好，Kafka的Confluent公司有一个Schema Registry组件，可以支持Avro序列化格式的scheme管理，便于对kafka消息格式的维护和变更。可以考虑KafkaCenter与Schema Registry进行对接，在创建topic的指定topic的scheme格式。不知道社区有没有这方面的计划或者有什么好的建议

Config via UI

Hi,

Great application, I do hope to contribute it when I can.

It'd be great to have an initial UI that can be used to configure the application, instead of doing it manually via the application.properties files and of course, that includes the execution of the SQL script as well.

Improve English Documentation

The feature set of Kafka Center really looks promising so far. In order advance more quickly it would be great to enable non-Chinese developers to participate more easily.

I suggest to improve the English documentation in several places:

  • Improve main README spelling and formulation
  • Add English contributing guidelines
  • Have a version of application.properties that is clean in English with improved comments
  • Promote writing issues and PRs in English by adding templates

Optionally also:

  • Change titles of existing issues to English
  • Translate other guides (user guide, admin guide) to English

I'm happy to help on this via a PR if you agree.

Topic Reasign

Topic manipulation action ( reassign , manual reassign and etc) not active this is bug or not functionality operation ?

Missing English Translations / Improvable English Translations

Issue and Steps to Reproduce

There are missing English translations in the UI in several places:

  1. please choose role label for drop down in Setting --> User --> Add User: Role
  2. please choose location for drop down in Kafka Manager --> Cluster --> Add New Cluster: Location
  3. Items Per Page label at bottom of tables (all tables affected)
  4. Previous and Next at bottom of tables (all tables affected)
  5. No Data label for empty tables (all empty tables affected)
  6. No Options if clicking on dropdowns without any options (all dropdowns affected, e.g. Monitor --> Alert --> Create Alert: Topic Name)

There are some translations which are improvable:

  1. Kafka Manager --> Cluster --> Add New Cluster: Advance should be Advanced
  2. Kafka Manager --> Cluster --> Add New Cluster: ClusterName should be Cluster Name
  3. Monitor --> Alert --> Create Alert: Diapause is only used in context with animals. What is meant here? Maybe Time Window in which threshold has to be reached?
  4. Monitor --> Alert --> Create Alert: DisableAlerta should probably be Disable Alerts
  5. My Favorite is usually not used in singular, I would change it to Favorites
  6. Setting is usually not used in singular, I would change it to Settings
  7. My Task is singular but should be plural *My Tasks

Versions

2.1.0 (reproducible with Docker image xaecbd/kafka-center:2.1.0)

Screenshots

Missing Translations

image

image

image

image

image

image

Improvable Translations

  1. / 2.
    image

  2. / 4.
    image

  3. / 6. / 7.
    image

es 7.8版本，center启动报错

in /root/KafkaCenter/KafkaCenter-Core/target)
2020-06-24 04:00:49,384 INFO [main] Application: No active profile set, falling back to default profiles: default
2020-06-24 04:00:51,467 INFO [main] TomcatWebServer: Tomcat initialized with port(s): 8088 (http)
2020-06-24 04:00:51,525 INFO [main] ContextLoader: Root WebApplicationContext: initialization completed in 2080 ms
2020-06-24 04:00:51,869 WARN [main] HikariConfig: KafkaCenterHikariCP - idleTimeout is close to or more than maxLifetime, disabling it.
2020-06-24 04:00:51,871 INFO [main] HikariDataSource: KafkaCenterHikariCP - Starting...
2020-06-24 04:00:52,227 INFO [main] HikariDataSource: KafkaCenterHikariCP - Start completed.
2020-06-24 04:00:52,843 WARN [main] AnnotationConfigServletWebServerApplicationContext: Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'elasticsearchService': Invocation of init method failed; nested exception is java.lang.ArrayIndexOutOfBoundsException: 1
2020-06-24 04:00:52,844 INFO [main] HikariDataSource: KafkaCenterHikariCP - Shutdown initiated...
2020-06-24 04:00:52,859 INFO [main] HikariDataSource: KafkaCenterHikariCP - Shutdown completed.
2020-06-24 04:00:52,881 INFO [main] ConditionEvaluationReportLoggingListener:

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2020-06-24 04:00:52,885 ERROR [main] SpringApplication: Application run failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'elasticsearchService': Invocation of init method failed; nested exception is java.lang.ArrayIndexOutOfBoundsException: 1
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:160)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:416)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1788)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:595)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:517)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:323)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:222)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:321)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:202)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:879)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:878)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:550)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215)
at org.nesc.ec.bigdata.Application.main(Application.java:17)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:51)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:52)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
at org.nesc.ec.bigdata.common.util.ElasticsearchUtil.(ElasticsearchUtil.java:37)
at org.nesc.ec.bigdata.service.ElasticsearchService.init(ElasticsearchService.java:48)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:389)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:333)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:157)
... 26 common frames omitted

[Feature][auth] consumer and acl

  1. ACL support
  2. Is it possible to support the application of rights to consumers

1.acl 支持
2.是否可以支持对消费者的权限申请

deploy kafka with docker and can't add cluster info

rt. my boss let me to use ksql&connector with kafkacenter. so i find a solution to deploy it
referring to https://docs.confluent.io/current/quickstart/ce-docker-quickstart.html

now i can connect to kafka with an app named kafkatool. but i can't add kafka in page.it tell me kafka is inactived.log like this:

2020-05-27 16:11:57,997 WARN  [http-nio-8080-exec-1] KafkaAdminService: validateKafkaAddress  result is inactive
java.util.concurrent.TimeoutException: null
	at java.util.concurrent.FutureTask.get(FutureTask.java:205)
	at org.nesc.ec.bigdata.service.KafkaAdminService.validateKafkaAddress(KafkaAdminService.java:120)
	at org.nesc.ec.bigdata.controller.ClusterController.validateKafkaAddress(ClusterController.java:50)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
	at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:634)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:92)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:158)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:92)
	at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:77)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.boot.actuate.metrics.web.servlet.WebMvcMetricsFilter.doFilterInternal(WebMvcMetricsFilter.java:109)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:541)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:367)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1639)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:748)
2020-05-27 16:13:16,537 WARN  [Thread-17] KafkaAdminService: connect kafka error.
java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
	at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
	at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
	at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
	at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
	at org.nesc.ec.bigdata.service.KafkaAdminService$1.call(KafkaAdminService.java:102)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
2020-05-27 16:13:55,002 WARN  [Thread-18] KafkaAdminService: connect kafka error.
java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
	at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
	at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
	at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
	at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
	at org.nesc.ec.bigdata.service.KafkaAdminService$1.call(KafkaAdminService.java:102)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.

ElasticsearchService: queryDateIntervalOffset faild!

Monitor ConsumerLag is not data...
Monitor Lag The page cannot be load

env: elasticsearch 7.4.2 Authentication is not enabled

application.properties configure
monitor.enable=true
monitor.collect.period.minutes=5
monitor.elasticsearch.hosts=10.x.x.x:9200
monitor.elasticsearch.index=kafka_center_monitor
#是否启用收集线程指定集群收集
monitor.collector.include.enable=true
#收集线程指定location，必须属于remote.locations之中
monitor.collector.include.location=E4
collect.topic.enable=true
collect.topic.period.minutes=10
remote.query.enable=false
remote.hosts=gqc@localhost2:8080
remote.locations=dev,gqc,prd

kafkacenter.log
2020-05-08 23:59:52,499 ERROR [http-nio-8080-exec-1] ElasticsearchService: queryDateIntervalOffset faild!
java.lang.NullPointerException: null
at org.nesc.ec.bigdata.service.ElasticsearchService.parseDateIntervalResponse(ElasticsearchService.java:110)
at org.nesc.ec.bigdata.service.ElasticsearchService.queryDateIntervalOffset(ElasticsearchService.java:94)
at org.nesc.ec.bigdata.controller.MonitorController.getOffsetStatsInterval(MonitorController.java:223)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:106)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:879)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:793)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)

API 怎么访问?

如 kafka-manager 自带的API:
curl -s -XGET 'http://admin:[email protected]:80/api/status/clusters'

GET /api/status/:c/brokers controllers.api.KafkaStateCheck.brokers(c:String)
GET /api/status/:c/brokers/extended controllers.api.KafkaStateCheck.brokersExtended(c:String)
GET /api/status/:c/topics controllers.api.KafkaStateCheck.topics(c:String)
GET /api/status/:c/topicIdentities controllers.api.KafkaStateCheck.topicIdentities(c:String)
GET /api/status/clusters controllers.api.KafkaStateCheck.clusters
GET /api/status/:c/:t/underReplicatedPartitions controllers.api.KafkaStateCheck.underReplicatedPartitions(c:String,t:String)
GET /api/status/:c/:t/unavailablePartitions controllers.api.KafkaStateCheck.unavailablePartitions(c:String,t:String)
GET /api/status/:cluster/:consumer/:topic/:consumerType/topicSummary controllers.api.KafkaStateCheck.topicSummaryAction(cluster:String, consumer:String, topic:String, consumerType:String)
GET /api/status/:cluster/:consumer/:consumerType/groupSummary controllers.api.KafkaStateCheck.groupSummaryAction(cluster:String, consumer:String, consumerType:String)
GET /api/status/:cluster/consumersSummary controllers.api.KafkaStateCheck.consumersSummaryAction(cluster:String)

ES问题

配置完ES之后出现type is missing
`public boolean batchInsertES(List datas, String index) throws IOException {
if(client==null){
return false;
}
BulkRequest request = new BulkRequest();
if (datas == null || datas.isEmpty()) {
return true;
}
datas.forEach(data -> {
// TODO new IndexRequest(index)时,client.bulk一直报type is missing,这个方法过时了,但保证了不报错暂时还能用
request.add(new IndexRequest(index).source(data));
});

	LOG.info("batchInsertES : index : " + index + ",data : " + datas + "");

	BulkResponse bulkResponse = client.bulk(request, RequestOptions.DEFAULT);

	LOG.info("client.bulk result :" + JSON.toJSONString(bulkResponse));

	return !bulkResponse.hasFailures();

}`

打印日志如下:
ElasticsearchUtil: batchInsertES : index : kafka_center_monitor,type : kafka_center_monitor-2020-07-07,data : [{"date":"2020-07-07T09:26:00.000+0800","jmxPort":"21006","metricName":"BytesOutPerSec","fiveMinuteRate":84428.0895,"count":198054151932,"clusterID":"1","broker":"10.102.0.19","oneMinuteRate":63170.8368,"fifteenMinuteRate":53117.433,"meanRate":465406.0547,"port":"21005","clusterName":"cluster","location":"dev","timestamp":1594085160000}, {"date":"2020-07-07T09:26:00.000+0800","jmxPort":"21006","metricName":"BytesInPerSec","fiveMinuteRate":96238.9229,"count":118288142172,"clusterID":"1","broker":"10.102.0.19","oneMinuteRate":73428.1492,"fifteenMinuteRate":65046.5471,"meanRate":277964.4716,"port":"21005","clusterName":"cluster","location":"dev","timestamp":1594085160000}, {"date":"2020-07-07T09:26:00.000+0800","jmxPort":"21006","metricName":"BytesOutPerSec","fiveMinuteRate":54553.1159,"count":109035702157,"clusterID":"1","broker":"10.102.0.18","oneMinuteRate":37763.0596,"fifteenMinuteRate":37694.6493,"meanRate":256222.4317,"port":"21005","clusterName":"cluster","location":"dev","timestamp":1594085160000}, {"date":"2020-07-07T09:26:00.000+0800","jmxPort":"21006","metricName":"BytesInPerSec","fiveMinuteRate":55574.1832,"count":89660100777,"clusterID":"1","broker":"10.102.0.18","oneMinuteRate":43037.1803,"fifteenMinuteRate":37120.41,"meanRate":210691.8068,"port":"21005","clusterName":"cluster","location":"dev","timestamp":1594085160000}, {"date":"2020-07-07T09:26:00.000+0800","jmxPort":"21006","metricName":"BytesOutPerSec","fiveMinuteRate":52880.7928,"count":138662488364,"clusterID":"1","broker":"10.102.0.17","oneMinuteRate":40038.6967,"fifteenMinuteRate":33190.2336,"meanRate":325841.291,"port":"21005","clusterName":"cluster","location":"dev","timestamp":1594085160000}, {"date":"2020-07-07T09:26:00.000+0800","jmxPort":"21006","metricName":"MessagesInPerSec","fiveMinuteRate":33.2382,"count":47663123,"clusterID":"1","broker":"10.102.0.19","oneMinuteRate":27.5254,"fifteenMinuteRate":24.8562,"meanRate":112.0032,"port":"21005","clusterName":"cluster","location":"dev","timestamp":1594085160000}, {"date":"2020-07-07T09:26:00.000+0800","jmxPort":"21006","metricName":"BytesInPerSec","fiveMinuteRate":17254.7238,"count":2988950547,"clusterID":"1","broker":"10.102.0.16","oneMinuteRate":13566.8426,"fifteenMinuteRate":11686.4836,"meanRate":56040.6149,"port":"21005","clusterName":"cluster","location":"dev","timestamp":1594085160000}, {"date":"2020-07-07T09:26:00.000+0800","jmxPort":"21006","metricName":"BytesInPerSec","fiveMinuteRate":65539.8609,"count":91101226682,"clusterID":"1","broker":"10.102.0.17","oneMinuteRate":52961.1351,"fifteenMinuteRate":43281.3703,"meanRate":214077.6627,"port":"21005","clusterName":"cluster","location":"dev","timestamp":1594085160000}, {"date":"2020-07-07T09:26:00.000+0800","jmxPort":"21006","metricName":"BytesOutPerSec","fiveMinuteRate":15024.0503,"count":2917432782,"clusterID":"1","broker":"10.102.0.16","oneMinuteRate":11390.5053,"fifteenMinuteRate":9521.2635,"meanRate":54699.7085,"port":"21005","clusterName":"cluster","location":"dev","timestamp":1594085160000}, {"date":"2020-07-07T09:26:00.000+0800","jmxPort":"21006","metricName":"MessagesInPerSec","fiveMinuteRate":93.5886,"count":29947462,"clusterID":"1","broker":"10.102.0.17","oneMinuteRate":80.7972,"fifteenMinuteRate":57.5723,"meanRate":70.3732,"port":"21005","clusterName":"cluster","location":"dev","timestamp":1594085160000}, {"date":"2020-07-07T09:26:00.000+0800","jmxPort":"21006","metricName":"MessagesInPerSec","fiveMinuteRate":16.0738,"count":22906131,"clusterID":"1","broker":"10.102.0.18","oneMinuteRate":12.142,"fifteenMinuteRate":10.1072,"meanRate":53.827,"port":"21005","clusterName":"cluster","location":"dev","timestamp":1594085160000}, {"date":"2020-07-07T09:26:00.000+0800","jmxPort":"21006","metricName":"MessagesInPerSec","fiveMinuteRate":13.3404,"count":887933,"clusterID":"1","broker":"10.102.0.16","oneMinuteRate":10.9706,"fifteenMinuteRate":9.8997,"meanRate":16.6481,"port":"21005","clusterName":"cluster","location":"dev","timestamp":1594085160000}] 2020-07-07 09:26:10,326 ERROR [pool-2-thread-5] CollectMetricsJob: monitor metric error. org.elasticsearch.ElasticsearchStatusException: Elasticsearch exception [type=action_request_validation_exception, reason=Validation Failed: 1: type is missing;2: type is missing;3: type is missing;4: type is missing;5: type is missing;6: type is missing;7: type is missing;8: type is missing;9: type is missing;10: type is missing;11: type is missing;12: type is missing;] at org.elasticsearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:177) at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1727) at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1704) at org.elasticsearch.client.RestHighLevelClient.internalPerformRequest(RestHighLevelClient.java:1467) at org.elasticsearch.client.RestHighLevelClient.performRequest(RestHighLevelClient.java:1424) at org.elasticsearch.client.RestHighLevelClient.performRequestAndParseEntity(RestHighLevelClient.java:1394) at org.elasticsearch.client.RestHighLevelClient.bulk(RestHighLevelClient.java:492) at org.nesc.ec.bigdata.common.util.ElasticsearchUtil.batchInsertES(ElasticsearchUtil.java:64) at org.nesc.ec.bigdata.job.CollectMetricsJob.collectMetric(CollectMetricsJob.java:67) at org.nesc.ec.bigdata.job.CollectorAndAlertJob.lambda$runCollectMetricsJob$1(CollectorAndAlertJob.java:91) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:835)
求帮助看下是哪里出现了问题

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.