confluentinc / common Goto Github PK

View Code? Open in Web Editor NEW

98.0 284.0 241.0 30.38 MB

Common utilities library containing metrics, config and utils

License: Apache License 2.0

Java 97.25% Python 2.75%

common's Introduction

Confluent Commons

Contains 3 libraries: metrics, config, utils.

Development

Overview

Typically you will run standard maven commands such as mvn install from the top-level directory (i.e. the directory that contains this README file). The sections below explain how to build specific sub-projects independently.

Metrics

To build the metrics sub-project independently, run this from the parent directory:

$ mvn -pl :common-metrics package

Config

To build the config sub-project independently, run this from the parent directory:

$ mvn -pl :common-config package

Utils

To build the utils sub-project independently, run this from the parent directory:

$ mvn -pl :common-utils package

License

common's People

Stargazers

Watchers

Forkers

uber-archive chornyi knservis fpj rajim17 yushuyao frenchf rekhajoshm linearregression kevinjmao chiyoung0258 yanbit epeinado xingfeng2510 bestnaja ravink theduderog cotedm rspurgeon norwood dhanuka84 blbradley narasimhaup kkonstantine adamlincoln huang-tao adarshlal ewencp guptaanamika cstroe sergiopersil hotcool premsantosh nagyistge anujsrc auroraxlh rainy worximity kafkagogogo ggrossetie intvprep jcustenborder xvrl rhauch mmmtoasted ggchangan abhinavsinha1991 dougmack jrea xunux panchul william179825800 lewisdawson songweihong slockey ashishsachdeva hachikuji therealhaui ychantit akfak wicknicks ylyao ycboy99 lihaolixuewei112612 chenshengzun cmccabe larrywugit rayokota binarchitecture sunilkrdeep saidbouras a2393439531 codequester ashu-walmart kevinliangsgithub amitsela pwherman73 avocader raisingstar zefr-inc sammytech richrelevance qipengzhou big-andy-coates fossabot rayxcc ailsa84 icezjy mohnishbasha anushreeringne maduhu satchel9 tkess dindin5258 bwilliams42 wind-fly shivam355 rodesai youngwookim hjafarpour

common's Issues

There is a bug in CopyOnWriteMap

CopyOnWriteMap.entrySet() cannot simply delegate to map.entrySet() because we can use entryset.iterator().remove() method to remove elements from the map.As an example,the CopyOnWriterArrayList not allow to use iterator to remove elements.

After read your code, I implemented the CopyOnWriteMap, maybe it is better:
https://github.com/gaohanghbut/freetookit/blob/master/src/main/java/cn/yxffcode/freetookit/collection/CopyOnWriterMap.java

Cannot build master and latest 7.3.1-post branch

Apologies for possibly overlooking something in the documentation, but I cannot build this project on my MacOS machine using latest Maven 3.8.7.

According to this project readme file, I am building the master branch as

mvn install

That fails with the error message

[ERROR] Failed to execute goal io.confluent:resolver-maven-plugin:0.6.0:resolve-kafka-range (default) on project common-parent: : MojoExecutionException: Could not find any ccs-kafka artifacts. -> [Help 1]

(even though I've built and installed Apache Kafka master branch locally).

I'm trying to follow the build instructions in the kafka-streams-examples:

$ git clone https://github.com/confluentinc/common.git
$ cd common
$ git checkout master

# Build and install common locally
$ mvn -DskipTests=true clean install

That fails with the "Could not find any ccs-kafka artifacts" as well.

I've seen hints out there that I should build non-master branch. So I pick the latest 7.3.1-post branch:

git co 7.3.1-post
mvn clean install -DskipTests

I get another strange error:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.5.2:install-file (custom-install) on project common-parent: The specified file '/Users/sergey/Projects/confluent/common/installed_pom.xml' not exists -> [Help 1]

In fact, the latest release branch that I can build is the 6.0.9-post branch. That works, but I need 7.x.x version to build the kafka-streams-examples eventually.

It seems to me that this project build is broken at the moment and the issue is in the custom io.confluent:resolver-maven-plugin plugin, that gets installed_pom.xml passed in its configuration.

After update to 5.0.0 I get "No name matching <hostname> found"

Caused by: java.security.cert.CertificateException: No name matching ec2-18-213-29-168.compute-1.amazonaws.com found. The script I usually use to generate my creds is setup-certs. I think I need to add -dname = CN=host.... or something of that sort. Could someone please help me? I have 3 urls for 3 brokers.

Thank you.

Add support for Scala 2.13

Support for Scala 2.13 was added to Kafka in 2.4.0, but Confluent artifacts are still lacking it.

It would be amazing of you to add it soon, so that third party libraries built in Scala could be released (e.g., embeddedkafka/embedded-kafka-schema-registry#78).

org.apache.kafka.common.serialization.StringDeserializer is not an instance of org.apache.kafka.common.serialization.Deserializer

kafka-client version is 1.0.0

Message: Failed to construct kafka consumer
StackTrace: at org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:781)
at org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:604)
at org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:587)
at org.apache.spark.streaming.kafka010.SubscribePattern.onStart(ConsumerStrategy.scala:140)
at org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.consumer(DirectKafkaInputDStream.scala:70)
at org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.start(DirectKafkaInputDStream.scala:240)
at org.apache.spark.streaming.DStreamGraph$$anonfun$start$7.apply(DStreamGraph.scala:54)
at org.apache.spark.streaming.DStreamGraph$$anonfun$start$7.apply(DStreamGraph.scala:54)
at scala.collection.parallel.mutable.ParArray$ParArrayIterator.foreach_quick(ParArray.scala:143)
at scala.collection.parallel.mutable.ParArray$ParArrayIterator.foreach(ParArray.scala:136)
at scala.collection.parallel.ParIterableLike$Foreach.leaf(ParIterableLike.scala:972)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.scala:49)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:51)
at scala.collection.parallel.ParIterableLike$Foreach.tryLeaf(ParIterableLike.scala:969)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.compute(Tasks.scala:152)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:443)
at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:160)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
at ... run in separate thread using org.apache.spark.util.ThreadUtils ... ()
at org.apache.spark.streaming.StreamingContext.liftedTree1$1(StreamingContext.scala:578)
at org.apache.spark.streaming.StreamingContext.start(StreamingContext.scala:572)
at com.wealthfront.data.scala.adhoc.WFStreamProvider.start(WFStreamProvider.scala:47)
... 44 elided
Caused by: org.apache.kafka.common.KafkaException: org.apache.kafka.common.serialization.StringDeserializer is not an instance of org.apache.kafka.common.serialization.Deserializer
at org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:248)
at org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:680)
at org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:604)
at org.apache.kafka.clients.consumer.KafkaConsumer.(KafkaConsumer.java:587)
at org.apache.spark.streaming.kafka010.SubscribePattern.onStart(ConsumerStrategy.scala:140)
at org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.consumer(DirectKafkaInputDStream.scala:70)
at org.apache.spark.streaming.kafka010.DirectKafkaInputDStream.start(DirectKafkaInputDStream.scala:240)
at org.apache.spark.streaming.DStreamGraph$$anonfun$start$7.apply(DStreamGraph.scala:54)
at org.apache.spark.streaming.DStreamGraph$$anonfun$start$7.apply(DStreamGraph.scala:54)
at scala.collection.parallel.mutable.ParArray$ParArrayIterator.foreach_quick(ParArray.scala:143)
at scala.collection.parallel.mutable.ParArray$ParArrayIterator.foreach(ParArray.scala:136)
at scala.collection.parallel.ParIterableLike$Foreach.leaf(ParIterableLike.scala:972)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.scala:49)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:51)
at scala.collection.parallel.ParIterableLike$Foreach.tryLeaf(ParIterableLike.scala:969)
at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.compute(Tasks.scala:152)
at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:443)
at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:160)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)

DataException raised in AvroData.toConnectData

when I use confluent to load data from DB2 to HDFS I found something strange ,when there are some NULL value in a table of DB2, It CAN NOT be LOADED successfully,there is a EXCEPTION raised.

The POM for io.confluent:resolver-maven-plugin:jar:0.4.0 is missing, no dependency information available

Though I have successfully built kafka following steps in https://github.com/confluentinc/kafka.git and using:
./gradlewAll install
./gradlew install
** Using ./gradlew installAll I get the error :
Task 'installAll' not found in root project 'kafka'. Some candidates are: 'install'.

while building Common with mvn clean install I get the error:
The POM for io.confluent:resolver-maven-plugin:jar:0.4.0 is missing, no dependency information available

Delete The Issue

Delete it.

Zookeeper throws timeout after several restarts of nodes

I have a problem with zookeeper deployed on kubernetes using Confluentinc Docker Images.

Cluster setup:

3 nodes in digital ocean ( 4 CPU, 8 GB or RAM) as kafka dedicated nodepool. ( Only zookeeper and kafka are running here, and some monitoring daemon set).
If a zookeeper node/pod ( any node) restarts several times the zookeeper cluster is not accessible anymore, and to make it work I have to redeploy all the nodes.
Once the node rejoined the cluster, all zookeeper nodes start to show this kind of error:
Each node has one kafka pod and one zookeeper pod.

Errors one each node:

zookeeper-node-0 logs:
[2020-10-29 00:29:20,852] INFO Refusing session request for client /10.244.4.86:47054 as it has seen zxid 0xd00000000 our last zxid is 0xc00000004 client must try another server (org.apache.zookeeper.server.ZooKeeperServer)
FULL LOGS: node-0-log.log

zookeeper-node-1 logs:
244.3.185:36040 as it has seen zxid 0xd00000000 our last zxid is 0xc00000004 client must try another server (org.apache.zookeeper.server.ZooKeeperServer).
FULL LOGS: node-1-log.log

zookeeper-node-2 logs:
[2020-10-29 00:28:59,554] WARN Unable to read additional data from client sessionid 0x102508ff4380002, likely client has closed socket (org.apache.zookeeper.server.NIOServerCnxn)
FULL LOGS: node-2-log.log

although the zookeeper cluster seems to be up, I'm not able to get anything from it. If I try to use cli to list a path I'm getting a timeout.
Any Ideas?

If this is not the place for this issue please guide me.
thanks

utility-belt build failure : io.confluent.admin.utils.ClusterStatusSASLTest

On CDH 5.13.1
When trying to build with $ mvn clean install
Have already built Kafka branch 1.1 using $ gradlew installAll with no errors
Can't see any SASL params set in zoo.cfg

Error log shows

Test set: io.confluent.admin.utils.ClusterStatusSASLTest

Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.005 s <<< FAILURE! - in io.confluent.admin.utils.ClusterStatusSASLTest
io.confluent.admin.utils.ClusterStatusSASLTest Time elapsed: 0.005 s <<< ERROR!
org.apache.zookeeper.KeeperException$AuthFailedException: KeeperErrorCode = AuthFailed for /consumers
at io.confluent.admin.utils.ClusterStatusSASLTest.setup(ClusterStatusSASLTest.java:4

No means of removing password from logs

AbstractConfig's constructor calls a private method that logs all key-value pairs in the config map. In the case of the JdbcSourceConnector, this includes the connection URL and password.

Unfortunately, that means that the password is then persisted in plain text for anyone with access to the logs to read.

Can we provide a means of either overriding this logging behavior or providing the password as a separate config?

Failed to execute goal on project webdriverextensions: Could not resolve dependencies for project

I am getting below error while maven install:

com.github.webdriverextensions:webdriverextensions:jar:3.5.2.4-SNAPSHOT: Failed to collect dependencies for [org.apache.poi:poi:jar:3.16 (compile), org.apache.poi:poi-ooxml:jar:3.16 (compile), org.apache.poi:poi-ooxml-schemas:jar:3.16 (compile), com.sun.codemodel:codemodel:jar:2.6 (compile), org.apache.commons:commons-lang3:jar:3.5 (compile), com.google.code.gson:gson:jar:2.8.0 (compile), junit:junit:jar:4.12 (compile), com.googlecode.lambdaj:lambdaj:jar:2.3.3 (compile), ch.qos.logback:logback-classic:jar:1.2.3 (compile), org.slf4j:slf4j-log4j12:jar:1.8.0-alpha2 (compile), org.seleniumhq.selenium:selenium-java:jar:3.4.0 (compile), org.seleniumhq.selenium:selenium-remote-driver:jar:3.4.0 (compile), org.seleniumhq.selenium:selenium-api:jar:3.4.0 (compile), org.seleniumhq.selenium:htmlunit-driver:jar:2.26 (compile), net.sourceforge.htmlunit:htmlunit:jar:2.26 (compile), com.codeborne:phantomjsdriver:jar:1.4.2 (compile), com.google.code.findbugs:annotations:jar:2.0.1 (compile), org.mockito:mockito-core:jar:1.9.5 (test), org.assertj:assertj-core:jar:1.7.1 (test)]: Failed to read artifact descriptor for org.seleniumhq.selenium:selenium-support:jar:3.12.0: Could not transfer artifact org.seleniumhq.selenium:selenium-support:pom:3.12.0 from/to central (http://repo.maven.apache.org/maven2): Access denied to http://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-support/3.12.0/selenium-support-3.12.0.pom. Error code 403, Forbidden -> [Help 1]

Can you please let me know what the issue is?

Build failure -

Hello,

mvn compile fails with following error:

...
[INFO]
[INFO] --------------< io.confluent:assembly-plugin-boilerplate >--------------
[INFO] Building assembly-plugin-boilerplate 6.0.0-SNAPSHOT               [1/11]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] ----------------------< io.confluent:build-tools >----------------------
[INFO] Building Build Tools 6.0.0-SNAPSHOT                               [2/11]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ build-tools ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ build-tools ---
[INFO] No sources to compile
[INFO]
[INFO] ---------------------< io.confluent:common-parent >---------------------
[INFO] Building common 6.0.0-SNAPSHOT                                    [3/11]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for common 6.0.0-SNAPSHOT:
[INFO]
[INFO] assembly-plugin-boilerplate ........................ SUCCESS [  0.002 s]
[INFO] Build Tools ........................................ SUCCESS [  0.341 s]
[INFO] common ............................................. FAILURE [  0.077 s]
[INFO] utils .............................................. SKIPPED
[INFO] metrics ............................................ SKIPPED
[INFO] config ............................................. SKIPPED
[INFO] common-logging ..................................... SKIPPED
[INFO] confluent-log4j-extensions ......................... SKIPPED
[INFO] confluent-log4j2-extensions ........................ SKIPPED
[INFO] package ............................................ SKIPPED
[INFO] Common ............................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.675 s
[INFO] Finished at: 2020-05-26T08:34:24-07:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project common-parent: Could not resolve dependencies for project io.confluent:common-parent:pom:6.0.0-SNAPSHOT: Failure to find io.confluent:assembly-plugin-boilerplate:zip:resources:6.0.0-SNAPSHOT in https://myrepo.jfrog.io/myrepo/maven was cached in the local repository, resolution will not be reattempted until the update interval of au has elapsed or updates are forced -> [Help 1]

Building individual submodule per README file yields similar error, mvn -pl :common-metrics compile:

[INFO] --------------------< io.confluent:common-metrics >---------------------
[INFO] Building metrics 6.0.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[WARNING] The POM for io.confluent:common-utils:jar:6.0.0-SNAPSHOT is missing, no dependency information available
[WARNING] The POM for io.confluent:assembly-plugin-boilerplate:zip:resources:6.0.0-SNAPSHOT is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.412 s
[INFO] Finished at: 2020-05-26T08:39:24-07:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project common-metrics: Could not resolve dependencies for project io.confluent:common-metrics:jar:6.0.0-SNAPSHOT: The following artifacts could not be resolved: io.confluent:common-utils:jar:6.0.0-SNAPSHOT, io.confluent:assembly-plugin-boilerplate:zip:resources:6.0.0-SNAPSHOT: Failure to find io.confluent:common-utils:jar:6.0.0-SNAPSHOT in https://myrepo.jfrog.io/myrepo/maven was cached in the local repository, resolution will not be reattempted until the update interval of au has elapsed or updates are forced -> [Help 1]

Is the kafka.version value correct?

run command mvn install failed!

I modify <kafka.version>2.1.0-SNAPSHOT</kafka.version> to <kafka.version>1.1.0</kafka.version>
in pom.xml , then build success!

common-config transitively depends on, but does not use, zookeeper and netty

common-utils defines dependencies on both zookeeper and zkclient, which transitively include netty 3.10.6.Final. common-config depends on common-utils, but does not use any zookeeper-related functionality, however it still transitively pulls along zookeeper and netty libs, and anything with a transitive dependency on common-config gets those too.

We have an application that uses kafka-avro-serializer, with transitive deps on common-config, common-util, zookeeper, and netty 3.10.6.Final. We're also using an http client lib that pulls in (and actually uses) netty 4.1.33.Final, and we tracked down a NoClassDefFoundError to having both netty 4 and 3 libs on the classpath.

Since common-config doesn't use any zookeeper/netty stuff, it's probably a good idea that it does not pull along those jars. Maybe exclude them in common-config pom, or extract out a new common-util-zookeeper lib?

Build failure on project common-utils: groups/excludedGroups require TestNG or JUnit48+ on project test classpath

On the master branch:

mvn install
...
[INFO] Reactor Summary for common 5.4.0-SNAPSHOT:
[INFO] 
[INFO] assembly-plugin-boilerplate ........................ SUCCESS [  0.442 s]
[INFO] Build Tools ........................................ SUCCESS [  0.358 s]
[INFO] common ............................................. SUCCESS [  1.048 s]
[INFO] utils .............................................. FAILURE [  0.620 s]
[INFO] metrics ............................................ SKIPPED
[INFO] config ............................................. SKIPPED
[INFO] common-logging ..................................... SKIPPED
[INFO] confluent-log4j-extensions ......................... SKIPPED
[INFO] confluent-log4j2-extensions ........................ SKIPPED
[INFO] package ............................................ SKIPPED
[INFO] Common ............................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  2.721 s
[INFO] Finished at: 2019-06-04T15:12:11-07:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.21.0:test (default-test) on project common-utils: groups/excludedGroups require TestNG or JUnit48+ on project test classpath -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :common-utils

I have successfully built Kafka with ./gradlew installAll. Any troubleshooting tips?

Jars since v7.5.3 not being published to Confluent public Maven repo

It seems no jars for tags after v7.5.3 are landing in https://packages.confluent.io/maven/io/confluent/common/
As reported on confluentinc/kafka-connect-storage-cloud#704 we are unable to build downstream projects (e.g. kafka-connect-storage-cloud) without first building all confluent library dependency projects locally.
Note that this problem extends to all Confluent libraries and isn't just related to common, however we thought this might be a good place to report it to get the issue noticed and addressed.

Please let us know if more information is required.

Thanks!

Log4J security vulnerability verification

A 0-day exploit (CVE-2021-44228 – also called log4shell) was discovered in the popular Java logging library log4j2 that allows remote code execution (RCE) by logging a specific string (similar to SQL injections):

The vulnerability occurs in the log library log4j version >= 2.0-beta9 and < 2.15 . The older version of the log4j library version 1 is not directly affected as of today.

Please verify this Log4J security vulnerability issue with your products, and provide users your verification results.

Add Support for Avro 1.9.0

Would be good to see Confluent and the schema registry supporting the recent release of Avro 1.9.0 http://avro.apache.org/docs/1.9.0/, the first release in two years.

Affected by CVE-2023-4586 due to dependency with netty

Trivy has flagged kafka-connect-jdbc:10.7.3 as affected by CVE-2023-4586 due to Netty dependecy, the issues is described here netty/netty#8537 and won't be fixed until Netty 5.x.

Just creating this issue for tracking and have something to watch regarding our confluent dependencies.

Sources jars are missing in Maven repository

Sources jar are missing from maven repo which makes it hard to debug or quick look at the source code inside the IDE where sources for given version can be downloaded with one click. Whereas now one has to go and clone the github repo and attach sources manually.

example:
http://packages.confluent.io/maven/io/confluent/kafka-avro-serializer/5.2.1/kafka-avro-serializer-5.2.1.jar <- works great
http://packages.confluent.io/maven/io/confluent/kafka-avro-serializer/5.2.1/kafka-avro-serializer-5.2.1-sources.jar <- doesn't 😞

mvn clean install -X on version 7.4.1-post fails with installed_pom.xml not exists

Everything works just fine on version 6.0.0.

But as soon as I run

mvn clean install -X

on version 7.4.1-post, I am getting this error:

[ERROR] The specified file '/home/ihor/Projects/confluent/common/installed_pom.xml' not exists
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for common 7.4.1:
[INFO]
[INFO] assembly-plugin-boilerplate ........................ SUCCESS [  0.546 s]
[INFO] Build Tools ........................................ SUCCESS [  0.479 s]
[INFO] common ............................................. FAILURE [  2.253 s]
[INFO] utils .............................................. SKIPPED
[INFO] metrics ............................................ SKIPPED
[INFO] config ............................................. SKIPPED
[INFO] common-logging ..................................... SKIPPED
[INFO] confluent-log4j-extensions ......................... SKIPPED
[INFO] confluent-log4j2-extensions ........................ SKIPPED
[INFO] package ............................................ SKIPPED
[INFO] Common ............................................. SKIPPED
[INFO] disk-usage-agent ................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  3.596 s
[INFO] Finished at: 2023-07-27T11:14:54+03:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.5.2:install-file (custom-install) on project common-parent: The specified file '/home/ihor/Projects/confluent/common/installed
_pom.xml' not exists -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-install-plugin:2.5.2:install-file (custom-install) on project common-parent: The specified file
 '/home/ihor/Projects/confluent/common/installed_pom.xml' not exists
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.plugin.MojoFailureException: The specified file '/home/ihor/Projects/confluent/common/installed_pom.xml' not exists
    at org.apache.maven.plugin.install.InstallFileMojo.execute (InstallFileMojo.java:188)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[ERROR]
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :common-parent
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Could anyone help to find out what is the problem and how to fix that?

Unable to build Master with MVN compile or install

Running Git Clone on master and running mvn compile/mvn install fails due to lack of connect-api:jar:2.3.0-SNAPSHOT jars amongst others.
Below is a copy of the output running mvn compile on master.
Please advise.

[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Detecting the operating system and CPU architecture
[INFO] ------------------------------------------------------------------------
[INFO] os.detected.name: windows
[INFO] os.detected.arch: x86_64
[INFO] os.detected.version: 10.0
[INFO] os.detected.version.major: 10
[INFO] os.detected.version.minor: 0
[INFO] os.detected.classifier: windows-x86_64
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] assembly-plugin-boilerplate                                        [pom]
[INFO] Build Tools                                                        [jar]
[INFO] common                                                             [pom]
[INFO] utils                                                              [jar]
[INFO] metrics                                                            [jar]
[INFO] config                                                             [jar]
[INFO] common-logging                                                     [jar]
[INFO] confluent-log4j-extensions                                         [jar]
[INFO] confluent-log4j2-extensions                                        [jar]
[INFO] package                                                            [pom]
[INFO] Common                                                             [pom]
[INFO]
[INFO] --------------< io.confluent:assembly-plugin-boilerplate >--------------
[INFO] Building assembly-plugin-boilerplate 5.3.0-SNAPSHOT               [1/11]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] ----------------------< io.confluent:build-tools >----------------------
[INFO] Building Build Tools 5.3.0-SNAPSHOT                               [2/11]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ build-tools ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ build-tools ---
[INFO] No sources to compile
[INFO]
[INFO] ---------------------< io.confluent:common-parent >---------------------
[INFO] Building common 5.3.0-SNAPSHOT                                    [3/11]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] --- maven-checkstyle-plugin:2.17:check (validate) @ common-parent ---
[INFO] Starting audit...
Audit done.
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) @ common-parent ---
[INFO]
[INFO] ---------------------< io.confluent:common-utils >----------------------
[INFO] Building utils 5.3.0-SNAPSHOT                                     [4/11]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-checkstyle-plugin:2.17:check (validate) @ common-utils ---
[INFO] Starting audit...
Audit done.
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) @ common-utils ---
[INFO]
[INFO] --- maven-resources-plugin:3.0.2:resources (default-resources) @ common-utils ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory C:\dev\common\utils\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ common-utils ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 13 source files to C:\dev\common\utils\target\classes
[INFO]
[INFO] --------------------< io.confluent:common-metrics >---------------------
[INFO] Building metrics 5.3.0-SNAPSHOT                                   [5/11]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-checkstyle-plugin:2.17:check (validate) @ common-metrics ---
[INFO] Starting audit...
Audit done.
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) @ common-metrics ---
[INFO]
[INFO] --- maven-resources-plugin:3.0.2:resources (default-resources) @ common-metrics ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory C:\dev\common\metrics\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ common-metrics ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 26 source files to C:\dev\common\metrics\target\classes
[INFO]
[INFO] ---------------------< io.confluent:common-config >---------------------
[INFO] Building config 5.3.0-SNAPSHOT                                    [6/11]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-checkstyle-plugin:2.17:check (validate) @ common-config ---
[INFO] Starting audit...
Audit done.
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) @ common-config ---
[INFO]
[INFO] --- maven-resources-plugin:3.0.2:resources (default-resources) @ common-config ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory C:\dev\common\config\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ common-config ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 5 source files to C:\dev\common\config\target\classes
[INFO] /C:/dev/common/config/src/main/java/io/confluent/common/config/AbstractConfig.java: Some input files use unchecked or unsafe operations.
[INFO] /C:/dev/common/config/src/main/java/io/confluent/common/config/AbstractConfig.java: Recompile with -Xlint:unchecked for details.
[INFO]
[INFO] --------------------< io.confluent:common-logging >---------------------
[INFO] Building common-logging 5.3.0-SNAPSHOT                            [7/11]
[INFO] --------------------------------[ jar ]---------------------------------
[WARNING] The POM for org.apache.kafka:connect-api:jar:2.3.0-SNAPSHOT is missing, no dependency information available
[WARNING] The POM for org.apache.kafka:connect-json:jar:2.3.0-SNAPSHOT is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for common 5.3.0-SNAPSHOT:
[INFO]
[INFO] assembly-plugin-boilerplate ........................ SUCCESS [  0.015 s]
[INFO] Build Tools ........................................ SUCCESS [  0.806 s]
[INFO] common ............................................. SUCCESS [  2.004 s]
[INFO] utils .............................................. SUCCESS [  4.725 s]
[INFO] metrics ............................................ SUCCESS [  1.372 s]
[INFO] config ............................................. SUCCESS [  0.755 s]
[INFO] common-logging ..................................... FAILURE [  0.084 s]
[INFO] confluent-log4j-extensions ......................... SKIPPED
[INFO] confluent-log4j2-extensions ........................ SKIPPED
[INFO] package ............................................ SKIPPED
[INFO] Common ............................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  10.379 s
[INFO] Finished at: 2019-03-06T12:10:27Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project common-logging: Could not resolve dependencies for project io.confluent:common-logging:jar:5.3.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.kafka:connect-api:jar:2.3.0-SNAPSHOT, org.apache.kafka:connect-json:jar:2.3.0-SNAPSHOT: Failure to find org.apache.kafka:connect-api:jar:2.3.0-SNAPSHOT in http://packages.confluent.io/maven/ was cached in the local repository, resolution will not be reattempted until the update interval of confluent has elapsed or updates are forced -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :common-logging

Could not find artifact io.confluent:licenses:jar:4.1.0-SNAPSHOT in central

Hello, I got following error trying to install from master.

[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.5.0:java (create-licenses-for-docker) on project common-parent: Execution create-licenses-for-docker of goal org.codehaus.mojo:exec-maven-plugin:1.5.0:java failed: Plugin org.codehaus.mojo:exec-maven-plugin:1.5.0 or one of its dependencies could not be resolved: Could not find artifact io.confluent:licenses:jar:4.1.0-SNAPSHOT in central (https://repo1.maven.org/maven2) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :common-parent

Build fails for common-config

running mvn -pl :common-config package from the parent folder gives the following error:

[ERROR] Failed to execute goal on project common-config: Could not resolve dependencies for project io.confluent:common-config:jar:5.2.0-SNAPSHOT: The following artifacts could not be resolved: io.confluent:common-utils:jar:5.2.0-SNAPSHOT, io.confluent:assembly-plugin-boilerplate:zip:resources:5.2.0-SNAPSHOT: Failure to find io.confluent:common-utils:jar:5.2.0-SNAPSHOT in http://packages.confluent.io/maven/ was cached in the local repository, resolution will not be reattempted until the update interval of confluent has elapsed or updates are forced -> [Help 1]

KAFKA-2026 - Logging of unused options always shows null for the value and is misleading if the option is used by serializers

Our AbstractConfig is also affected by this issue and we need to port whatever patch ends up being used to resolve it. https://issues.apache.org/jira/browse/KAFKA-2026

Change license header in all files

Latest version of master doesn't build

I am on c14de12 , running the maven build results in:

[INFO] Resolving range for org.apache.kafka:kafka-clients::[6.1.0-0, 6.1.1-0), excluding snapshots
Downloading from central: https://myarty.example.com/public-maven-virtual/org/apache/kafka/kafka-clients/maven-metadata.xml
Downloaded from central: https://myarty.example.com/public-maven-virtual/org/apache/kafka/kafka-clients/maven-metadata.xml (59 kB at 2.4 kB/s)
[WARNING] Could not find any ce-kafka artifacts.
[WARNING] Continuing with build because fail if not found for this version is set to false.
[WARNING] Could not find any ccs-kafka artifacts.
...snip...
[ERROR] Failed to execute goal io.confluent:resolver-maven-plugin:0.5.0:resolve-kafka-range (default) on project common-parent: : MojoExecutionException: Could not find any ccs-kafka artifacts. -> [Help 1]

In the meantime I will try a specific label and see if that helps. I am trying to contribute to confluentinc/schema-registry#898 of which this is a 2 hop transitive dependency (via common -> rest-utils -> schema-registry)... but I am not sure if I will be able to make changes on latest master of that project if I don't have the latest changes from this repo as well...

Cannot resolve org.apache.kafka:connect-json:5.5.0-ccs-SNAPSHOT

I have build kafka (trunk branch -- commit: 1675115ec193acf4c7d44e68a57272edfec0b455) successfully with ./gradlew installAll

Now I'm trying to compile this project common, but I'm getting the following error:


[INFO] --------------------< io.confluent:common-logging >---------------------
[INFO] Building common-logging 5.5.0-SNAPSHOT                            [7/11]
[INFO] --------------------------------[ jar ]---------------------------------
[WARNING] The POM for org.apache.kafka:connect-api:jar:5.5.0-ccs-SNAPSHOT is missing, no dependency information available
[WARNING] The POM for org.apache.kafka:connect-json:jar:5.5.0-ccs-SNAPSHOT is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for common 5.5.0-SNAPSHOT:
[INFO] 
[INFO] assembly-plugin-boilerplate ........................ SUCCESS [  1.555 s]
[INFO] Build Tools ........................................ SUCCESS [  1.057 s]
[INFO] common ............................................. SUCCESS [  3.355 s]
[INFO] utils .............................................. SUCCESS [  1.799 s]
[INFO] metrics ............................................ SUCCESS [  3.836 s]
[INFO] config ............................................. SUCCESS [  2.585 s]
[INFO] common-logging ..................................... FAILURE [  0.103 s]
[INFO] confluent-log4j-extensions ......................... SKIPPED
[INFO] confluent-log4j2-extensions ........................ SKIPPED
[INFO] package ............................................ SKIPPED
[INFO] Common ............................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  15.149 s
[INFO] Finished at: 2019-11-13T16:14:10Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project common-logging: Could not resolve dependencies for project io.confluent:common-logging:jar:5.5.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.kafka:connect-api:jar:5.5.0-ccs-SNAPSHOT, org.apache.kafka:connect-json:jar:5.5.0-ccs-SNAPSHOT: Failure to find org.apache.kafka:connect-api:jar:5.5.0-ccs-SNAPSHOT in http://packages.confluent.io/maven/ was cached in the local repository, resolution will not be reattempted until the update interval of confluent has elapsed or updates are forced -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :common-logging

Request an upgrade of zookeeper to 3.4.14 to fix security vulnerability

Requesting an upgrade of zookeeper to 3.4.14 to fix the security vulnerability.

https://github.com/confluentinc/common/blob/master/pom.xml#L85

https://nvd.nist.gov/vuln/detail/CVE-2019-0201

A little problem in `io.confluent.common.metrics.stats.Histogram#toString`

When i debug io.confluent.common.metrics.stats.HistogramTest#testHistogram test case, i find the instance of Histogram cannot display right info...

-Infinity:0,-5.0000000000:1,-4.0000000000:1,-3.0000000000:1,-2.0000000000:1,-1.0000000000:1,0.0000000000:1,1.0000000000:1,2.0000000000:1,3.0000000000:1,4.0000000000:1,Infinity:0.0}

It should be...

{-Infinity:0,-5.0000000000:1,-4.0000000000:1,-3.0000000000:1,-2.0000000000:1,-1.0000000000:1,0.0000000000:1,1.0000000000:1,2.0000000000:1,3.0000000000:1,4.0000000000:1,Infinity:0.0}

How is the bin package of confluent kafka built?

I see that the tarball present in http://packages.confluent.io/archive/5.1/confluent-5.1.0-2.11.tar.gz

contains dependency jars arranged in different folder structure ,with dependent jar files distributed according to categories in share/java/ folder

However , when i clone the git of confluent kafka v_5.1 and build the project , all the dependent jar files are present in lib/ folder instead .

Is this because of a different gradle definition ? where can i obtain the gradle file for publishing http://packages.confluent.io/archive/5.1/confluent-5.1.0-2.11.tar.gz ?

Build failure for common-logging

Hi, when I tried to mvn install, I found I cannot make it because of the printed below:
[ERROR] Failed to execute goal on project common-logging: Could not resolve dependencies for project io.confluent:common-logging:jar:5.5.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.kafka:connect-api:jar:5.5.0-ccs-SNAPSHOT, org.apache.kafka:connect-json:jar:5.5.0-ccs-SNAPSHOT: Failure to find org.apache.kafka:connect-api:jar:5.5.0-ccs-SNAPSHOT in http://packages.confluent.io/maven/ was cached in the local repository, resolution will not be reattempted until the update interval of confluent has elapsed or updates are forced -> [Help 1]

Build Fails

[ERROR] Failed to execute goal on project common-metrics: Could not resolve dependencies for project io.confluent:common-metrics:jar:5.2.0-SNAPSHOT: The following artifacts could not be resolved: io.confluent:common-utils:jar:5.2.0-SNAPSHOT, io.confluent:assembly-plugin-boilerplate:zip:resources:5.2.0-SNAPSHOT: Failure to find io.confluent:common-utils:jar:5.2.0-SNAPSHOT in http://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced -> [Help 1]

Unable to resolve the assembly-plugin-boilerplate 6.0.3 jar file

   > Could not determine artifacts for io.confluent:assembly-plugin-boilerplate:6.0.3
      > Could not get resource 'https://packages.confluent.io/maven/io/confluent/assembly-plugin-boilerplate/6.0.3/assembly-plugin-boilerplate-6.0.3.jar'.
         > Could not HEAD 'https://packages.confluent.io/maven/io/confluent/assembly-plugin-boilerplate/6.0.3/assembly-plugin-boilerplate-6.0.3.jar'. Received status code 403 from server: Forbidden

The above is the error message we got when building. Checked https://packages.confluent.io/maven/io/confluent/assembly-plugin-boilerplate/6.0.3/ and looks like there is no such jar file. Can you guide what should we do to resolve the issue? Thanks!

shared httpRequest method

It may be useful to share the schema-registry httpRequest method (in RestUtils) by moving it to common

Build of 6.1.0-post branch and v6.1.0 tag fails

Building the 6.1.0-post branch of v6.1.0 tag of this repo fails with:

[ERROR] Failed to execute goal io.confluent:resolver-maven-plugin:0.5.0:resolve-kafka-range (default) on project common-parent: : MojoExecutionException: Could not find any ccs-kafka artifacts. -> [Help 1]

Issue appears to be with Maven resolver plugin, which is looking for 6.1.0 artifacts instead of 6.1.0-ccs. The following is a workaround:

mvn install -Dskip.maven.resolver.plugin=true

However, this should ideally be fixed in pom.xml.

Upgrade Avro version to 1.11.0

The 1.10.2 Avro version has several vulnerabilities AVRO-3227 which are fixed in the 1.11.0 version AVRO-3215.

java.lang.IllegalArgumentException: A metric named 'MetricName' already exists, can't register another one.

Hi,
After playing with Kafka Connect I'm getting following Exception.
After connect restarting everything is OK. I don't know how to reproduce this in controlled manner.

java.lang.IllegalArgumentException: A metric named 'MetricName [name=offset-commit-max-time-ms, group=connector-task-metrics, description=The maximum time in milliseconds taken by this task to commit offsets., tags={connector=test_source_test_bulk, task=0}]' already exists, can't register another one.
	at org.apache.kafka.common.metrics.Metrics.registerMetric(Metrics.java:532)
	at org.apache.kafka.common.metrics.Sensor.add(Sensor.java:256)
	at org.apache.kafka.common.metrics.Sensor.add(Sensor.java:241)
	at org.apache.kafka.connect.runtime.WorkerTask$TaskMetricsGroup.<init>(WorkerTask.java:328)
	at org.apache.kafka.connect.runtime.WorkerTask.<init>(WorkerTask.java:69)
	at org.apache.kafka.connect.runtime.WorkerSourceTask.<init>(WorkerSourceTask.java:100)
	at org.apache.kafka.connect.runtime.Worker.buildWorkerTask(Worker.java:445)
	at org.apache.kafka.connect.runtime.Worker.startTask(Worker.java:404)
	at org.apache.kafka.connect.runtime.distributed.DistributedHerder.startTask(DistributedHerder.java:852)
	at org.apache.kafka.connect.runtime.distributed.DistributedHerder.access$1600(DistributedHerder.java:108)
	at org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:866)
	at org.apache.kafka.connect.runtime.distributed.DistributedHerder$13.call(DistributedHerder.java:862)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

Parent pom should depend on children modules

This makes it easy to pull in the entire package in one dependency instead of listing all the modules separately.

Build fail with "Failed to execute goal on project common-logging"

My kafka project is on 'trunk' and common on 'master'.
I've successfully built kafka but on common, I get the below error.

[INFO] assembly-plugin-boilerplate ........................ SUCCESS [ 0.610 s]
[INFO] Build Tools ........................................ SUCCESS [ 0.694 s]
[INFO] common ............................................. SUCCESS [ 1.333 s]
[INFO] utils .............................................. SUCCESS [ 0.832 s]
[INFO] metrics ............................................ SUCCESS [ 1.995 s]
[INFO] config ............................................. SUCCESS [ 1.546 s]
[INFO] common-logging ..................................... FAILURE [ 5.408 s]
[INFO] confluent-log4j-extensions ......................... SKIPPED
[INFO] confluent-log4j2-extensions ........................ SKIPPED
[INFO] package ............................................ SKIPPED
[INFO] Common ............................................. SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12.692 s
[INFO] Finished at: 2019-06-04T15:36:45+10:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project common-logging: Could not resolve dependencies for project io.confluent:common-logging:jar:5.4.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.kafka:connect-api:jar:5.4.0-ccs-SNAPSHOT, org.apache.kafka:connect-json:jar:5.4.0-ccs-SNAPSHOT: Could not find artifact org.apache.kafka:connect-api:jar:5.4.0-ccs-SNAPSHOT in confluent (http://packages.confluent.io/maven/) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :common-logging

I've cleared out my local maven repository and retried but still getting the same error. Changed the branch to different ones but it doesn't seem to be working..
Been googling for 3 hours and got nowhere.

Can anyone point me to the right direction?

mvn -pl :common-utils package fails

mvn -pl :common-utils package
[INFO] Scanning for projects...
[ERROR] Could not find the selected project in the reactor: :common-utils -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MavenExecutionException

Module names should include 'common'

Modules should include the name of the parent project. They should probably be common-metrics, common-utils, etc.

Statsd MetricReporter

Hi,
We have developed Statsd implementation of MetricReporter to track our schema registry metrics. Is it worth to include to common-metrics project ?? If it is, I can create a Pull Requests...

Thanks
Raji

logredactor project lost?

Apologies for posting here, but as it's a dependency of common it felt like something of value here as well as in general for confluentinc.

I came across Maven artifacts for the logredactor project today, and they point to https://github.com/confluentinc/logredactor as the source repository - but this project does not appear to exist (anymore). Is this something the confluentinc folk here know the history of?

I see it as a dependency here: https://github.com/confluentinc/common/blob/master/log4j-extensions/pom.xml#L42 so am figuring it's a concern for this repo.

        <groupId>io.confluent</groupId>
        <artifactId>logredactor</artifactId>

Failed to execute goal on project utility-belt

Cannot build of master, getting the below error
Could not resolve dependencies for project io.confluent:utility-belt:jar:4.1.0-SNAPSHOT: The following artifacts could not be resolved: org.apache.kafka:kafka_2.11:jar:1.1.0-SNAPSHOT, org.apache.kafka:kafka-clients:jar:1.1.0-SNAPSHOT, org.apache.kafka:kafka-clients:jar:test:1.1.0-SNAPSHOT, org.apache.kafka:kafka_2.11:jar:test:1.1.0-SNAPSHOT: Failure to find org.apache.kafka:kafka_2.11:jar:1.1.0-SNAPSHOT in http://packages.confluent.io/maven/ was cached in the local repository, resolution will not be reattempted until the update interval of confluent has elapsed or updates are forced

Mac M1 docker error building base with Spotify `microdef` `returned a non-zero code: 1`

I am trying to build some personal M1 images for myself, and I performed a maven build inside of the base directory with tag 7.1.1 (bd600b37) using the following command

$ mvn clean package -e -Pdocker -DskipTests -Ddocker.registry=dhinojosa/ -DCONFLUENT_VERSION=7.1.1

I receive during that build a failure with the following stack trace

Caused by: com.spotify.docker.client.exceptions.DockerException: The command '/bin/sh -c microdnf
 --nodocs install yum    
&& rpm --import https://www.azul.com/files/0xB1998361219BD9C9.txt     
&& yum --nodocs -y install https://cdn.azul.com/zulu/bin/zulu-repo-1.0.0-1.noarch.rpm     
&& yum --nodocs install -y --setopt=install_weak_deps=False         
git  "openssl${OPENSSL_VERSION}"  "wget${WGET_VERSION}"    
"nmap-ncat${NETCAT_VERSION}"         
"python36${PYTHON36_VERSION}"         
"tar${TAR_VERSION}"  "procps-ng${PROCPS_VERSION}"       
"krb5-workstation${KRB5_WORKSTATION_VERSION}"   "iputils${IPUTILS_VERSION}"         
"hostname${HOSTNAME_VERSION}"         
"zulu11-ca-jdk-headless${ZULU_OPENJDK_VERSION}" 
"zulu11-ca-jre-headless${ZULU_OPENJDK_VERSION}"    
 && alternatives --set python /usr/bin/python3     
&& python3 -m pip install --upgrade "pip${PYTHON_PIP_VERSION}" 
"setuptools${PYTHON_SETUPTOOLS_VERSION}"     
&& python3 -m pip install --prefer-binary --prefix=/usr/local --upgrade "${PYTHON_CONFLUENT_DOCKER_UTILS_INSTALL_SPEC}"    
&& yum remove -y git     
&& yum --nodocs update -y tzdata libgcc libstdc++ cyrus-sasl-lib     
&& yum clean all     
&& rm -rf /tmp/*     && mkdir -p /etc/confluent/docker /usr/logs     
&& useradd --no-log-init --create-home --shell /bin/bash appuser    
&& chown appuser:appuser -R /etc/confluent/ /usr/logs' returned a non-zero code: 1