gwenshap / kafka-examples Goto Github PK

Without any exception call to consumer is hanged infinitely while trying to run this piece of consumer code.

Have you had any problems with these working with Kafka 0.9 ?

We bought the video from O'Reilly Media and tried to run example with steps, but we got some problem.
@gwenshap

We are trying to use the same from code , but consumer topic is not receiving any messages on target side
export CLASSPATH=/Users/sxe518/Documents/test/MirrorMakerHandler/target/TopicSwitchingHandler-1.0-SNAPSHOT.jar
bin/kafka-mirror-maker.sh --consumer.config config/consumer.properties --producer.config config/producer.properties --whitelist test_.* —message.handler com.shapira.examples.TopicSwitchingHandler --message.handler.args ‘test,test123’
MirrorMakerHandler.zip

I'm using the AdminClientExample in order to reset offset for a specific topics as follows:
I'm trying to reset offsets for specific group id as follows:

 // Pause Connectors
    Await.result(k8sService.setPauseToConnectorsAsync(true), Duration.Inf)

    // Get Current Offsets
    val originalState = kafkaService.getCurrentState()

    // Reset Offsets
    kafkaService.resetOffsets(originalState)

when resetOffsets is as follows:

 def resetOffsets(originalState: Map[GroupID, mutable.Map[TopicPartition, OffsetAndMetadata]]): Unit = {
    logger.info(s"resetOffsets(originalState = $originalState) Triggered")
    for ((groupId, partitionsToOffsets) <- originalState) {
      logger.info(s"resetOffsets($groupId) Triggered")
      client.alterConsumerGroupOffsets(groupId, partitionsToOffsets.view.mapValues(_ => new OffsetAndMetadata(0)).toMap.asJava).all().get(config.operationTimeoutInMillis, TimeUnit.MILLISECONDS)
    }
  }

But its fails with

org.apache.kafka.common.errors.UnknownMemberIdException: Failed altering consumer group offsets for the following partitions

Any ideas how to solve it? or how to reset the offsets for specific group.id with other way?

JavaSessionize.avro.LogLine reference unresolved

I've compared the SimpleMovingAvgZkConsumer.java code with what was presented in the Kafka video from O'Reilly, and have successfully compiled it, but it just does not want to calculate the moving average and present it to the screen.

I've tried using both the SimpleCounter.java and the kafka-console-producer.sh to produce the numbers that SimpleMovingAvgZkConsumer.java needs, but it just remains silent until the timeout is reached and then it exits.

I'm thinking that I'm getting some kind of mismatch between the Old and New producer types, but I'm too much of a newb to be able to figure it out.

I will add that the kafka-console-consumer.sh works fine reading from both SimpleCounter.java and kafka-console-producer.sh, so it would appear that my Kafka 2 node cluster is working properly

Any sugestions?