Code Monkey home page Code Monkey logo

kafka-in-one-container's Introduction

Kafka in One container

Build Status

Often, I need an instance of Kafka for development purpose - if you develop services in a microservice architecture you know my problem! - but the lastest versions of Kafka need an instance of Zookeeper to start. It’s very frustrating that you need a docker-compose only to make a test!

In most cases you don't need a cluster to test your work but only a single broker, so I have created a docker image containing Zookeeper and Kafka (version 2.12-2.3.0) together, the container starts one instance of supervisor http://supervisord.org/ that manage the processes life.

I'll show you how to do step by step:

Download a stable version of Kafka from https://kafka.apache.org/, then:

FROM adoptopenjdk/openjdk11-openj9
LABEL maintainer="[email protected]" 
COPY start.sh /
RUN apt update && apt install -y supervisor && chmod a+x /start.sh
ADD kafka_2.12-2.3.0 kafka_2.12-2.3.0
COPY supervisord.conf /etc/supervisor/conf.d/supervisord.conf
EXPOSE 2181 9092
CMD ["/start.sh"]
  • In the first line I want to use adoptopenjdk as base image because it is debian based (I want to use apt) and it has a valid openjdk already installed
  • COPY the start.sh file in the root (details below)
  • An apt update it's needed to synchronize the package manager repositories
  • Then install supervisor and add a copy of Kafka as is and copy the following supervisor.conf file

The start.sh manages the environment variable KAFKA_ADVERTISED_LISTNERS used to configure the Kafka advertise url.

#!/bin/bash -e

if [[ -z "$KAFKA_ADVERTISED_LISTNERS" ]]; then
    echo "ERROR: missing mandatory config: KAFKA_ADVERTISED_LISTNERS"
    exit 1
fi

echo "advertised.listeners=PLAINTEXT://$KAFKA_ADVERTISED_LISTNERS:9092" >> /kafka_2.12-2.3.0/config/server.properties

exec "/usr/bin/supervisord" "-c" "/etc/supervisor/conf.d/supervisord.conf"

At the end of start.sh, it starts the supervisor daemon with the following configuration:

[supervisord]
nodaemon=true

[program:zookeeper]
directory=/
user=root
command=/kafka_2.12-2.3.0/bin/zookeeper-server-start.sh /kafka_2.12-2.3.0/config/zookeeper.properties
stdout_logfile=/dev/fd/1
stdout_logfile_maxbytes=0
redirect_stderr=true

[program:kafka]
directory=/
user=root
command=/kafka_2.12-2.3.0/bin/kafka-server-start.sh /kafka_2.12-2.3.0/config/server.properties
stdout_logfile=/dev/fd/1
stdout_logfile_maxbytes=0
redirect_stderr=true
starters=5

Supervisor starts Zookeeper and one Kafka broker: that's all. The container is now running with a valid instance of Zookeeper and Kafka, you can bind the standard ports to interact with Kafka and Zookeeper.

Build

docker build -t paspaola/kafka-one-container .

Run

docker run -it -p2181:2181 -p9092:9092 -e "KAFKA_ADVERTISED_LISTNERS={your-host-address}" paspaola/kafka-one-container

kafka-in-one-container's People

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.