The purpose of this simple project is to provide a quick way to start an arbitrary number of Kafka servers, relying on a single ZooKeeper server, and creating a Kafka topic on the cluster.
This has only been tested on MacOS. It might work on a random UNIX system. It won't work on any Windows system (unless you have a bash shell on it).
-rflag: replication factor i.e. the number of Kafka servers that will be spun up-tflag: name of the topic that will be created-cflag: location of the configuration directory - the configuration directory is expected to contain azookeper.propertiesfile and atplsubdirectory containing aserver.tpl.propertiesfile
Example
By issuing the command
./quickstart_kafka.sh -r 3 -t mytopic -c conf
the script will start 1 ZooKeeper server, 3 Kafka servers, and will then create a topic named mytopic:
replication factor: 3
topic name: mytopic
config location: conf
Starting ZooKeeper server...
Giving ZooKeeper 10 seconds to spin up...
...done sleeping
Starting 3 separate Kafka servers...
generating properties file server-0.properties for server 0
replacing placeholders in file conf/server-0.properties
starting server 0
generating properties file server-1.properties for server 1
replacing placeholders in file conf/server-1.properties
starting server 1
generating properties file server-2.properties for server 2
replacing placeholders in file conf/server-2.properties
starting server 2
All ZooKeeper and Kafka servers started, you can terminate them all by issuing the following command:
kill 11274 11618 11625 11638 && rm -rf /tmp/zookeeper && rm -rf /tmp/kafka-logs
or you can execute the 'kill_and_cleanup.sh' file that was generated by this script
Creating topic 'mytopic' with replication_factor '3' and port '9092' at localhost
Topic 'mytopic' has been created
the script also generates a kill_and_cleanup.sh file for removing all servers (through the kill command) and wiping the /tmp directories in one swoop.
Once the server(s) have been started, we can test that Kafka is actually working by:
- starting a consumer
kafka-console-consumer --bootstrap-server localhost:9092 --topic mytopic --from-beginning
- starting a producer and issuing some messages
kafka-console-producer --broker-list localhost:9092 --topic mytopic
> my
> kafka
> cluster
> is
> working
- verify that the consumer is actually receiving the messages
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent