How to Install Kafka

Created on 29 November, 2023 • Technolgy • 116 views • 2 minutes read

Kafka Installation

How to Install Kafka

Step-1:We must have Java installed. So below command will install Java

Step-2: To check the java version

java --version

Step-3: Now download Kafka file

wget https://downloads.apache.org/kafka/3.5.1/kafka_2.13-3.5.1.tgz

Step-4: Now we want to unzip the file

tar -xvzf kafka_2.13-3.5.1.tgz

Step-5:Now lets move this unzipped folder to /usr/local

Step-6:Now we need to set zookeeper either you can open file and copy paste below or use cat command

sudo vi /etc/systemd/system/zookeeper.service

Step-7:Enter the following unit definition into the file:

[Unit] Description=Apache Zookeeper server Documentation=http://zookeeper.apache.org Requires=network.target remote-fs.target After=network.target remote-fs.target [Service] Type=simple ExecStart=/usr/bin/bash /usr/local/kafka/bin/zookeeper-server-start.sh /usr/local/kafka/config/zookeeper.properties ExecStop=/usr/bin/bash /usr/local/kafka/bin/zookeeper-server-stop.sh Restart=on-abnormal [Install] WantedBy=multi-user.target

Step-8:Enter the following unit definition into the file:

$cat >/etc/systemd/system/zookeeper.service1 << EOF [Unit] Description=Apache Zookeeper server Documentation=http://zookeeper.apache.org Requires=network.target remote-fs.target After=network.target remote-fs.target [Service] Type=simple ExecStart=/usr/bin/bash /usr/local/kafka/bin/zookeeper-server-start.sh /usr/local/kafka/config/zookeeper.properties ExecStop=/usr/bin/bash /usr/local/kafka/bin/zookeeper-server-stop.sh Restart=on-abnormal [Install] WantedBy=multi-user.target EOF

Step-9:Now we need to set kafka services open below file and copy paste content or use cat

sudo vi /etc/systemd/system/kafka.service

Step-10:Enter the following unit definition into the file:

[Unit] Description=Apache Kafka Server Documentation=http://kafka.apache.org/documentation.html Requires=zookeeper.service [Service] Type=simple Environment="JAVA_HOME=/usr/lib/jvm/jre-17-openjdk" ExecStart=/usr/bin/bash /usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/server.properties ExecStop=/usr/bin/bash /usr/local/kafka/bin/kafka-server-stop.sh [Install] WantedBy=multi-user.target

Step-11:Enter the following unit definition into the file:

$ cat > /etc/systemd/system/kafka.service << EOF [Unit] Description=Apache Kafka Server Documentation=http://kafka.apache.org/documentation.html Requires=zookeeper.service [Service] Type=simple Environment="JAVA_HOME=/usr/lib/jvm/jre-17-openjdk" ExecStart=/usr/bin/bash /usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/server.properties ExecStop=/usr/bin/bash /usr/local/kafka/bin/kafka-server-stop.sh [Install] WantedBy=multi-user.target EOF

Step-12:Now we need to reload deamons to make sure we got latest changes

sudo systemctl daemon-reload sudo systemctl start zookeeper sudo systemctl start kafka sudo systemctl enable zookeeper sudo systemctl enable kafka sudo systemctl status zookeeper kafka

Step-13:If you want to open firwall ports

firewall-cmd --permanent --zone=public --add-port=9092/tcp firewall-cmd --permanent --zone=public --add-port=2181/tcp sudo firewall-cmd --reload

Step-14:Lets create topics in Kafka. First goto Kafka directory

cd /usr/local/kafka/bin

Step-15:Then create topic

./kafka-topics.sh --create --bootstrap-server [server]:[port] --replication-factor 1 --partitions 1 --topic [topic-name] ./kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --topic topic1 ./kafka-console-producer.sh --broker-list localhost:9092 --topic topic1

Step-16:Open another terminal of same machine and run below command

./kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic topic1 --from-beginning