0

I am trying to start Kafka in Docker using Docker-compose. I use Macbook Pro M1 machine with Ubuntu 20.04 in Parallels and also I use this post as a guideline and have changed a source code a little bit (authorization was turned off and I used another images from docker hub because those in a source code don't support ARM machines). Zookeeper and Kafka start in a different containers. It works fine until I try to connect to Kafka using console consumer or console producer. I use this script to create console consumer:

/home/ubuntu/kafka/bin/kafka-console-consumer.sh --topic new_topic --from-beginning --bootstrap-server 192.168.0.106:9092

and the following error outputs many times:

[2021-12-26 16:36:00,362] WARN [Consumer clientId=consumer-console-consumer-32734-1, groupId=console-consumer-32734] Error while fetching metadata with correlation id 3 : {new_topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)

I use this script to create console producer:

/home/ubuntu/kafka/bin/kafka-console-producer.sh --topic new_topic --bootstrap-server localhost:9092

The producer is being created, but when I try to send a message I get the following output a lot of times:

[2021-12-26 16:09:53,385] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 5 : {new_topic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)

My docker-compose.yml is following (all images used from docker hub):

version: '2'
services:
  zookeeper:
    image: zookeeper
    expose:
      - "2181"
  kafka:
    image: fogsyio/kafka:2.2.0
    ports:
      - "9092:9092"
    expose:
      - "9093"
    environment:
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
#      KAFKA_ADVERTISED_PORT: 9093
#      KAFKA_ADVERTISED_HOST_NAME: localhost
      KAFKA_ADVERTISED_LISTENERS: INSIDE://kafka:9093,OUTSIDE://192.168.0.106:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
      KAFKA_LISTENERS: INSIDE://0.0.0.0:9093,OUTSIDE://0.0.0.0:9092
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
      KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
      KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: PLAIN
    depends_on:
      - zookeeper

I have commented lines that I've tried to insert in an attempt to solve the problem, but it changed nothing. I've seen posts like this where it was said that the reason might be the fact that topic is not created yet, and after this error it will be created and everything would work fine. However, I get a lot of error messages that do not let me text something. I have tried to recreate consumer and producer without restarting kafka, but that didn't work.

What else can I try?

yala_by
  • 11
  • 2

0 Answers0