0

I'm running on mac m1. This is my docker-compose file:

version: '2'

networks:
  kafkanet:
    driver: bridge
    ipam:
      config:
        - subnet: 172.20.0.0/16

services:
  zookeeper:
    image: 'bitnami/zookeeper:latest'
    ports:
      - '2181:2181'
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
    networks:
      - kafkanet
  kafka:
    image: 'bitnami/kafka:latest'
    networks:
      kafkanet:
        ipv4_address: 172.20.128.2
    ports:
      - '9093:9093'
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:PLAINTEXT,EXTERNAL:PLAINTEXT
      - KAFKA_CFG_LISTENERS=CLIENT://:9092,EXTERNAL://:9093
      - KAFKA_CFG_ADVERTISED_LISTENERS=CLIENT://kafka:9092,EXTERNAL://localhost:9093
      - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=CLIENT
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
    depends_on:
      - zookeeper

  spring_boot:
    image: 'springapp'
    networks:
      - kafkanet
    depends_on:
      - kafka
      - zookeeper

  myapp:
    image: 'goapp:stable'
    networks:
      - kafkanet
    environment:
      - KAFKA_BROKERS=172.20.128.2:9092
    ports:
      - '8080:8080'

The goapp is built by running docker build . --platform linux/amd64. In the Dockerfile, I set

ENV KAFKA_BROKERS=kafka:9092
ENV KAFKA_ANALYTICS_TOPIC=mykafkatopic

Now, the spring-boot app works fine, it can produce message to kafka and consume message from kafka. But the goapp shows the following error when I check it on docker dashboard.

%3|1653633738.400|FAIL|rdkafka#producer-1| [thrd:172.20.128.2:9092/bootstrap]: 172.20.128.2:9092/bootstrap: Connect to ipv4#172.20.128.2:9092 failed: Connection refused (after 1053ms in state CONNECT)

%3|1653633739.342|FAIL|rdkafka#producer-1| [thrd:172.20.128.2:9092/bootstrap]: 172.20.128.2:9092/bootstrap: Connect to ipv4#172.20.128.2:9092 failed: Connection refused (after 0ms in state CONNECT, 1 identical error(s) suppressed)

Update: I tried to change the dockerfile to

ENV KAFKA_BROKERS=172.80.128.2:9092
ENV KAFKA_ANALYTICS_TOPIC=mykafkatopic

still getting the same error.

John Smith
  • 1
  • 1
  • 1
    dont use an ip adress, use a service name as hostname instead. – The Fool May 27 '22 at 08:17
  • I've tried all the combinations. Choose from `kafka` and `172.80.128.2` to set the value of `ENV KAFKA_BROKERS` and `environment: - KAFKA_BROKERS`, but getting the same error with all of them. – John Smith May 27 '22 at 08:36
  • I see that you have in your advertise address localhost. This works only within the container, like it does on a computer or server. You should bind all network interfaces. Use 0.0.0.0:9093 instead. And the other one doesn't have a port, it may work, but probably better to do the same for all addresses to be explicit. – The Fool May 27 '22 at 08:38
  • `java.lang.IllegalArgumentException: requirement failed: advertised.listeners cannot use the nonroutable meta-address 0.0.0.0. Use a routable IP address.` changing from localhost to 0.0.0.0 will cause an error. – John Smith May 27 '22 at 08:54
  • 1
    well, localhost will never work. You need to bind the private network interface. 0.0.0.0 is one way of doing it, it will actually bind all network interfaces. For testing you can bind 172.20.128.2, although I would recommend finding a better solution in the long run. – The Fool May 27 '22 at 09:16
  • The variables in your Dockerfile don't matter if you're going to override them in Compose., but `KAFKA_BROKERS=kafka:9092` is correct, and should be in your compose file. This needs to match the correct `KAFKA_CFG_ADVERTISED_LISTENERS` address for the environment (which won't be localhost). To **bind** to 0.0.0.0, that's the other Kafka listener property – OneCricketeer May 27 '22 at 12:31

0 Answers0