How to Connect kafka-ui to an SSL enabled Kafka Cluster?

[dumm1]

I have an SSL enabled Kafka cluster installed by HDP.

Here is my docker compose file. After starting the container, the UI was up but could connect to the Kafka cluster which was said offline.

---

version: '3.4'
services:
kafka-ui:
container_name: kafka-ui
image: provectuslabs/kafka-ui:latest
ports:
- 58080:8080
environment:
KAFKA_CLUSTERS_0_NAME: kafka-cluster
KAFKA_CLUSTERS_0_PROPERTIES_SECURITY_PROTOCOL: SSL
KAFKA_CLUSTERS_0_PROPERTIES_SSL_KEYSTORE_LOCATION: /server.keystore.jks
KAFKA_CLUSTERS_0_PROPERTIES_SSL_KEYSTORE_PASSWORD: "password"
KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: <ip_address_1>:9093,<ip_address_2>:9093,<ip_address_3>:9093
KAFKA_CLUSTERS_0_SSL_TRUSTSTORELOCATION: /server.truststore.jks
KAFKA_CLUSTERS_0_SSL_TRUSTSTOREPASSWORD: "password"
KAFKA_CLUSTERS_0_PROPERTIES_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM: '' # DISABLE COMMON NAME VERIFICATION
volumes:
- ./ssl/server.truststore.jks:/server.truststore.jks
- ./ssl/server.keystore.jks:/server.keystore.jks

---

If mine is totally wrong, could share a workable example ?
Thank you in advance.

[Haarolean]

Hi, share your container logs please

[dumm1]

Hi Haarolean,

Below is a log sample that is now repeating, but there are also the other two brokers' hostname replacing <kafka_broker_hostanme_1> in different sample.
I have the /etc/hosts modified with the brokers' IP addresses and hostnames.

2023-03-31 05:26:46,366 WARN [kafka-admin-client-thread | kafka-ui-admin-1680237016-68] o.a.k.c.NetworkClient: [AdminClient clientId=kafka-ui-admin-1680237016-68] Error connecting to node <kafka_broker_hostanme_1>:9093 (id: 1003 rack: /default-rack)
java.net.UnknownHostException: <kafka_broker_hostanme_1>
at java.base/java.net.InetAddress$CachedAddresses.get(InetAddress.java:801)
at java.base/java.net.InetAddress.getAllByName0(InetAddress.java:1524)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1381)
at java.base/java.net.InetAddress.getAllByName(InetAddress.java:1305)
at org.apache.kafka.clients.DefaultHostResolver.resolve(DefaultHostResolver.java:27)
at org.apache.kafka.clients.ClientUtils.resolve(ClientUtils.java:110)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.currentAddress(ClusterConnectionStates.java:510)
at org.apache.kafka.clients.ClusterConnectionStates$NodeConnectionState.access$200(ClusterConnectionStates.java:467)
at org.apache.kafka.clients.ClusterConnectionStates.currentAddress(ClusterConnectionStates.java:173)
at org.apache.kafka.clients.NetworkClient.initiateConnect(NetworkClient.java:990)
at org.apache.kafka.clients.NetworkClient.ready(NetworkClient.java:301)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.sendEligibleCalls(KafkaAdminClient.java:1143)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.processRequests(KafkaAdminClient.java:1403)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1346)
at java.base/java.lang.Thread.run(Thread.java:833)

[Haarolean]

I have the /etc/hosts modified with the brokers' IP addresses and hostnames.

On the host machine or inside the container? If the latter, that's not supposed to work like you expect.
You can attach bash to the container and ping the host you need to see if it's reachable.
That's not the app's issue.

[dumm1]

It's on the host machine.

I tried to ping the host by attaching bash to container and it was not reachable.

It seems the container is using a different /etc/hosts.

[Haarolean]

@dumm1 yes, as I stated already, it doesn't work like that.
You can add extra hosts like this via docker-compose:

services:
  kafkaui:
    extra_hosts:
      - "memelord.lol:10.10.10.3"
      - "host1.kek:10.10.10.2"

[dumm1]

@dumm1 yes, as I stated already, it doesn't work like that. You can add extra hosts like this via docker-compose:

services:
  kafkaui:
    extra_hosts:
      - "memelord.lol:10.10.10.3"
      - "host1.kek:10.10.10.2"

@Haarolean It's awesome! By adding extra_hosts, my cluster is finally be recognized!
Thank you so much for helping me on this!!!

[Haarolean]

@dumm1 you're welcome :)