Install Kafka Cluster ( Ansible )

You can use this role to install a Kafka cluster on Kubernetes.

Prerequisites

By default we have some PVCs here to manage data, so you should define /mnt/kafka & /mnt/zoo directories on suitable nodes.

You can ignore this and make it ephemeral.

How-to

First you need instal the role:

• Clone this role:

  git clone git@github.com:hatamiarash7/Ansible-Install-Kafka.git install_kafka

• Or you can install using galaxy:

  ansible-galaxy install hatamiarash7.install_kafka

Then, Include role in Playbook:

- hosts: all
    roles:
        - hatamiarash7.install_kafka

Dependencies

You need Docker & Kubernetes. Take a look at these roles too:

• install_docker
• install_kubernetes

Usage

We have two service for Kafka and two for ZK.

• A headless service to create DNS records
• A Normal service for clients

You can use bootstrap service to handle connections between your apps and Kafka cluster.

Monitoring

The Java virtual machine (Java VM) has built-in instrumentation that enables you to monitor and manage it using the Java Management Extensions (JMX) technology. These built-in management utilities are often referred to as out-of-the-box management tools for the Java VM. You can also monitor any appropriately instrumented applications using the JMX API.

We have a second container in every Kafka POD to handle this.

- name: metrics
  command:
      - java
      - -jar
      - jmx.jar
      - "5556"
      - config.yml
  ports:
      - containerPort: 5556

Later, you can use this container to monitor your Kafka cluster.

Update 13-metrics-config.yml to configure metrics.

---

Support 💛

Contributing 🤝

Don't be shy and reach out to us if you want to contribute 😉

1. Fork it !
2. Create your feature branch : git checkout -b my-new-feature
3. Commit your changes : git commit -am 'Add some feature'
4. Push to the branch : git push origin my-new-feature
5. Submit a pull request

Issues

Each project may have many problems. Contributing to the better development of this project by reporting them. 👍