Derek Coleman & Associates Inc logoDerek Coleman & Associates Inc

Home / Support / Oracle Cloud Marketplace / Apache Kafka

In Oracle review

DCA Hardened Event Streaming Platform for Apache Kafka — Support & Quick Start

A hardened, CVE-patched virtual machine image running Apache Kafka® — the open-source distributed event-streaming platform behind data pipelines, change-data-capture and event-driven microservices — on Oracle Linux 9.

In Oracle certification review. This listing is not in the public Oracle Cloud Marketplace catalog yet; Oracle publishes it automatically when certification completes, and this page then links to it. The procedure below is the one submitted with the listing.

At a glance

StatusIn Oracle certification review
Version4.3.1
PlatformOracle Linux 9
Ports open in the host firewallSSH (22/tcp), 9092/tcp
CategoryApplication development
Upstream licenceApache-2.0
PricingPay-as-you-go software fee per OCPU-hour, billed by Oracle on your OCI invoice — see the listing's Pricing tab

Quick start

This is the listing's usage information, verbatim. Every command in it is run by our QA on a freshly launched instance before an image version can publish, so the text and the tested procedure cannot drift apart.

After launch:

  1. Connect over SSH as the opc user with the key you supplied at launch:
    ssh opc@<public-ip>
  2. Kafka runs as a systemd service in KRaft mode, bound to loopback (clients 9092, controller 9093). On this instance's first boot, kafka-firstboot.service generated an instance-unique cluster id and formatted /var/lib/kafka/data, then kafka.service started. Check status:
    sudo systemctl status kafka-firstboot kafka

    Broker and GC logs are in /var/log/kafka. If first boot failed, fix the cause it logged and reboot: it runs again on every boot until /var/lib/kafka/data/meta.properties exists, and kafka.service waits for it.

  3. There is no password: Kafka ships without authentication and listens only on loopback. The instance-unique cluster id and connection details are recorded for root:
    sudo cat /root/.kafka_cluster_info
  4. Smoke-test from the instance (create a topic, write one event, read it back):
    /opt/kafka/bin/kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --create --topic smoke --partitions 3 --replication-factor 1
    echo hello | /opt/kafka/bin/kafka-console-producer.sh --bootstrap-server 127.0.0.1:9092 --topic smoke
    /opt/kafka/bin/kafka-console-consumer.sh --bootstrap-server 127.0.0.1:9092 --topic smoke --from-beginning --max-messages 1
  5. To reach the broker from your application tier, expose it deliberately and only with authentication and encryption — never move a PLAINTEXT listener off loopback. In /etc/kafka/server.properties (owned root:kafka, mode 0640 — keep it that way, it will hold the keystore password) keep an internal loopback listener for inter-broker traffic and administration, and put an authenticated client listener on the instance's private IP:
    listeners=INTERNAL://127.0.0.1:9091,CLIENT://<private-ip>:9092,CONTROLLER://127.0.0.1:9093
    advertised.listeners=INTERNAL://127.0.0.1:9091,CLIENT://<private-ip>:9092,CONTROLLER://127.0.0.1:9093
    listener.security.protocol.map=CONTROLLER:PLAINTEXT,INTERNAL:PLAINTEXT,CLIENT:SASL_SSL
    inter.broker.listener.name=INTERNAL
  6. In the same file, enable SCRAM authentication over TLS on the client listener and point it at a keystore holding a certificate for the private IP or hostname (https://kafka.apache.org/documentation/#security); keep /etc/kafka/broker.jks root:kafka 0640 like server.properties:
    sasl.enabled.mechanisms=SCRAM-SHA-512
    listener.name.client.scram-sha-512.sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required;
    ssl.keystore.location=/etc/kafka/broker.jks
    ssl.keystore.password=<keystore-password>
  7. Restart the broker (sudo systemctl restart kafka) and create the first client credential over the internal listener; rotate it later with the same command, and rotate the broker certificate by replacing the keystore and restarting kafka:
    /opt/kafka/bin/kafka-configs.sh --bootstrap-server 127.0.0.1:9091 --alter --add-config 'SCRAM-SHA-512=[password=<client-password>]' --entity-type users --entity-name app

    Port 9092 is already permitted in the host firewall. Restrict it in your VCN security list to the application tier's CIDR; never expose it to the internet. The controller (9093) and internal (9091) listeners stay on loopback, blocked by the host firewall.

  8. Data lives in /var/lib/kafka/data, including the cluster id (meta.properties); the cluster id is permanent for the life of that directory. For anything beyond evaluation, attach a block volume: stop kafka, mount the volume temporarily and copy /var/lib/kafka/data onto it preserving ownership (rsync -a), then add the volume to /etc/fstab (defaults,_netdev,nofail) mounted at /var/lib/kafka and start kafka — both kafka units wait for that mount and will not start without it. The broker keeps its data one level below the mount root, so the lost+found directory an ext4 volume carries is harmless there; keep everything that is not Kafka's own topic data out of /var/lib/kafka/data itself, because the broker refuses to start when its log directory contains anything else. Mounting an empty volume at /var/lib/kafka instead makes first boot create /var/lib/kafka/data on it and format a brand-new cluster on the next reboot.

    Retention defaults to 7 days (log.retention.hours=168); add log.retention.bytes to cap disk per partition. JVM heap is set in /etc/kafka/kafka.env — 1 GB by default because Kafka serves reads from the page cache; raise it only on 32 GB+ shapes with many partitions.

The image is CVE-patched at build time. Apply ongoing operating-system and OpenJDK updates with:

sudo dnf -y update

Kafka itself lives under /opt/kafka_2.13-4.3.1 with /opt/kafka as a symlink; new Kafka releases ship as new versions of this image, or can be extracted beside the current one and activated by repointing the symlink (configuration in /etc/kafka is untouched).

What the image provides

Still stuck?

Email support@dcassociatesgroup.com (response within 1 business day) or use the contact form. Include the listing name, your OCI region, the instance OCID and the output of sudo journalctl -u <service> -n 100 for the unit named in the quick start.

Apache Kafka® is a registered trademark of the Apache Software Foundation. This image is an independent hardened distribution and is not affiliated with, endorsed by, or sponsored by the Apache Software Foundation.