Accessing Kafka on host machine from minikube pods

.. well this applies to accessing pretty much anything on your host machine. There are times when you want to access processes running on your host machine e.g. databases etc. from your minikube Kubernetes cluster in Virtual Box (or any other supported provider)

Mind you, this is not fancy at all. In fact, there are better ways e.g. simply using Kubernetes which comes along with Docker for Mac (but not everyone is using a Mac).

Photo by Joel Fulgencio on Unsplash

Anyway, this worked well for me….

ngrok to the rescue!

It’s as simple as …

• setting up ngrok
• sign up for a (free) account and get your auth token
• start Kafka on your host machine, and just type

ngrok tcp 9092
(assuming Kafka is listening on default port 9092)

This will start forwarding requests to your local Kafka cluster on port 9092. you should see something similar to this — notice the highlighted URL

Session Status online
Account Abhishek Gupta (Plan: Free)
Version 2.2.8
Region United States (us)
Web Interface http://127.0.0.1:4040
Forwarding tcp://0.tcp.ngrok.io:16387 -> localhost:9092

Try it out

Create a topic

./kafka-topics.sh --create --topic test --partitions 1 --replication-factor 1 --zookeeper localhost:2181

Produce some data

./kafka-console-producer --broker-list 0.tcp.ngrok.io:16387 --property "parse.key=true" --property "key.separator=:" --topic test
//prompt for data
>hi:there
>ola:amigo

Make sure that you

• use the above URL for--broker-list and
• use the correct topic name (one you created in the previous step)
• and enter data when prompted..

Start consuming

./kafka-console-consumer --bootstrap-server 0.tcp.ngrok.io:16387 --topic test --property print.key=true --from-beginning

Again, make sure

• use the above URL for--bootstrap-server and
• use the correct topic name (one you created in the previous step)

You should see the data coming through.. :-)

Short and sweet. Stay tuned for more

Cheers!