Bridge data to Kafka

Setup a Kafka, taking Mac OSX for instance:

  1. wget https://archive.apache.org/dist/kafka/2.8.0/kafka_2.13-2.8.0.tgz
  3. tar -xzf  kafka_2.13-2.8.0.tgz
  5. cd kafka_2.13-2.8.0
  7. # start Zookeeper
  8. ./bin/zookeeper-server-start.sh config/zookeeper.properties
  9. # start Kafka
  10. ./bin/kafka-server-start.sh config/server.properties

Create topics for Kafka:

  1. $ ./bin/kafka-topics.sh --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic testTopic --create
  3. Created topic testTopic.
  5. .. note:: Kafka topics should be created before creating the kafka rule, or the rule creation would not success.

Create a rule:

Go to EMQX DashboardBridge data to Kafka - 图1 (opens new window), select the “rule” tab on the menu to the left.

Select “message.publish”, then type in the following SQL:

  1. SELECT
  2.     *
  3. FROM
  4.     "message.publish"

image

Bind an action:

Click on the “+ Add” button under “Action Handler”, and then select “Data bridge to Kafka” in the pop-up dialog window.

image

Fill in the parameters required by the action:

Two parameters is required by action “Data to Kafka”:

1). Kafka Topic

2). Bind a resource to the action. Since the dropdown list “Resource” is empty for now, we create a new resource by clicking on the “New Resource” to the top right, and then select “Kafka”:

image

Configure the resource:

  1. Set the "Kafka Server" to "127.0.0.1:9092" (multiple servers should
  2. be separated by comma), and keep all other configs as default, and
  3. click on the "Testing Connection" button to make sure the connection
  4. can be created successfully, and then click on the "Create" button.

image

Back to the “Actions” dialog, and then click on the “Confirm” button.

image

Back to the creating rule page, then click on “Create” button. The rule we created will be show in the rule list:

image

We have finished, testing the rule by sending an MQTT message to emqx:

  1. Topic: "t/1"
  3. QoS: 0
  5. Retained: false
  7. Payload: "hello"

Then inspect Kafka by consume from the topic:

  1. $ ./bin/kafka-console-consumer.sh --bootstrap-server 127.0.0.1:9092  --topic testTopic --from-beginning

And from the rule list, verify that the “Matched” column has increased to 1:

image