Home

Welcome to the KAFKA wiki!

Kafka Wiki

Introduction

Welcome to the Kafka wiki for GitHub! This wiki provides an overview and implementation guide for using Kafka with Node.js and KafkaJS for building an Order Management System (OMS) or Inventory Management System.

Case Study

In a scenario where an order management system (OMS) needs to publish stock or inventory of SKUs based on company ID and channel ID.

Prerequisites

Before getting started, make sure you have the following prerequisites installed:

• Node.js: Download and install Node.js (Preferably Node.js 18.0 or the latest LTS version).
• Docker: Download and install Docker.
• VSCode: Download and install Visual Studio Code.
• KafkaJS: Install Kafka.js library.

Commands

What is ZooKeeper?

ZooKeeper is a high-performance coordination service for distributed applications. It provides common services such as naming, configuration management, synchronization, and group services.

Start ZooKeeper Container and expose PORT 2181:

docker run -p 2181:2181 zookeeper

Start Kafka Container, expose PORT 9092, and set up ENV variables:

docker run -p 9092:9092 \
-e KAFKA_ZOOKEEPER_CONNECT=<PRIVATE_IP>:2181 \
-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://<PRIVATE_IP>:9092 \
-e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 \
confluentinc/cp-kafka

File Structure

Your_Project_Directory
│   producer.js
│   consumer.js

First Run Publisher

Running the Producer Script

Navigate to your project directory:

cd your_project_directory

Execute the Producer Script to publish your inventory by SKU level:

node producer.js

Provide input when prompted. The script will prompt you to enter the following details:

• company_id
• channel_id
• sku_code
• no_of_inventory

Enter the values in the following format:

5432 365667 345 8

• company_id: 5432
• channel_id: 365667
• sku_code: 345
• no_of_inventory: 8

Publisher Output

Then Run Consumer

Running the Consumer Script

Run the Consumer Script to consume the inventory updates by SKU level:

node consumer.js user-1

Output example: When you run the consumer script with a specific user name (user-1 in this case), you will receive output similar to the following:

user-1 is the group name: Indicates the group name being used by the consumer script.
Consumer connected: Confirmation that the consumer is connected.
Log messages from the Kafka consumer library (KafkaJS) indicating the start of consumption and group membership details.
user-1: [inv-updates]: PART:0: {"company":"5432","channelId":"365667","skuCode":"345","inv":"8"}: Represents the consumed message, indicating the user, topic, partition, and message content.