Event-Driven Architecture with Node.js and Message Queues

Event-driven architecture (EDA) decouples services by communicating through events rather than direct API calls. This approach improves scalability, fault tolerance, and team autonomy—each service can evolve independently as long as the event contracts remain stable. Node.js, with its non-blocking I/O model and native event loop, is a natural fit for building event-driven systems.

Core Concepts

In EDA, an event is a record of something that happened: an order was placed, a payment was processed, a user signed up. An event producer publishes events without knowing who will consume them. An event consumer subscribes to events it cares about and reacts accordingly. A message broker mediates between producers and consumers, ensuring reliable delivery even when consumers are temporarily offline.

Choosing a Message Broker

RabbitMQ

RabbitMQ implements the AMQP protocol and excels at task distribution and request-response patterns. It supports complex routing through exchanges, dead-letter queues for failed messages, and acknowledgment-based delivery to ensure messages are processed exactly once. RabbitMQ is ideal for workloads where message ordering is not critical and individual message processing reliability is paramount.

Apache Kafka

Kafka is a distributed event streaming platform designed for high throughput and durability. Events are stored in ordered, immutable logs (topics) partitioned across brokers. Consumers track their position (offset) in the log, enabling replay of historical events. Kafka is the right choice when you need event sourcing, stream processing, or when multiple consumers need to independently process the same stream of events.

Implementing a Publisher in Node.js

import amqp from 'amqplib';

class EventPublisher {
    private channel: amqp.Channel;

    async connect(url: string): Promise<void> {
        const connection = await amqp.connect(url);
        this.channel = await connection.createChannel();
        await this.channel.assertExchange('app.events', 'topic', {
            durable: true
        });
    }

    async publish(eventType: string, payload: object): Promise<void> {
        const message = {
            eventId: crypto.randomUUID(),
            eventType,
            timestamp: new Date().toISOString(),
            payload
        };

        this.channel.publish(
            'app.events',
            eventType,
            Buffer.from(JSON.stringify(message)),
            { persistent: true }
        );
    }
}

// Usage
const publisher = new EventPublisher();
await publisher.connect('amqp://localhost');
await publisher.publish('order.placed', {
    orderId: '12345',
    customerId: 'cust-789',
    total: 149.99
});

Implementing a Consumer

class EventConsumer {
    private channel: amqp.Channel;

    async connect(url: string, queueName: string): Promise<void> {
        const connection = await amqp.connect(url);
        this.channel = await connection.createChannel();
        await this.channel.assertQueue(queueName, { durable: true });
        await this.channel.bindQueue(queueName, 'app.events', 'order.*');
        this.channel.prefetch(10);
    }

    async consume(handler: (event: any) => Promise<void>): Promise<void> {
        this.channel.consume(this.channel.queueName, async (msg) => {
            if (!msg) return;
            try {
                const event = JSON.parse(msg.content.toString());
                await handler(event);
                this.channel.ack(msg);
            } catch (error) {
                // Reject and requeue on failure
                this.channel.nack(msg, false, true);
            }
        });
    }
}

Idempotency and Exactly-Once Processing

Message brokers guarantee at-least-once delivery, meaning the same event may be delivered multiple times. Consumers must be idempotent—processing the same event twice should produce the same result. Track processed event IDs in a database table or Redis set, and skip events that have already been handled.

Dead-Letter Queues and Retry Strategies

When a consumer fails to process a message repeatedly, it should be moved to a dead-letter queue (DLQ) for investigation rather than blocking the main queue. Implement exponential backoff retries: wait 1 second, then 2, then 4, up to a maximum retry count. After exhausting retries, route the message to the DLQ with metadata about the failure reason.

Event Sourcing and CQRS

Event sourcing stores the complete history of state changes as a sequence of events rather than storing only the current state. CQRS (Command Query Responsibility Segregation) separates write operations (commands) from read operations (queries), allowing each side to be optimized independently. Together, these patterns enable powerful audit trails, temporal queries, and replay-based debugging.

Event-driven architecture introduces operational complexity—you need to monitor queue depths, consumer lag, and message processing latency. But the benefits in scalability and decoupling make it essential for systems that handle variable, high-volume workloads. Our backend engineering team at Nexis Limited has designed event-driven systems for e-commerce platforms processing thousands of orders per minute.