Handling Idempotency in Payment & Webhook APIs

Mastering fault-tolerant financial transactions and webhook handlers in Node.js by implementing idempotency keys, duplicate request prevention, and atomic database locks.

In distributed backend systems and payment processing workflows, network timeouts, dropped packets, and client-side retries frequently cause duplicate API requests. If a client submits a payment request or a webhook provider (like Stripe or PayPal) dispatches a webhook event twice, executing the transaction multiple times can result in accidental double-charges, duplicate order creations, and corrupted financial records.

To guarantee safe retries, modern backend architectures implement **Idempotency**—the property where an operation can be applied multiple times without changing the result beyond the initial application. This comprehensive guide explores how to design idempotency middleware, store idempotency keys in Redis or PostgreSQL, and handle webhook events safely in Node.js and Express.

How Idempotency Keys Prevent Duplicate Transactions

An idempotency key is a unique identifier (typically a UUIDv4) generated by the client and passed in request headers (e.g., `Idempotency-Key: 550e8400-e29b-41d4-a716-446655440000`).

• First Request: When a request arrives with an idempotency key, the server checks if the key exists in the database. If it does not, the server processes the transaction, stores the response payload alongside the key with an expiration TTL, and returns the result.

• Retried Request: If a network timeout occurs and the client repeats the exact same request with the identical idempotency key, the server detects the existing key and immediately returns the cached response without re-executing the business logic.

Building a Redis-Backed Idempotency Middleware

To implement robust idempotency in Node.js, we can build a reusable Express middleware using Redis to store request locks and cached responses atomically.

import { Request, Response, NextFunction } from 'express';
import { createClient } from 'redis';

const redisClient = createClient({ url: process.env.REDIS_URL || 'redis://localhost:6379' });
redisClient.connect().catch(console.error);

export async function idempotencyMiddleware(req: Request, res: Response, next: NextFunction): Promise<void> {
  const idempotencyKey = req.headers['idempotency-key'] as string;

  if (!idempotencyKey) {
    // If no key is provided, proceed normally (or enforce it for critical routes)
    return next();
  }

  const cacheKey = `idempotency:${idempotencyKey}`;

  try {
    const cachedResponse = await redisClient.get(cacheKey);

    if (cachedResponse) {
      const { status, body } = JSON.parse(cachedResponse);
      res.status(status).json({ ...body, cached: true });
      return;
    }

    // Intercept res.json to capture and cache the response before sending
    const originalJson = res.json.bind(res);
    res.json = (body: any) => {
      const responseData = { status: res.statusCode, body };
      // Store key with 24-hour TTL
      redisClient.setEx(cacheKey, 86400, JSON.stringify(responseData)).catch(console.error);
      return originalJson(body);
    };

    next();
  } catch (error) {
    console.error('Idempotency middleware error:', error);
    next(error);
  }
}

Webhook Deduplication and Event Processing

Payment gateways like Stripe send webhook events (e.g., `payment_intent.succeeded`) asynchronously. Because webhook deliveries can be retried due to network timeouts, your server must handle duplicate events gracefully.

import { Request, Response } from 'express';
import Stripe from 'stripe';

const stripe = new Stripe(process.env.STRIPE_SECRET_KEY || '', { apiVersion: '2025-02-28.acacia' });

export async function stripeWebhookHandler(req: Request, res: Response): Promise<void> {
  const sig = req.headers['stripe-signature'] as string;
  let event: Stripe.Event;

  try {
    event = stripe.webhooks.constructEvent(req.body, sig, process.env.STRIPE_WEBHOOK_SECRET || '');
  } catch (err: any) {
    res.status(400).send(`Webhook Error: ${err.message}`);
    return;
  }

  const eventId = event.id;

  // Check if event has already been processed in database or Redis
  // const processed = await db.processedEvents.findOne({ eventId });
  // if (processed) { res.status(200).json({ received: true }); return; }

  switch (event.type) {
    case 'payment_intent.succeeded':
      const paymentIntent = event.data.object as Stripe.PaymentIntent;
      console.log(`Processing successful payment: ${paymentIntent.id}`);
      // Fulfill order in database
      break;
    default:
      console.log(`Unhandled event type ${event.type}`);
  }

  // Mark eventId as processed
  // await db.processedEvents.create({ eventId, processedAt: new Date() });

  res.status(200).json({ received: true });
}

Summary

Handling idempotency in payment and webhook APIs is essential for building resilient backend systems that withstand network retries, client timeouts, and duplicate event transmissions.

By implementing idempotency keys, caching responses in Redis, and deduplicating webhook events using unique transaction identifiers, engineering teams can guarantee financial accuracy and reliable API operations.