Idempotent APIs: Robust Interfaces for Distributed Systems

Why Idempotency Is Not a Nice-to-Have

Anyone running microservices knows the scenario: a client sends a POST request, the server processes it successfully, but the response is lost on the way back -- network timeout, load balancer reset, whatever. The client does not know whether the request was received. So it sends it again. And suddenly the order exists twice, the payment was triggered twice, or the data is inconsistent.

In a monolith with a single database, this can still be handled via transactions. In a microservice architecture, where requests flow through multiple services and each has its own data store, this becomes an architectural problem. The solution: idempotent APIs. An API is idempotent when the same request -- no matter how many times it is sent -- always produces the same result and has no additional side effects.

This sounds trivial. But the implementation involves more than one might initially think.

What HTTP Provides Out of the Box

The HTTP specification already defines which methods are idempotent and which are not. GET, PUT, and DELETE are idempotent by definition. A GET always returns the same data (given the same state). A PUT overwrites a resource -- whether once or ten times, the result is the same. A DELETE removes a resource; the second call no longer finds it and returns 404, but does not change the system state.

POST is the exception. A POST creates a new resource, and each call potentially creates another one. This is exactly where the problem with retries arises. If a client repeats a POST /api/orders because the response was missing, two orders are created. With a POST /api/payments, the charge is applied twice.

PATCH is also not automatically idempotent, depending on the implementation. A PATCH that sets a field to an absolute value ({"status": "confirmed"}) is de facto idempotent. A PATCH that applies a relative value ({"balance": "+100"}) is not.

For GET, PUT, and DELETE, we generally do not need to do anything. For POST -- the method that most frequently produces side effects in APIs -- we need an explicit pattern.

The Idempotency Key Pattern

The idea is elegantly simple: the client generates a unique key per logical operation and sends it as an HTTP header. The server checks before processing whether this key has already been used. If so, it returns the stored result without executing the operation again.

Stripe popularized this pattern, and it has become the de facto standard for payment APIs. The header is typically called Idempotency-Key:

POST /api/payments HTTP/1.1
Content-Type: application/json
Idempotency-Key: 7a3f8b2e-4d1c-4e5f-9a6b-1c2d3e4f5a6b

{"amount": 4999, "currency": "EUR", "customer_id": "cust_42"}

The key is a UUID generated by the client. The same key means: the same logical operation. A new key means: a new operation. The server decides based on the key whether to process or return the cached result.

The elegant part of this approach: the business logic does not need to change. The idempotency check can be implemented as middleware or a filter that acts before the actual processing.

Implementation with Spring Boot and Redis

For storing idempotency keys, we use Redis with TTL. Redis is fast enough to perform a check on every request, and the TTL ensures that old keys are automatically cleaned up -- no cron job, no manual cleanup, no unbounded growth.

First, the filter that intercepts every request with an Idempotency-Key:

@Component
public class IdempotencyFilter extends OncePerRequestFilter {

    private final StringRedisTemplate redisTemplate;
    private final ObjectMapper objectMapper;

    private static final Duration KEY_TTL = Duration.ofHours(24);
    private static final String KEY_PREFIX = "idempotency:";

    public IdempotencyFilter(StringRedisTemplate redisTemplate,
                             ObjectMapper objectMapper) {
        this.redisTemplate = redisTemplate;
        this.objectMapper = objectMapper;
    }

    @Override
    protected void doFilterInternal(HttpServletRequest request,
                                    HttpServletResponse response,
                                    FilterChain filterChain)
            throws ServletException, IOException {

        String idempotencyKey = request.getHeader("Idempotency-Key");

        if (idempotencyKey == null || !"POST".equals(request.getMethod())) {
            filterChain.doFilter(request, response);
            return;
        }

        String redisKey = KEY_PREFIX + idempotencyKey;
        String cachedResponse = redisTemplate.opsForValue().get(redisKey);

        if (cachedResponse != null) {
            // Schlüssel existiert bereits -- cached Response zurückgeben
            response.setStatus(HttpServletResponse.SC_CONFLICT);
            response.setContentType("application/json");
            response.getWriter().write(cachedResponse);
            return;
        }

        // Request verarbeiten und Response cachen
        ContentCachingResponseWrapper wrappedResponse =
            new ContentCachingResponseWrapper(response);
        filterChain.doFilter(request, wrappedResponse);

        if (wrappedResponse.getStatus() >= 200
                && wrappedResponse.getStatus() < 300) {
            String body = new String(wrappedResponse.getContentAsByteArray());
            redisTemplate.opsForValue()
                .set(redisKey, body, KEY_TTL);
        }

        wrappedResponse.copyBodyToResponse();
    }
}

The logic is straightforward: check the header, query Redis, return 409 Conflict on a hit, process on a miss and cache the result. The 24-hour TTL is a pragmatic value -- long enough to catch retry storms, short enough to keep Redis from growing unbounded.

The Controller Stays Clean

This is the crucial point: the controller knows nothing about idempotency. It implements its business logic as usual. The hexagonal separation of infrastructure concerns and domain logic is preserved.

@RestController
@RequestMapping("/api/payments")
public class PaymentController {

    private final PaymentService paymentService;

    public PaymentController(PaymentService paymentService) {
        this.paymentService = paymentService;
    }

    @PostMapping
    public ResponseEntity<PaymentResponse> createPayment(
            @Valid @RequestBody PaymentRequest request) {
        Payment payment = paymentService.processPayment(request);
        return ResponseEntity
            .status(HttpStatus.CREATED)
            .body(PaymentResponse.from(payment));
    }
}

No idempotency check in the controller, no Redis access in the service. The filter catches duplicates before they reach the controller. For the client, the contract is clear: 201 Created on the first processing, 409 Conflict on a duplicate.

Alternative: Database Constraints

Not every project needs Redis. If a relational database is already in use, idempotency can also be enforced via a unique constraint:

CREATE TABLE idempotency_keys (
    key         VARCHAR(255) PRIMARY KEY,
    response    JSONB NOT NULL,
    created_at  TIMESTAMP NOT NULL DEFAULT NOW()
);

CREATE INDEX idx_idempotency_keys_created
    ON idempotency_keys (created_at);

The unique constraint on key prevents duplicate entries at the database level. A regular cleanup job removes expired entries -- this is the disadvantage compared to Redis, where the TTL handles this automatically.

We use the database approach in services that already have a PostgreSQL connection and do not run Redis. Redis is the better choice when processing many requests per second and wanting to minimize the latency of the check.

Error Handling and Edge Cases

A few situations that arise in practice and that you should think through.

Request fails. If the processing ends with an error -- such as a validation violation or an internal error -- the idempotency key should not be stored. Only successful operations are cached. Otherwise, a client could not reuse the same key after a transient error, even though the operation was never performed. This is why our filter checks the status code and only stores on 2xx.

Different payloads, same key. What happens when a client sends the same idempotency key with a different request body? Stripe returns a 422 Unprocessable Entity in this case. We handle it more pragmatically: the first request wins, the second gets 409. In practice, this scenario is almost always a client bug, not a legitimate use case.

Concurrent requests. Two identical requests arrive simultaneously. Redis helps here with SETNX (SET if Not eXists) as an atomic operation:

Boolean wasSet = redisTemplate.opsForValue()
    .setIfAbsent(redisKey, "processing", KEY_TTL);

if (Boolean.FALSE.equals(wasSet)) {
    // Anderer Request wird gerade verarbeitet
    response.setStatus(HttpServletResponse.SC_CONFLICT);
    return;
}

The setIfAbsent is atomic -- only one request wins the race. The other receives a 409 Conflict immediately.

When Idempotency Is Truly Necessary

Not every endpoint needs an idempotency key. We apply the pattern selectively:

• Payment endpoints: Always. Double charges are the worst case.
• Order creation: Always. Duplicate orders cause real damage.
• Data mutations with external side effects: If a POST triggers an email, a webhook notification, or a third-party call, it should be idempotent.
• Internal service-to-service calls: If Service A calls Service B over HTTP and a retry mechanism is built in, Service B needs idempotency protection.

Pure CRUD endpoints without critical side effects -- such as creating a comment or saving a draft -- can often do without an idempotency key. A duplicate comment is annoying, but not financial damage.

Conclusion

Idempotency is not an exotic pattern. It is a fundamental requirement for robust APIs in distributed systems. The idempotency key pattern -- a header, a Redis lookup, a TTL -- is simple enough to build into every service as middleware without touching the business logic.

The effort is minimal: a servlet filter, a Redis connection, a few lines of configuration. The payoff is significant: clients can safely retry, network problems do not lead to inconsistent data, and the system behaves predictably -- even when the infrastructure is not cooperating.

Anyone building microservices should consider idempotency from the start. Not as an optimization, but as part of the API design.