Idempotency In APIs: Planning for Uncertainty

Idempotency is a property that can be applied to operations, algorithms, and code. In software engineering, it refers to the ability of an operation to be performed multiple times on the same input without resulting in an unnatural state. An idempotent operation is one that can be invoked any number of times without changing the result. The opposite of idempotency is non-idempotency, which occurs when the result changes with every call. This blog post explains why you should care about idempotency and provides examples of how you can design your APIs to make them more idempotent.

Introduction

Idempotency is a property of operations that can be performed without changing the result of that operation. Idempotent operations can be repeated multiple times and have the same effect as if they had been performed once. In non-idempotent APIs, It'll be difficult for us to handle issues that may result from errors and network uncertainties, which causes clients to resend some successfully handled requests.

Consider a request a backend application which receives a debit order from a client (Let's keep it simple) ``` import express, { Request, Response } from "express"; import * as bodyParser from "body-parser";

const app = express(); app.use(bodyParser.urlencoded({ extended: true })); app.use(bodyParser.json());

let orders = []; let balance = 500; Defining the `process-order` endpoint to handle the request. app.post('/process-order', (req: Request, res: Response) => {

if (balance < req.body.amount) { return res.status(400).json({ message: 'Insufficient wallet balance!' }); }

balance = balance - Number(req.body.amount);

let newOrder = { id: Math.floor(Date.now() + Math.random()), //random 4 digit number to act as ID item: req.body.item, amount: req.body.amount, };

//add it to orders database orders.push(newOrder);

const response = { message: Order placed successfully!, };

return res.status(201).json(response); }); Finally start the server on a port app.listen(3000, () => console.log('Server running!')); ```

Now to the interesting part, Consider a request sent to the backend application containing a debit order from your wallet ``` function async sendOrder() { return await axios.post("https://server.url/process-order", { amount: 100, item: "lord-of-the-rings" }) }

const response = await sendOrder(); // response.status is 201 Assuming there was a network error on the client side, retrying this request will result in multiple debits and performance implications. const response = await sendOrder(); // response.status is 201 again If this request is tried one more time, then we get another debit on our users wallet, which is redundant const response = await sendOrder(); // response.status is 201 again ``` The performance implications of idempotency come from not having to execute an operation more than once if it doesn’t change the state of the system. You only need one request to trigger an action, not a second one with the same parameters. Moreover, you don’t need a second request in case something goes wrong on the first try and you need to retry it.

Idempotency by examples

Idempotency is an important property for APIs because it allows them to be invoked multiple times with the same input without failing or producing unpredictable results. Idempotency can be applied at the service endpoint, inputs and outputs of data transformations, and individual request-response interactions within services.

We can implement idempotency key in any way we deem fit, however common ones are

Adding idempotent key in the body // request body { amount: 100, item: 'item-key', idempotentKey: 'unique-key-id' }
Adding idempotent key in the url query parameters // request url const url = `https://server.url/endpoint?idempotentKey=unique-key-id`
Adding idempotent key in the header (Most recommended) X-Idempotent-Key: some-unique-key-id In the example shown above, Consider the backend application is redesigned to include support for idempotency. We'll add the Idempotent key in an header X-Idempotent-Key.

// To keep it simple, in real-world application, you'll setup an external cache server like redis
// or any distributed caching strategy you find fit for your application
const cache: Request<string, OrderItem|boolean> = {}

// This middleware intercepts every requests going to the process-order endpoints
app.use('/process-order', (req: Request, res: Response, next: NextFunction) {
  // fetching the key from the header when it exists
  const idempotentKey = req.headers['x-idempotent-key'] 

  if (!idempotentKey) {
   // proceed to handle the request because the request is not idempotent;
    return next() 
  }

  const processedOrder = cache[idempotentKey];

  if (!processedOrder) {
    return next() // Proceed to processing because the request was not previously processed
  }

  // Here, the request has been handled already
  return res.status(200).json({
    message: `Order processed successfully!`,
  });
})

The implementation above, which depended on a cache includes a flaw which should be considered. The use of a non-consistent cache for idempotency results in the introduction of a consistency bug because there is no atomicity guarantee between the cache and the "main database" (whatever that may be). Depending on the precise timing of the idempotence information update, you can:

end up with duplicate operations if you update the idempotence DB too late (or not at all, if the process fails or encounters an error!)
fail to register some operations because you update the idempotence DB too soon and crash, and when the client tries again, you act as though the change has already been done but in fact, it hasn't because it crashed the first time!

To solve this I recommend relying on the actual main database to decide when the operation fails or succeeds. The implementation above can be re-written as

app.use('/process-order', (req: Request, res: Response, next: NextFunction) {
  // fetching the key from the header when it exists
  const idempotentKey = req.headers['x-idempotent-key'] 

  if (!idempotentKey) {
   // proceed to handle the request because the request is not idempotent;
    return next() 
  }

  // Relying on the actual data-source to figure out if the operation has been completed before
  // This operation usually involves queries from the database
  const processedOrder = orders.find(order => order.requestId === idempotentKey);

  if (!processedOrder) {
    return next() // Proceed to processing because the request was not previously processed
  }

  // Here, the request has been handled already
  return res.status(200).json({
    message: `Order processed successfully!`,
  });
})

The actual process-order endpoint can be written to look like this implementation

// Normal processing
app.post('/process-order', (req: Request, res: Response) => {
  if (balance < req.body.amount) {
    return res.status(400).json({ message: 'Insufficient wallet balance!' });
  }

  balance = balance - Number(req.body.amount);

  const idempotentKey = req.headers['x-idempotent-key']

  let newOrder = {
    id: Math.floor(Date.now() + Math.random()), //random 4 digit number to act as ID
    item: req.body.item,
    amount: req.body.amount,
    requestId: idempotentKey ?? null // Added the idempotentKey as requestId
  };

  //add it to orders database
  orders.push(newOrder);

  // If you employ a cache, but this has its considerations and issues
  // cache[idempotentKey] = true;

  const response = {
    message: `Order placed successfully!`,
  };

  return res.status(201).json(response);
});

Now "idempotently 😂" thinking and re-writing the client implementation earlier to be idempotent. ``` // Probably persisted in the SessionStorage/LocalStorage or Component memory. // This will be added to all unique actions and changed when a new action needs to be triggered let idempotentKey = 'some-unique-key'

function async sendOrder(idempotentKey: string) { return await axios.post("https://server.url/process-order", { amount: 100, item: "lord-of-the-rings" }, { headers: { "X-Idempotent-Key": idempotentKey } }) }

const response = await sendOrder(idempotentKey); // response.status is 201 ```

Subsequent request will result in 200 success without causing multiple debit or redundant orders. const response = await sendOrder(idempotentKey); // response.status is 200

Trying again results in no state change const response = await sendOrder(idempotentKey); // response.status is 200 again

This has important performance implications for applications, so it’s worth thinking about how to design idempotent APIs from the start.

Idempotent and High-Performance APIs

APIs that don’t guarantee idempotency can lead to very slow applications. Imagine an e-commerce website where users are allowed to place orders. If an order is placed in error, you might want to cancel it. To be able to cancel an order, you need to be able to identify the order in question. To identify the order, you need to know the order ID. To know the order ID, you need to know the order total. But first, you need to know what payment method was used to place the order. If an order ID is non-idempotent, you’ll have to execute a whole chain of operations before you can find the order you want to cancel. Each of these operations involves communicating with the backend services, and each of them has the potential to fail due to network issues or other problems. If there are problems along the way, they will involve retrying the same chain of operations again, possibly even slower than before.

API Responses without Idempotency

An example of an API response that is not idempotent is one that includes a unique ID for each resource that is returned. This is the approach you’ll find in many database APIs, where you get an ID as part of the result. These IDs are not suitable as unique identifiers since they can change each time you retry an operation. IDs generated based on a combination of the current time and the internal state of the server are also not suitable as unique identifiers. These time-based IDs are bound to change on each retry, making them non-idempotent. Any other approach that relies on some internal server state that changes with each request is not idempotent.

The problem with non-idempotent APIs

While the examples above are easy to identify, real-world APIs often break idempotency in ways that are harder to spot. What if the response to an order details request also includes the order ID? It may seem like you now have everything you need in one call. But as long as the order ID is tied to the current state of the server, it’s not idempotent. It’s still possible that the first attempt at placing the order fails due to misconfiguration or network issues, and you need to retry the order details request. What if the order ID is stored as a global variable on the server side? This isn’t idempotent either, as it will be reset as soon as the order is placed.

How to make your API idempotent

To make your API idempotent, you need to identify a state that might change between retries and make it immutable. Data that can’t change between retries is suitable for storing the unique identifiers. Furthermore, such data can be shared across multiple requests without needing to be duplicated. You should use a distributed system to store the state. One thing to consider is that you should never use any kind of separate database for idempotence, unless you can make this separate DB consistent with the main DB in one transactional context (which is slow and complex).

Hence, Avoid implementing idempotence as a bolt-on solution using an external DB for which you cannot guarantee the consistency of writes relative to where the business entities live. Usually, this means you need one of:

Atomicity - you must write the operation ID together with the changes that it introduces, so that you can rely on the equivalence: operation_ID_is_present === the_operation_was_already_applied. This usually means DB transactions, sometimes even distributed transactions (JTA/XA, but don't go down that rabbit hole).
Natural idempotence - you don't rely on any external operation identifier, but instead deduplicate the essence of the command. This is tricky, for example you might not be able to have an "Add to cart" command, but instead "Make it so that there is 1 of this item in the cart". Then, deduplicating commands is a matter of seeing: is there currently 1 unit of this item in the cart? If so, do nothing.

Conclusion

APIs that are designed for high performance from the start are easier to develop and maintain. They are less likely to break in unexpected ways, and they are easier to optimise. This includes making sure they are idempotent. An API that is not idempotent may work just fine in testing, but once it’s in production, it’s bound to have serious performance issues. The key to idempotency is identifying the state that can change in each retry and making it immutable. Data that can’t change between retries is suitable for storing the unique identifiers. Furthermore, such data can be shared across multiple requests without needing to be duplicated. This way, you can reduce the number of RPC calls significantly, which saves on network overhead, as well as processing resources. It can also make your system more robust by reducing the number of points of failure.

Idempotency In APIs: Planning for Uncertainty

Comments

More from this blog

Protocols: Nothing Works Without Rules

Distributed Systems: Consensus Protocols

Fault tolerance in distributed systems

Distributed Systems: Synchronisation in Complex Systems

Database Engines: Overview

Introduction

Idempotency by examples

Idempotent and High-Performance APIs

API Responses without Idempotency

The problem with non-idempotent APIs

How to make your API idempotent

Conclusion

Command Palette

Comments

More from this blog

Introduction

Idempotency by examples

Idempotent and High-Performance APIs

API Responses without Idempotency

The problem with non-idempotent APIs

How to make your API idempotent

Conclusion