Advanced Promise Patterns: Promise Memoization

In this post, we’ll show how a common caching implementation exposes a race condition. We’ll then show how to fix it, and (unlike most optimizations) we’ll simplify our code in the process.

We’ll do this by introducing the Promise Memoization pattern, which builds on the Singleton Promise pattern.

Use Case: Caching Asynchronous Results

Consider the following simple API client:

const getUserById = async (userId: string): Promise<User> => {
  const user = await request.get(`https://users-service/${userId}`);
  return user;
};

Very straightforward.

But, what should we do if performance is a concern? Perhaps users-service is slow to resolve user details, and maybe we’re frequently calling this method with the same set of user IDs.

We’ll probably want to add caching. How might we do this?

Naive Solution

Here’s the naive solution I frequently see:

const usersCache = new Map<string, User>();

const getUserById = async (userId: string): Promise<User> => {
  if (!usersCache.has(userId)) {
    const user = await request.get(`https://users-service/${userId}`);
    usersCache.set(userId, user);
  }

  return usersCache.get(userId);
};

It’s pretty simple: after we resolve the user details from users-service, we populate an in-memory cache with the result.

The Race Condition (Again)

What’s wrong with this implementation? Well, it’s subject to the same race condition from my Singleton Promises post.

Specifically, it will make duplicate network calls in cases like the following:

await Promise.all([
  getUserById('user1'),
  getUserById('user1')
]);

The problem is that we don’t assign the cache until after the first call resolves. But wait, how can we populate a cache before we have the result?

Singleton Promises to the Rescue

What if we cache the promise for the result, rather than the result itself? The code looks like this:

const userPromisesCache = new Map<string, Promise<User>>();

const getUserById = (userId: string): Promise<User> => {
  if (!userPromisesCache.has(userId)) {
    const userPromise = request.get(`https://users-service/v1/${userId}`);
    userPromisesCache.set(userId, userPromise);
  }

  return userPromisesCache.get(userId)!;
};

Very similar, but instead of awaiting the network request, we place its promise into a cache and return it to the caller (who will await the result).

Note that we no longer declare our method async, since it no longer calls await. Our method signature hasn’t changed though - we still return a promise, but we do so synchronously. (If this seems confusing, I encourage you to run an experiment.)

This fixes the race condition. Regardless of timing, only one network request fires when we make multiple calls to getUserById('user1'). This is because all subsequent callers receive the same singleton promise as the first.

Problem solved! Now let’s take a step back.

Promise Memoization

Seen from another angle, our last caching implementation is literally just memoizing getUserById! When given an input we’ve already seen, we simply return the result that we stored (which happens to be a promise).

So, memoizing our async method gave us caching without the race condition.

The upside of this insight is that there are many libraries that make memoization dead-simple, including lodash.

This means we can simplify our last solution to:

import _ from 'lodash';

const getUserById = _.memoize(async (userId: string): Promise<User> => {
  const user = await request.get(`https://users-service/${userId}`);
  return user;
});

We took our original cache-less implementation and dropped in the _.memoize wrapper! Very simple and noninvasive.

(For production use cases, you’ll probably definitely want something like memoizee that lets you control the caching strategy.)

Update: In production, you should absolutely be using memoizee with the promise: true flag to avoid caching errors. See my update for details.

Conclusion

In this post, we saw how we can use memoization wrappers around our async methods.

While this approach is simpler than manually populating a result cache, we also learned that it’s better because it avoids a common race condition.

Update 1/15: Error Handling

A lot of folks have reached out to ask about error handling, which is a great question that I absolutely should have explained in the original post!

Especially in the case of API clients, you should consider the possibility that an operation may fail. If our memoization implementation has cached a rejected promise, then all future calls will reject with this same failed promise!

Therefore, it’s very important to evict rejected promises from the memoization cache.

Fortunately, memoizee supports this out of the box. Our final example becomes:

import memoize from 'memoizee';

const getUserById = memoize(async (userId: string): Promise<User> => {
  const user = await request.get(`https://users-service/${userId}`);
  return user;
}, { promise: true});