Approach caching with caution.

In this post, I’d like to describe some important considerations that I think teams should consider before applying a caching solution. I’ve found:

  • Caching is often applied too early.
  • Caching can be a slippery slope.
  • Caching is hard to do right.

Often Applied Too Early

Caching is often applied too early. It’s wise to try other alternatives first.

Here’s an example: suppose we have an order-management system that fetches all the orders, and it’s performing slowly. Should we cache it?

We could. But first, here are some good questions.

  • Is our query optimized on the frontend, API layer, and data layer?
  • Could we apply a default filter to limit results? Perhaps showing recent orders, rather than all?
  • Could we be lazy? Implement client-side lazy loading, or server-side pagination?

These are describing the question we are asking. We want that question to be as efficient as possible before we try to optimize how frequently we ask the question.

On many teams, caching is applied before these options are explored.

Can Be a Slippery Slope

A second challenge is that caching can be a slippery slope: once you cache, it’s easy to cache again.

Back to our example, now we’ve done some caching, and we’re building a page that shows all the customers who use our product. Before we do any benchmarking, what tool is likely to be applied to that query? Caching. Why? Because that’s what we use for APIs in this codebase. If it’s even a deliberate decision, and many times it isn’t via well-meaning copy-pasta, it might not be a reasoned one.

Ideally, when a new API is added, the merits of caching are discussed in the open from first principles. Is the complexity of adding caching to this endpoint correct, right now?

Hard To Do Right

A third challenge is that caching is hard.

Phil Karlton taught us that cache invalidation is considered one of the two hard problems in computer science. It’s additional complexity. Invite complexity into your codebase with caution.

Back to our example app. Consider a security issue. We discover that our customer page leaks sensitive PII. Without caching, we can fix the issue and force a page reload for all clients. With caching, we must ensure that we invalidate every cache, everywhere.

And whoops, we’ve gone for it: perhaps we have caching at the data layer, API layer, and frontend. Can you invalidate all those caches while maintaining data integrity and a decent user experience?

Security issues aside, caching can be a vector for sneaky bugs, both to find and reproduce. It’s another place where data is stored, another tool to manipulate.

And it can create UX problems when users expect data to be updated more often than it is. Yes, customers want fast applications, but not at the expense of a coherent user experience.

Conclusion

I feel drawn to this argument because caching is so widespread and accepted. A tool can be “blazing-fast” and still be wrong for your team, today.

Sometimes, you gotta cache. Make sure that isn’t being applied too early, it isn’t spreading through your codebase, and you’re trying hard to do it right.

Cache away— when you’ve exhausted other options and are willing to accept the tradeoffs.