.NET Caching: Can a One Second Cache Really Make a Difference?

Cache invalidation is hard, but if the data allows, using low TTLs can simplify it by automatically expiring cached items instead of requiring explicit invalidation on updates.

How low can we set our cache expiration timeouts to be while still providing performance benefits?

Obviously it depends on a number of different factors but this post uses Azure Load Testing to demonstrate that even a one-second cache can still significantly boost performance.

Test setup

The two .NET 9 Minimal API endpoints I tested both hit the DB for a list of 1000 dummy currencies using Entity Framework with AsNoTracking().

The in-memory-cache endpoint caches the currencies for one second.

Click on the image for a larger in a new window.


The app was deployed to Azure and used these resources:
App Service instance in North Europe – Free F1 Service Tier.
SQL DB instance in North Europe – Basic Service Tier (5 DTUs).

Azure Load Testing setup

The Azure Load Test instance was located in North Europe.

I configured two tests, one for each endpoint with the following load configuration settings:


I also added some server side metrics to the tests to capture CPU utilization of the Azure SQL DB:


The tests were ran sequentially, not in parallel for the most accurate results.

Azure Load Testing results

The P90 Performance Table below shows us that even with a cache expiration of only one second the in-memory endpoint still performs significantly better than the no cache endpoint.


We can see a massive 87% reduction in response time and a 876% increase in throughput when currencies are cached for just one second.

In an approximately three minute run, the no cache endpoint (in pink) could only handle around 11K total requests, while the average for both test runs was ~60K:


The P90 response time was 119ms for the in memory cache endpoint, while the average for both test runs was 520ms:


The graph below shows the in memory cache endpoint reached a throughput of almost 600 requests per second, while the no cache endpoint reached only 60 requests per second.

The average throughput across both tests was ~323 requests per second.

Database CPU utilization

We can see, even with just a one second expiration time on our cache, DB CPU usage is still dramatically reduced:

Conclusion

Short-lived caching can significantly boost performance, even with timeouts as brief as a second. It reduces database load, improves response times, and enhances scalability. This approach often eliminates complex cache invalidation logic, simplifying code and enabling in-memory caching even in multi-instance deployments.

GitHub repo with endpoints used above.

Leave a Reply

Your email address will not be published. Required fields are marked *