Skip to main content

Adding Cross-Cutting Memory Caching to an HttpClientFactory in ASP.NET Core with Polly

5480805977_27d92598ca_oCouple days ago I Added Resilience and Transient Fault handling to your .NET Core HttpClient with Polly. Polly provides a way to pre-configure instances of HttpClient which apply Polly policies to every outgoing call. That means I can say things like "Call this API and try 3 times if there's any issues before you panic," as an example. It lets me move the cross-cutting concerns - the policies - out of the business part of the code and over to a central location or module, or even configuration if I wanted.

I've been upgrading my podcast of late at https://www.hanselminutes.com to ASP.NET Core 2.1 and Razor Pages with SimpleCast for the back end. Since I've removed SQL Server as my previous data store and I'm now sitting entirely on top of a third party API I want to think about how often I call this API. As a rule, there's no reason to call it any more often that a change might occur.

I publish a new show every Thursday at 5pm PST, so I suppose I could cache the feed for 7 days, but sometimes I make description changes, add links, update titles, etc. The show gets many tens of thousands of hits per episode so I definitely don't want to abuse the SimpleCast API, so I decided that caching for 4 hours seemed reasonable.

I went and wrote a bunch of caching code on my own. This is fine and it works and has been in production for a few months without any issues.

A few random notes:

  • Stuff is being passed into the Constructor by the IoC system built into ASP.NET Core
    • That means the HttpClient, Logger, and MemoryCache are handed to this little abstraction. I don't new them up myself
  • All my "Show Database" is, is a GetShows()
    • That means I have TestDatabase that implements IShowDatabase that I use for some Unit Tests. And I could have multiple implementations if I liked.
  • Caching here is interesting.
    • Sure I could do the caching in just a line or two, but a caching double check is more needed that one often realizes.
    • I check the cache, and if I hit it, I am done and I bail. Yay!
    • If not, Let's wait on a semaphoreSlim. This a great, simple way to manage waiting around a limited resource. I don't want to accidentally have two threads call out to the SimpleCast API if I'm literally in the middle of doing it already.
      • "The SemaphoreSlim class represents a lightweight, fast semaphore that can be used for waiting within a single process when wait times are expected to be very short."
    • So I check again inside that block to see if it showed up in the cache in the space between there and the previous check. Doesn't hurt to be paranoid.
    • Got it? Cool. Store it away and release as we finally the try.

Don't copy paste this. My GOAL is to NOT have to do any of this, even though it's valid.

public class ShowDatabase : IShowDatabase

{

    private readonly IMemoryCache _cache;

    private readonly ILogger _logger;

    private SimpleCastClient _client;


    public ShowDatabase(IMemoryCache memoryCache,

            ILogger<ShowDatabase> logger,

            SimpleCastClient client)

    {

        _client = client;

        _logger = logger;

        _cache = memoryCache;

    }


    static SemaphoreSlim semaphoreSlim = new SemaphoreSlim(1);


    static HttpClient client = new HttpClient();


    public async Task<List<Show>> GetShows()

    {

        Func<Show, bool> whereClause = c => c.PublishedAt < DateTime.UtcNow;


        var cacheKey = "showsList";

        List<Show> shows = null;


        //CHECK and BAIL - optimistic

        if (_cache.TryGetValue(cacheKey, out shows))

        {

            _logger.LogDebug($"Cache HIT: Found {cacheKey}");

            return shows.Where(whereClause).ToList();

        }


        await semaphoreSlim.WaitAsync();

        try

        {

            //RARE BUT NEEDED DOUBLE PARANOID CHECK - pessimistic

            if (_cache.TryGetValue(cacheKey, out shows))

            {

                _logger.LogDebug($"Amazing Speed Cache HIT: Found {cacheKey}");

                return shows.Where(whereClause).ToList();

            }


            _logger.LogWarning($"Cache MISS: Loading new shows");

            shows = await _client.GetShows();

            _logger.LogWarning($"Cache MISS: Loaded {shows.Count} shows");

            _logger.LogWarning($"Cache MISS: Loaded {shows.Where(whereClause).ToList().Count} PUBLISHED shows");


            var cacheExpirationOptions = new MemoryCacheEntryOptions();

            cacheExpirationOptions.AbsoluteExpiration = DateTime.Now.AddHours(4);

            cacheExpirationOptions.Priority = CacheItemPriority.Normal;


            _cache.Set(cacheKey, shows, cacheExpirationOptions);

            return shows.Where(whereClause).ToList(); ;

        }

        catch (Exception e)

        {

            _logger.LogCritical("Error getting episodes!");

            _logger.LogCritical(e.ToString());

            _logger.LogCritical(e?.InnerException?.ToString());

            throw;

        }

        finally

        {

            semaphoreSlim.Release();

        }

    }

}


public interface IShowDatabase

{

    Task<List<Show>> GetShows();

}

Again, this is great and it works fine. But the BUSINESS is in _client.GetShows() and the rest is all CEREMONY. Can this be broken up? Sure, I could put stuff in a base class, or make an extension method and bury it in there, so use Akavache or make a GetOrFetch and start passing around Funcs of "do this but check here first":

IObservable<T> GetOrFetchObject<T>(string key, Func<Task<T>> fetchFunc, DateTimeOffset? absoluteExpiration = null);

Could I use Polly and refactor via subtraction?

Per the Polly docs:

The Polly CachePolicy is an implementation of read-through cache, also known as the cache-aside pattern. Providing results from cache where possible reduces overall call duration and can reduce network traffic.

First, I'll remove all my own caching code and just make the call on every page view. Yes, I could write the Linq a few ways. Yes, this could all be one line. Yes, I like Logging.

public async Task<List<Show>> GetShows()

{

    _logger.LogInformation($"Loading new shows");

    List<Show> shows = await _client.GetShows();

    _logger.LogInformation($"Loaded {shows.Count} shows");

    return shows.Where(c => c.PublishedAt < DateTime.UtcNow).ToList(); ;

}

No caching, I'm doing The Least.

Polly supports both the .NET MemoryCache that is per process/per node, an also .NET Core's IDistributedCache for having one cache that lives somewhere shared like Redis or SQL Server. Since my podcast is just one node, one web server, and it's low-CPU, I'm not super worried about it. If Azure WebSites does end up auto-scaling it, sure, this cache strategy will happen n times. I'll switch to Distributed if that becomes a problem.

I'll add a reference to Polly.Caching.MemoryCache in my project.

I ensure I have the .NET Memory Cache in my list of services in ConfigureServices in Startup.cs:

services.AddMemoryCache();

STUCK...for now!

AND...here is where I'm stuck. I got this far into the process and now I'm either confused OR I'm in a Chicken and the Egg Situation.

Forgive me, friends, and Polly authors, as this Blog Post will temporarily turn into a GitHub Issue. Once I work through it, I'll update this so others can benefit. And I still love you; no disrespect intended.

The Polly.Caching.MemoryCache stuff is several months old, and existed (and worked) well before the new HttpClientFactory stuff I blogged about earlier.

I would LIKE to add my Polly Caching Policy chained after my Transient Error Policy:

services.AddHttpClient<SimpleCastClient>().

    AddTransientHttpErrorPolicy(policyBuilder => policyBuilder.CircuitBreakerAsync(

            handledEventsAllowedBeforeBreaking: 2,

            durationOfBreak: TimeSpan.FromMinutes(1)

    )).

    AddPolicyHandlerFromRegistry("myCachingPolicy"); //WHAT I WANT?

However, that policy hasn't been added to the Policy Registry yet. It doesn't exist! This makes me feel like some of the work that is happening in ConfigureServices() is a little premature. ConfigureServices() is READY, AIM and Configure() is FIRE/DO-IT, in my mind.

If I set up a Memory Cache in Configure, I need to use the Dependency System to get the stuff I want, like the .NET Core IMemoryCache that I put in services just now.

public void Configure(IApplicationBuilder app, IPolicyRegistry<string> policyRegistry, IMemoryCache memoryCache)

{

    MemoryCacheProvider memoryCacheProvider = new MemoryCacheProvider(memoryCache);

    var cachePolicy = Policy.CacheAsync(memoryCacheProvider, TimeSpan.FromMinutes(5));

    policyRegistry.Add("cachePolicy", cachePolicy);

    ...

But at this point, it's too late! I need this policy up earlier...or I need to figure a way to tell the HttpClientFactory to use this policy...but I've been using extension methods in ConfigureServices to do that so far. Perhaps some exception methods are needed like AddCachingPolicy(). However, from what I can see:

  • This code can't work with the ASP.NET Core 2.1's HttpClientFactory pattern...yet. https://github.com/App-vNext/Polly.Caching.MemoryCache
  • I could manually new things up, but I'm already deep into Dependency Injection...I don't want to start newing things and get into scoping issues.
  • There appear to be changes between  v5.4.0 and 5.8.0. Still looking at this.
  • Bringing in the Microsoft.Extensions.Http.Polly package brings in Polly-Signed 5.8.0...

I'm likely bumping into a point in time thing. I will head to bed and return to this post in a day and see if I (or you, Dear Reader) have solved the problem in my sleep.

"code making and code breaking" by earthlightbooks - Licensed under CC-BY 2.0 - Original source via Flickr

Sponsor: Check out JetBrains Rider: a cross-platform .NET IDE. Edit, refactor, test and debug ASP.NET, .NET Framework, .NET Core, Xamarin or Unity applications. Learn more and download a 30-day trial!


© 2018 Scott Hanselman. All rights reserved.
     


from Scott Hanselman's Blog http://feeds.hanselman.com/~/542148714/0/scotthanselman~Adding-CrossCutting-Memory-Caching-to-an-HttpClientFactory-in-ASPNET-Core-with-Polly.aspx
Labels:

Comments

Post a Comment

Popular posts from this blog

dotnet sdk list and dotnet sdk latest

Can someone make .NET Core better with a simple global command? Fanie Reynders did and he did it in a simple and elegant way. I'm envious, in fact, because I spec'ed this exact thing out in a meeting a few months ago but I could have just done it like he did and I would have used fewer keystrokes! Last year when .NET Core was just getting started, there was a "DNVM" helper command that you could use to simplify dealing with multiple versions of the .NET SDK on one machine. Later, rather than 'switching global SDK versions,' switching was simplified to be handled on a folder by folder basis. That meant that if you had a project in a folder with no global.json that pinned the SDK version, your project would use the latest installed version. If you liked, you could create a global.json file and pin your project's folder to a specific version. Great, but I would constantly have to google to remember the format for the global.json file, and I'd constan
Post a Comment
Read more

R vs Python for Machine Learning

There are so many things to learn before to choose which language is good for Machine Learning. We will discuss each and everything about R as well as Python and the situation or problem in which situation we have to use which language. Let’s start Python and R are the two most Commonly used Programming Languages for Machine Learning and because of the popularity of both the languages Novice or you can say fresher are getting confused, whether they should choose R or Python language to commence their career in the Machine learning domain. Don’t worry guys through this article we will discuss R vs Python for Machine Learning. So, without exaggerating this article let’s get started. We will start it from the very Basics things or definitions. R vs Python for Machine Learning Introduction R is a programming language made by statisticians and data miners for statistical analysis and graphics supported by R foundation for statistical computing. R also provides high-quality graphics and
1 comment
Read more

Top Tips For PCB Design Layout

Are you thinking about designing a printed circuit board? PCBs are quite complicated, and you need to make sure that the layout that you choose is going to operate as well as you want it to. For this reason, we have put together some top tips for PCB design layout. Keep reading if you would like to find out more about this. Leave Enough Space One of the most important design tips for PCB layout is that you need to make sure that you are leaving enough space between the components. While many people might think that packing components closely is the best route to take, this can cause problems further down the line. This is why we suggest leaving extra space for the wires that will spread. This way, you’ll have the perfect PCB design layout. Print Out Your Layout Struggling to find out if your components sizes match? Our next tip is to print out your layout and compare the printed version to your actual components. Datasheets can sometimes come with errors, so it doesn’t hurt to do
1 comment
Read more