Multi-node of same service VS different service

[riccardomerlin]

I'm working with a microservices system and I want to add Fusion Cache for the Authorization service.
Since the Authorization service is called by all the other services for obvious reasons, this generates a high network traffic, putting the network under stress and response time can becomes a bottleneck.

I'm thinking to create a component with Fusion Cache for the authorization service that is then used by all the consumer services so that each service have their own L1 memory cache + L2 with Redis when looking for fetching Authorization information.
This way, I'm hoping to optimize the system reducing drastically the network traffic. Also, setting properly factory timeouts and eager refresh, I can optimize the response times when there is a refresh needed.

So, unlike the typical scenarios described in the docs, I'm pushing the cache one more step toward the consumer, effectively considering the Authorization service a "slow" repository of data and using the cache to boost performance.
In this configuration, the cache nodes are all the microservices, each one of them deployed in a kubernetes cluster that manages replicas.

Given I couldn't find any specific examples of this configuration in the docs, I'd like to know if this is still considered a supported multi-node scenario and how should I configure the Fusion Cache so that each separate service can read the most up-to-date cache values?
At the same time, I want the Authorization service to update/expire cache entries when receiving update requests.
Basically, I want to be able to have a shared hybrid cache that all services can use.

I'm working on a PoC atm and I've noticed that using the same Redis InstanceName seems to help the syncing. Am I on the right track?
Here is a snippet of the configuration that each service is going to use through the shared library I'm developing:

         var serializer = new FusionCacheSystemTextJsonSerializer();

         services.AddStackExchangeRedisCache(options =>
         {
            options.ConfigurationOptions = redisConfiguration;
            options.InstanceName = "my-fusion-cache";
         });

         services.AddFusionCache()
            .WithDefaultEntryOptions(o =>
            {
               o.Duration = TimeSpan.FromSeconds(60);
               o.AllowBackgroundBackplaneOperations = true;
               o.AllowBackgroundDistributedCacheOperations = true;
               o.IsFailSafeEnabled = true;
               o.DistributedCacheSoftTimeout = TimeSpan.FromSeconds(5);
               o.FailSafeThrottleDuration = TimeSpan.FromMinutes(120);
               configureDefaultEntry?.Invoke(o);
            })
            .WithSerializer(serializer)
            .WithDistributedCache(sp =>
               sp.GetRequiredService<IDistributedCache>(), serializer)
            .WithStackExchangeRedisBackplane(options =>
               options.ConfigurationOptions = redisConfiguration);

As you can see, my-fusion-cache is the Redis instance name I'm leveraging to ensure all services can use the same cache. If I'm getting this right, the backplane then should do its magic to ensure all is synced up.

When reading the cache from any service, I have something like this:

            var statusCode = await _fusionCache.GetOrSetAsync<int>(cacheKey,
               async (_, ct) =>
               {
                  factoryExecuted = true;
                  fullResponse = await httpClient.GetAsync(requestUri, HttpCompletionOption.ResponseHeadersRead, ct);
                  return (int)fullResponse.StatusCode;
               },
               options => options
                  .SetDuration(TimeSpan.FromMinutes(60))
                  .SetFactoryTimeouts(TimeSpan.FromMilliseconds(5))
                  .SetEagerRefresh(0.8f)
            );

I just need to store the response code for now in the cache.
The _fusionCache is the default instance but I'm thinking to create a better abstraction around the shared library so that I can use a named instance instead for the cache, to ensure separation in case of other cache instances being used in the system.

On an update request, I'm setting the expiration by simply doing something like this in the authn service. Expire instead of Set works better for the particular implementation of the Authz service I have.

await _fusionCache.ExpireAsync(cacheKey);

Just to clarify, all updates will always be propagated by the Fusion Cache instance in the authorization service. This may or may not make a difference, since the backplane should ensure the syncing happens for updates from any node, but I thought to specify that anyway.

Any advice would be greatly appreciated!🙏 🙇
Thanks for building and maintaining this fantastic library, as an Italian myself, I'm also proud that this is another great Italian product! 🇮🇹

[riccardomerlin]

I thought to add a schema to better explain how I think my architecture would work.
The blue lines are GET requests that read from each service own cache first, where the first request gets data from the Authz service to populate the cache.
The green lines are requests that UPDATE some data so the cache needs to be invalidated.
The orange lines represents the background work that the backplane does to keep all Fusion Cache instances in sync.
Now, connecting this diagram to my initial question, all Fusion Cache instances share the same InstanceName for Redis, which should guarantee the syncing according to my theory.

I hope this helps clarifying my scenario.

[jodydonetti]

Hi @riccardomerlin , sorry for the delay, busy with the second course (Deep Dive).

I'll answer you this afternoon!

[jodydonetti]

Hi @riccardomerlin

the other day's afternoon came and went, so here we are 😅

I'm working with a microservices system and I want to add Fusion Cache for the Authorization service.
[...]
I'm thinking to create a component with Fusion Cache for the authorization service that is then used by all the consumer services so that each service have their own L1 memory cache + L2 with Redis when looking for fetching Authorization information.
[...]
Given I couldn't find any specific examples of this configuration in the docs, I'd like to know if this is still considered a supported multi-node scenario and how should I configure the Fusion Cache so that each separate service can read the most up-to-date cache values?

From my understanding of your description: yes, totally.

What is normally thought of as "the database" can be anything, really.
Any source of information potentially slow or unreliable (even just sometimes, in a transient way) can be "the database", meaning you can put a cache in front of it and, in caso of FusionCache, add on top Fail-Safe, Eager Refresh, Factory Timeouts, etc to cover transient issues.

Each app instance has its own L1 (for super fast access), and all are connected to a shared L2 and backplane (for sharing cached data and as the source of data after receiving a backplane notification).

At the same time, I want the Authorization service to update/expire cache entries when receiving update requests. Basically, I want to be able to have a shared hybrid cache that all services can use.

Yup, that's how it works: when you do, for example, a cache.Set(...) call on a node/pod/instance, that call will update the local L1 (memory) and the shared L2 (Redis), then the backplane will send a notification to all the other nodes to warn them that "this thing changed, do your thing".

And with this flow, all is taken care of.

I'm working on a PoC atm and I've noticed that using the same Redis InstanceName seems to help the syncing. Am I on the right track?

Not really, but let me clarify.

The InstanceName you are referring to is this, and as explained there it is just a prefix that will be added to each cache key, it's not really a separate instance or something like that.

This is not bad, mind you, it's just something that works specifically for the RedisCache impl of IDistributedCache.

A similar but better solution, that will work holistically in FusionCache and for "all the things" is the use of Named Caches and CacheKeyPrefix, see here.

Basically, by creating a Named Cache + setting the CacheKeyPrefix at the FusionCache level, you are in fact creating a proper separation for the entries related to one "logical cache" in the correct way. This will also take into account everything else potentially related, like Tagging, Clear support and more.

Here is a snippet of the configuration that each service is going to use through the shared library I'm developing:

A couple of things here.

     services.AddStackExchangeRedisCache(options =>
     {
        options.ConfigurationOptions = redisConfiguration;
        options.InstanceName = "my-fusion-cache";
     });

By doing this, you are registering this RedisCache as the global IDistributedCache one: this means that if some other component will ask for IDistributedCache they'll get this one, configured with the specific InstanceName, not sure this is what you want to do.

        .WithDefaultEntryOptions(o =>
        {
           o.Duration = TimeSpan.FromSeconds(60);

[...]
o.FailSafeThrottleDuration = TimeSpan.FromMinutes(120);

Not sure about this: 60s duration and 2h of fail-safe throttle duration?

Translation of this config: "cache stuff or 60s, and if there's a transient issue during the refresh, put the old value back in the cache for 2h".
Maybe you meant to configure the MaxDuration instead?

        .WithDistributedCache(sp =>
           sp.GetRequiredService<IDistributedCache>(), serializer)

If you decide to stay with the IDistributedCache registered in the DI container (see my point above) this works, but 2 things:

• in this case you can just call WithRegisteredDistributedCache() which does the same thing but is shorter and - imho - more explicit
• you are specifying again the serializer: this will not create issues (the second one specified will overwrite the first one, which in your case is also the same one) but may create confusionfor other team members + will leave the door open for bugs with future changes/refactoring

If instead you decide not to register and use one global IDistributedCache, you can simply call WithDistributedCache(new RedisCache(...)) and specify one directly for the FusionCache use case, so it will not get somewhere else in the DI container, inadvertently used by other DI consumers.

        .WithStackExchangeRedisBackplane(options =>
           options.ConfigurationOptions = redisConfiguration);

One more trick I can suggest: since the IConnectionMultiplexer is quite heavy and is suggested to be reused as much as possible, instead of using ConfigurationOptions you create an IConnectionMultiplexer isntance first, and then pass it to both the distributed cache and the backplane, by using the ConnectionMultiplexerFactory.

(look at the end a sample of the resulting code)

As you can see, my-fusion-cache is the Redis instance name I'm leveraging to ensure all services can use the same cache.

I know I said it already, but just to highlight the previous point more, since it's important: they are using the "same cache" anyway, it's just a prefix.

If I'm getting this right, the backplane then should do its magic to ensure all is synced up.

This is correct!

When reading the cache from any service, I have something like this:

            var statusCode = await _fusionCache.GetOrSetAsync<int>(cacheKey,
               async (_, ct) =>
               {
                  factoryExecuted = true;
                  fullResponse = await httpClient.GetAsync(requestUri, HttpCompletionOption.ResponseHeadersRead, ct);
                  return (int)fullResponse.StatusCode;
               },
               options => options
                  .SetDuration(TimeSpan.FromMinutes(60))
                  .SetFactoryTimeouts(TimeSpan.FromMilliseconds(5))
                  .SetEagerRefresh(0.8f)
            );

Looks good to me!

I just need to store the response code for now in the cache. The _fusionCache is the default instance but I'm thinking to create a better abstraction around the shared library so that I can use a named instance instead for the cache, to ensure separation in case of other cache instances being used in the system.

Ah, tere you have it, you already thought about it 😅

On an update request, I'm setting the expiration by simply doing something like this in the authn service. Expire instead of Set works better for the particular implementation of the Authz service I have.

Just to be clear:

• Set: updates the entry in the cache with a new value
• Remove: removes the entry from the cache
• Expire: mark the entry as expired, so it's logically expired but still there in case of future need with Fail-Safe

Just to clarify, all updates will always be propagated by the Fusion Cache instance in the authorization service.
This may or may not make a difference, since the backplane should ensure the syncing happens for updates from any node, but I thought to specify that anyway.

I'm not sure I 100% understood this point.

Anyway, the resulting setup code I mentioned above:

// CREATE THE MULTIPLEXER, SPECIFY OPTIONS, ETC
IConnectionMultiplexer conn = ... ;

// LATER...
services.AddFusionCache("AuthCache")
  // CACHE KEY PREFIX
  .WithCacheKeyPrefixByCacheName()
  // SERIALIZER
  .WithSerializer(new FusionCacheSystemTextJsonSerializer())
  // L2
  .WithDistributedCache(
    new RedisCache(new RedisCacheOptions
    {
      ConnectionMultiplexerFactory = async () => conn
    })
  )
  // BACKPLANE
  .WithBackplane(
    new RedisBackplane(new RedisBackplaneOptions
    {
      ConnectionMultiplexerFactory = async () => conn
    })
  );

Of course you can add here all the rest like specific options, etc.

Thanks for building and maintaining this fantastic library

Thanks to you for using it, I hope it will help with yout project!

as an Italian myself, I'm also proud that this is another great Italian product!

Ahah grazie 🤌, apprezzo molto.

Hope this helps, let me know!

[riccardomerlin]

Hi @jodydonetti ,
thanks a lot for the detailed reply, it was definitely worth the wait!

The main concern for me was how I was using the redis instance name, named cache + cache prefix is what I need.
The tip on the IConnectionMultiplexer is definitely something I didn't know, thanks for pointing that out.

Not sure about this: 60s duration and 2h of fail-safe throttle duration?

Regarding this, it looked weird to me too. I tried to copy this example I think I mixed up the parameters, I will take a better look in the final implementation for sure.

The final full example is just class! 🙇

You 've answered very clearly and I know what to do now. 🙏

Ciao e grazie ancora.

[jodydonetti]

Hi @jodydonetti , thanks a lot for the detailed reply, it was definitely worth the wait!
[...]
The final full example is just class! 🙇

You 've answered very clearly and I know what to do now. 🙏

Happy it helped!