How to Expire Many Items From a .NET MemoryCache

Question

What is the recommended way to remove a large number of items from a MemoryCache instance?

Based on the discussion around this question it seems that the preferred approach is to use a single cache for the whole application and use namespaces for keys to allow multiple logical types of items to be cached in the same instance.

However, using a single cache instance leaves the problem of expiring (removing) a large number of items from the cache. Particularly in the case where all items of a certain logical type must be expired.

At the moment the only solution I found was based on the answer to this question but it's really not very good from a performance stand-point since you would have to enumerate through all keys in the cache, and test the namespace, which could be quite time-consuming!

The only work-around I came up with at the moment is to create a thin wrapper for all the objects in the cache with a version number and whenever an object is accessed, discard it if the cached version doesn't match the current version. So whenever I need to clear all items of a certain type, I would bump up the current version number rendering all cached items invalid.

The work-around above seems pretty solid. But I can't help but wonder if there isn't a more straight-forward way to accomplish the same?

This is my current implementation:

private class MemCacheWrapper<TItemType> 
              where TItemType : class
{            
  private int _version;
  private Guid _guid;
  private System.Runtime.Caching.ObjectCache _cache;

  private class ThinWrapper
  {
     public ThinWrapper(TItemType item, int version)
     {
        Item = item;
        Version = version;
     }

     public TItemType Item { get; set; }
     public int Version { get; set; }
  }

  public MemCacheWrapper()
  {
      _cache = System.Runtime.Caching.MemoryCache.Default;
      _version = 0;
      _guid = Guid.NewGuid();
  }

  public TItemType Get(int index)
  {                
     string key = string.Format("{0}_{1}", _guid, index);

     var lvi = _cache.Get(key) as ThinWrapper;

     if (lvi == null || lvi.Version != _version)
     {
         return null;
     }

     return lvi.Item;
  }

  public void Put(int index, TItemType item)
  {                
     string key = string.Format("{0}_{1}", _guid, index);

     var cip = new System.Runtime.Caching.CacheItemPolicy();
     cip.SlidingExpiration.Add(TimeSpan.FromSeconds(30));

     _cache.Set(key, new ThinWrapper(item, _version), cip);
  }

  public void Clear()
  {
     _version++;                
  }
}

Answer 1

My recommended way to remove a large number of items from a MemoryCache instance is to use ChangeMonitor, and especially CacheEntryChangeMonitor.

Provides a base class that represents a ChangeMonitor type that can be implemented in order to monitor changes to cache entries.

So, it allows us to handle dependencies between cache items.

A vey basic example is

    var cache = MemoryCache.Default;
    cache.Add("mycachebreakerkey", "mycachebreakerkey", DateTime.Now.AddSeconds(15));

    CacheItemPolicy policy = new CacheItemPolicy();
    policy.ChangeMonitors.Add(cache.CreateCacheEntryChangeMonitor(new string[] { "mycachebreakerkey" }));
    // just to debug removal
    policy.RemovedCallback = args => { Debug.WriteLine(args.CacheItem.Key + "-->" + args.RemovedReason); };
    cache.Add("cacheKey", "cacheKey", policy);

    // after 15 seconds mycachebreakerkey will expire
    // dependent item "cacheKey" will also be removed

As for most of the things, you can also create a custom cache implementation or a derived change monitor type.

Not tested, but the CreateCacheEntryChangeMonitor suggests that you can create dependencies between MemoryCache.

Edit

ChangeMonitor is the .net way to invalidate content in the runtime cache. Invalidate means here = remove from the cache. It's used by SqlDependency or by a few asp.net components to monitor file change. So, I suppose this solution is scalable.

Here is a very simple benchmark, run on my laptop.

        const int NbItems = 300000;

        var watcher = Stopwatch.StartNew();
        var cache = MemoryCache.Default;

        var breakerticks = 0L;
        var allticks = new List<long>();

        cache.Add("mycachebreakerkey", "mycachebreakerkey", new CacheItemPolicy() { RemovedCallback = args => { breakerticks = watcher.ElapsedTicks; } });

        foreach (var i in Enumerable.Range(1, NbItems))
        {
            CacheItemPolicy policy = new CacheItemPolicy();
            if (i % 4 == 0)
                policy.ChangeMonitors.Add(cache.CreateCacheEntryChangeMonitor(new string[] { "mycachebreakerkeyone" }));
            policy.RemovedCallback = args => { allticks.Add(watcher.ElapsedTicks); };// just to debug removal
            cache.Add("cacheKey" + i.ToString(), "cacheKey", policy);
        }

        cache.Remove("mycachebreakerkey");
        Trace.WriteLine("Breaker removal=>" + TimeSpan.FromTicks(breakerticks).TotalMilliseconds);
        Trace.WriteLine("Start removal=>" + TimeSpan.FromTicks(allticks.Min()).TotalMilliseconds);
        Trace.WriteLine("End removal=>" + TimeSpan.FromTicks(allticks.Max()).TotalMilliseconds);
        Trace.WriteLine(cache.GetCount());

        // Trace
        // Breaker removal: 225,8062 ms
        // Start removal: 0,251 ms
        // End removal: 225,7688 ms
        // 225000 items

So, it takes 225 ms to remove 25% of my 300 000 items (again on my laptop, 3 year's old). Do you really need something faster ? Note, that the parent is remove at the end. Advantage of this solution :

• invalidated items are removed from the cache
• you are close to the cache (less callstack, less cast, less indirection)
• the remove callback allow you to auto-reload cache item if needed
• if the cachebreaker expire, then the callback is on another thread which will not impact asp.net requests.

I find your implementation pertinent and will keep it in mind for later. Your choice should be based upon your scenario : number of items, size of cache item, hit ratio, number of dependencies, ... also keeping too much data is the cache is generally slow and can increase probability of eviction.