Memcached + Spring Caching
I'm attempting to setup a Spring 3.1 caching solution with Memcached. I've incorporated ehcache (Spring has built in support for this) successfully. I'm however stuck on a problem with memcached. I apologize for the length in advance (it's mostly boilerplate code)...
I'm using Java configurations, so I've annotated my controller configuration to enable caching as below.
@Configuration @EnableWebMvc @EnableCaching
@ComponentScan("com.ehcache.reference.web")
public class ControllerConfig extends WebMvcConfigurerAdapter {
//Bean to create ViewResolver and add the Resource handler
}
This sets up three controllers which allow for basic CRUD actions on a Stock element. The business object looks like:
public class Stock implements Serializable {
private String name;
private double cost; //This isn't a real app, don't care about correctness of value
//Getters, Setters contructors, etc... left out just a standard POJO
}
I am using MyBatis, so I create a Mapper for the Stock object. The mapper is then injected into my DAO, and the DAO is then injected into the service. The working cache (in both implementations) occurs in the "service" layer. Below is the service layer which utilizes the DAO injected in:
public class TradingServiceImpl implements TradingService {
@Autowired
private final StockDao stockDao;
public TradingServiceImpl(final StockDao stockDao) {
this.stockDao = stockDao;
}
@Override
public void addNewStock(final Stock stock) {
stockDao.save(stock);
}
@Override
@Cacheable(value = "stockCache")
public Stock getStock(final String stockName) {
return stockDao.findByName(stockName);
}
@Override
public List<Stock> getAll() {
return stockDao.findAll();
}
@Override
@CacheEvict(value = "stockCache", key = "#stock.name")
public void removeStock(final Stock stock) {
stockDao.delete(stock);
}
@Override
@CacheEvict(value = "stockCache", key = "#stock.name")
public void updateStock(final Stock stock) {
stockDao.update(stock);
}
}
Here is the sample controller, which is supposed to cache the results of displaying all the stocks (this cache is completely evicted when a stock is added, updated, or deleted from the DB):
@Controller
public class ListAllStocksController {
@Autowired
private TradingService tradingService;
@Cacheable("viewCache")
@RequestMapping(value = "listStocks.html", method = RequestMethod.GET)
public ModelAndView displayAllStocks() {
//The SerializableModelAndView extends Spring's ModelAndView and implements Serializable due to spymemcached not being able to add a non-serializable object to the cache
final ModelAndView mav = new SerializableModelAndView("listStocks");
mav.addObject("stocks", tradingService.getAll());
return mav;
}
@CacheEvict(value = "viewCache", allEntries = true)
@RequestMapping(value = "editStock.html", method = RequestMethod.POST, params = "submit=Edit")
public ModelAndView editStock(final Stock stock, final BindingResult result) {
final ModelAndView mav = new ModelAndView("redirect:listStocks.html");
tradingService.updateStock(stock);
return mav;
}
@CacheEvict(value = "viewCache", allEntries = true)
@RequestMapping(value = "listStocks.html", method = RequestMethod.POST, params = "submit=Delete")
public ModelAndView deleteStockAction(@RequestParam("name") final String name) {
final ModelAndView mav = new ModelAndView("redirect:listStocks.html");
tradingService.removeStock(stock);
return mav;
}
}
Here is my small CacheManager:
public class MemCacheManager extends AbstractCacheManager {
private final Collection<MemCache> internalCaches;
public MemCacheManager(final Collection<MemCache> internalCaches) {
this.internalCaches = internalCaches;
}
@Override
protected Collection<? extends Cache> loadCaches() {
Assert.notNull(internalCaches, "A collection caches is required and cannot be empty");
return internalCaches;
}
}
And here is what my MemCache class looks like:
public class MemCache implements Cache {
private MemcachedClient cache;
private final String name;
private static final Logger LOGGER = Logger.getLogger(MemCache.class);
public MemCache(final String name, final int port) throws URISyntaxException {
this.name = name;
try {
cache = new MemcachedClient(AddrUtil.getAddresses("localhost:" + port));
final SerializingTranscoder stc = (SerializingTranscoder) cache.getTranscoder();
stc.setCompressionThreshold(600000);
} catch (final IOException e) { //Let it attempt to reconnect }
}
@Override
public String getName() {
return name;
}
@Override
public Object getNativeCache() {
return cache;
}
@Override
public ValueWrapper get(final Object key) {
Object value = null;
try {
value = cache.get(key.toString());
} catch (final Exception e) {
LOGGER.warn(e);
}
if (value == null) {
return null;
}
return new SimpleValueWrapper(value);
}
@Override
public void put(final Object key, final Object value) {
cache.set(key.toString(), 7 * 24 * 3600, value);
Assert.assertNotNull(get(key)); //This fails on the viewCache
}
@Override
public void evict(final Object key) {
this.cache.delete(key.toString());
}
@Override
public void clear() {
cache.flush();
}
}
I have tried this with both CouchBase and regular memcached. The setup below shows for memcached being up by itself only.
@Configuration @EnableCaching @Profile("memcached")
public class MemCacheConfiguration implements CachingConfigurer {
@Override @Bean
public CacheManager cacheManager() {
CacheManager cacheManager;
try {
cacheManager = new MemCacheManager(internalCaches());
return cacheManager;
} catch (final URISyntaxException e) {
throw new RuntimeException(e);
}
}
@Bean
public Collection<MemCache> internalCaches() throws URISyntaxException {
final Collection<MemCache> caches = new ArrayList<MemCache>();
// caches.add(new MemCache("stockCache", 11212));
caches.add(new MemCache("viewCache", 11211));
return caches;
}
@Override
public KeyGenerator keyGenerator() {
return new DefaultKeyGenerator();
}
}
In the above example, we are simply going to use memcached. Here is the logging I see when I first start the application and hit the listing controller:
58513 [qtp1656205248-16] TRACE org.springframework.cache.interceptor.CacheInterceptor - Computed cache key 0 for operation CacheableOperation[public org.springframework.web.servlet.ModelAndView com.ehcache.reference.web.ListAllStocksController.displayAllStocks()] caches=[viewCache] | condition='' | key='0'
58519 [qtp1656205248-16] WARN com.memcache.MemCache - Retrieved: null from the cache 'viewCache' at <0> key of type <java.lang.Integer>
58519 [qtp1656205248-16] ERROR com.memcache.MemCache - Returning null
58520 [qtp1656205248-16] DEBUG org.mybatis.spring.SqlSessionUtils - Creating a new SqlSession
58520 [qtp1656205248-16] DEBUG org.mybatis.spring.SqlSessionUtils - SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@640f434f] was not registered for synchronization because synchronization is not active
58520 [qtp1656205248-16] DEBUG org.mybatis.spring.transaction.SpringManagedTransaction - JDBC Connection [org.hsqldb.jdbc.JDBCConnection@260c2adb] will not be managed by Spring
58521 [qtp1656205248-16] DEBUG com.ehcache.reference.dao.StockMapper.findAll - ooo Using Connection [org.hsqldb.jdbc.JDBCConnection@260c2adb]
58521 [qtp1656205248-16] DEBUG com.ehcache.reference.dao.StockMapper.findAll - ==> Preparing: SELECT * from STOCK
58521 [qtp1656205248-16] DEBUG com.ehcache.reference.dao.StockMapper.findAll - ==> Parameters:
58521 [qtp1656205248-16] DEBUG org.mybatis.spring.SqlSessionUtils - Closing non transactional SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@640f434f]
58521 [qtp1656205248-16] WARN com.memcache.MemCache - Setting: ModelAndView: reference to view with name 'listStocks'; model is {stocks=[]} into the cache 'viewCache' at <0> key of type <java.lang.Integer>
58527 [qtp1656205248-16] WARN com.memcache.MemCache - Retrieved: ModelAndView: materialized View is [null]; model is null from the cache 'viewCache' at <0> key of type <java.lang.Integer>
This all looks correct, it's a fresh launch there should be nothing in the cache. I'll now add a stock to the cache:
263036 [qtp1656205248-14] DEBUG org.mybatis.spring.SqlSessionUtils - Creating a new SqlSession
263036 [qtp1656205248-14] DEBUG org.mybatis.spring.SqlSessionUtils - SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@3d20b8d5] was not registered for synchronization because synchronization is not active
263038 [qtp1656205248-14] DEBUG org.mybatis.spring.transaction.SpringManagedTransaction - JDBC Connection [org.hsqldb.jdbc.JDBCConnection@40b5f9bb] will not be managed by Spring
263038 [qtp1656205248-14] DEBUG com.ehcache.reference.dao.StockMapper.save - ooo Using Connection [org.hsqldb.jdbc.JDBCConnection@40b5f9bb]
263038 [qtp1656205248-14] DEBUG com.ehcache.reference.dao.StockMapper.save - ==> Preparing: INSERT INTO STOCK (name, cost) VALUES (?, ?)
263039 [qtp1656205248-14] DEBUG com.ehcache.reference.dao.StockMapper.save - ==> Parameters: A(String), 1.0(Double)
263039 [qtp1656205248-14] DEBUG org.mybatis.spring.SqlSessionUtils - Closing non transactional SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@3d20b8d5]
263039 [qtp1656205248-14] TRACE org.springframework.cache.interceptor.CacheInterceptor - Invalidating cache key 0 for operation CacheEvictOperation[public org.springframework.web.servlet.ModelAndView com.ehcache.reference.web.AddStockController.addNewStock(com.ehcache.reference.business.Stock,org.springframework.validation.BindingResult)] caches=[viewCache] | condition='' | key='0',false,false on method public org.springframework.web.servlet.ModelAndView com.ehcache.reference.web.AddStockController.addNewStock(com.ehcache.reference.business.Stock,org.springframework.validation.BindingResult)
263039 [qtp1656205248-14] WARN com.memcache.MemCache - Evicting value at <0> in cache 'viewCache'
263049 [qtp1656205248-18] TRACE org.springframework.cache.interceptor.CacheInterceptor - Computed cache key 0 for operation CacheableOperation[public org.springframework.web.servlet.ModelAndView com.ehcache.reference.web.ListAllStocksController.displayAllStocks()] caches=[viewCache] | condition='' | key='0'
263051 [qtp1656205248-18] WARN com.memcache.MemCache - Retrieved: null from the cache 'viewCache' at <0> key of type <java.lang.Integer>
263051 [qtp1656205248-18] ERROR com.memcache.MemCache - Returning null
263051 [qtp1656205248-18] DEBUG org.mybatis.spring.SqlSessionUtils - Creating a new SqlSession
263051 [qtp1656205248-18] DEBUG org.mybatis.spring.SqlSessionUtils - SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@7050d0a7] was not registered for synchronization because synchronization is not active
263051 [qtp1656205248-18] DEBUG org.mybatis.spring.transaction.SpringManagedTransaction - JDBC Connection [org.hsqldb.jdbc.JDBCConnection@49b2bd8c] will not be managed by Spring
263051 [qtp1656205248-18] DEBUG com.ehcache.reference.dao.StockMapper.findAll - ooo Using Connection [org.hsqldb.jdbc.JDBCConnection@49b2bd8c]
263051 [qtp1656205248-18] DEBUG com.ehcache.reference.dao.StockMapper.findAll - ==> Preparing: SELECT * from STOCK
263052 [qtp1656205248-18] DEBUG com.ehcache.reference.dao.StockMapper.findAll - ==> Parameters:
263053 [qtp1656205248-18] DEBUG org.mybatis.spring.SqlSessionUtils - Closing non transactional SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@7050d0a7]
263053 [qtp1656205248-18] WARN com.memcache.MemCache - Setting: ModelAndView: reference to view with name 'listStocks'; model is {stocks=[Stock Name: <A> Current Cost <1.0>]} into the cache 'viewCache' at <0> key of type <java.lang.Integer>
263055 [qtp1656205248-18] WARN com.memcache.MemCache - Retrieved: ModelAndView: materialized View is [null]; model is null from the cache 'viewCache' at <0> key of type <java.lang.Integer>
Minus the last line, that all looks correct. Here is my output from memcached --vv:
<156 get 0 ##Initial listing of all
>156 END
<156 set 0 1 604800 79 ##Cache initial result on startup
>156 STORED
<156 get 0
>156 sending key 0
>156 END
<156 delete 0 ##Invalidation when stock added
>156 DELETED
<156 get 0 ##Add redirects to the get page
>156 END
<156 set 0 1 604800 79 ##Store the new value
>156 STORED
<156 get 0
>156 sending key 0 ##Refresh the page
>156 END
The unfun bit here is that when I add a stock to the system, I am redirected to the displayAllStocks method. It initially does this correctly, however if I refresh the page, I am sent the original version (the one which shows no stocks). I'm quite stuck here, and am unsure of what could be causing this issue at this point. If I invalidate the cache in any way, the redirect works correctly. It's on subsequent refreshes that I retrieve what seems to be the first value put in (which was deleted)
Is this a configuration issue? A limitation/bug in memcache or spymemcached or simply a bug in my MemCache code?
Answer
In TradingServiceImpl the method getStock always returns the same object from cache independently of stock name. You should change it to:
@Override
@Cacheable(value = "stockCache", key="#stockName")
public Stock getStock(final String stockName) {
return stockDao.findByName(stockName);
}
Where do you evict viewCache? I don't see it in your code.
If you want to use memcached over Spring 3.1 cache abstraction you may tray Simple-Spring-Memcached (current snapshot version - you need to build it from sources). Here you can find sample configuration how to integrate memcached with Spring Cache using SSM.
Sample project: spring-cache-integration-test using this integration is available in SSM svn.
UPDATE: Simple Spring Memcached 3.0.0 with Spring Cache integration is already available.