Is there any good known solution for a fast persistent cache?

java caching persistent
tomasb picture tomasb · Aug 20, 2011 · Viewed 11.7k times · Source

I need really fast and persistent cache for my web crawler. It doesnt need to be as fast as ConcurrentSkipListSet in Java, but definitely it cannot be MySQL with hash-index based table, which i tried. After 1m+ of records it takes like 80% of processor time.

Does any one know or heard of something useful for this case?
Thanks for any hint.

Answer

skaffman picture skaffman · Aug 20, 2011

Try EhCache. It's a primarily in-memory cache with options for overflow and persistence to disk backing store. Been around for years, still actively developed, and very mature.

Related questions

Looking for simple Java in-memory cache

I'm looking for a simple Java in-memory cache that has good concurrency (so LinkedHashMap isn't good enough), and which can be serialized to disk periodically. One feature I need, but which has proved hard to find, is a way to "…

Read More
Java time-based map/cache with expiring keys

Do any of you know of a Java Map or similar standard data store that automatically purges entries after a given timeout? This means aging, where the old expired entries “age-out” automatically. Preferably in an open source library that is …

Read More
How would you implement an LRU cache in Java?

Please don't say EHCache or OSCache, etc. Assume for purposes of this question that I want to implement my own using just the SDK (learning by doing). Given that the cache will be used in a multithreaded environment, which datastructures …

Read More