Which key:value store to use with Python?
So I'm looking at various key:value (where value is either strictly a single value or possibly an object) stores for use with Python, and have found a few promising ones. I have no specific requirement as of yet because I am in the evaluation phase. I'm looking for what's good, what's bad, what are the corner cases these things handle well or don't, etc. I'm sure some of you have already tried them out so I'd love to hear your findings/problems/etc. on the various key:value stores with Python. I'm looking primarily at:
memcached - http://www.danga.com/memcached/ python clients: http://pypi.python.org/pypi/python-memcached/1.40 http://www.tummy.com/Community/software/python-memcached/
CouchDB - http://couchdb.apache.org/ python clients: http://code.google.com/p/couchdb-python/
Tokyo Tyrant - http://1978th.net/tokyotyrant/ python clients: http://code.google.com/p/pytyrant/
Lightcloud - http://opensource.plurk.com/LightCloud/ Based on Tokyo Tyrant, written in Python
Redis - http://redis.io/ python clients: http://pypi.python.org/pypi/txredis/0.1.1
MemcacheDB - http://memcachedb.org/
So I started benchmarking (simply inserting keys and reading them) using a simple count to generate numeric keys and a value of "A short string of text":
memcached: CentOS 5.3/python-2.4.3-24.el5_3.6, libevent 1.4.12-stable, memcached 1.4.2 with default settings, 1 gig memory, 14,000 inserts per second, 16,000 seconds to read. No real optimization, nice.
memcachedb claims on the order of 17,000 to 23,000 inserts per second, 44,000 to 64,000 reads per second.
I'm also wondering how the others stack up speed wise.
Answer
That mostly depends on your need.
Read Caveats of Evaluating Databases to understand how to evaluate them.