Embedded non-relational (nosql) data store

Igor Brejc picture Igor Brejc · Jan 17, 2010 · Viewed 8.6k times · Source

I'm thinking about using/implementing some kind of an embedded key-value (or document) store for my Windows desktop application. I want to be able to store various types of data (GPS tracks would be one example) and of course be able to query this data. The amount of data would be such that it couldn't all be loaded into memory at the same time.

I'm thinking about using sqlite as a storage engine for a key-value store, something like y-serial, but written in .NET. I've also read about FriendFeed's usage of MySQL to store schema-less data, which is a good pointer on how to use RDBMS for non-relational data. sqlite seems to be a good option because of its simplicity, portability and library size.

My question is whether there are any other options for an embedded non-relational store? It doesn't need to be distributable and it doesn't have to support transactions, but it does have to be accessible from .NET and it should have a small download size.

UPDATE: I've found an article titled SQLite as a Key-Value Database which compares sqlite with Berkeley DB, which is an embedded key-value store library.

Answer

Laurion Burchall picture Laurion Burchall · Feb 23, 2010

Windows has a built-in embedded non-relational store. It is called ESENT and is used by several Windows applications, including the Active Directory and Windows Desktop Search.

http://blogs.msdn.com/windowssdk/archive/2008/10/23/esent-extensible-storage-engine-api-in-the-windows-sdk.aspx

If you want .NET access you can use the ManagedEsent layer on CodePlex.

http://managedesent.codeplex.com/

That project has a PersistentDictionary class that implements a key-value store that implements the IDictionary interface, but is backed by a database.