I've had problems with MemcacheDB and I'm curious to hear about other people's experiences.
Basically, I've found that under high read/write loads I get occasional socket timeouts (tested on linux/osx). I think the underlying reason for the timeouts are Berkeley DB locks when data gets flushed to disk. Beyond the hassle of getting intermittent socket timeouts the real problem is that the memcache client API fails gracefully because it was specifically designed to be fault tolerant. You can check for socket errors and retry queries but you'll still get unpredictable query times.
I'm coming to the conclusion that it's an architecture issue - MemcacheDB is a persistent database abstracted behind an interface specifically designed for non-persistence. The abstraction leaks when the database locks.
I have had similar issues like you (i.e. very poor performance of Berkley DB, especially under load). I am sure thought one can configure this (some options are listed here: http://www.oracle.com/technology/documentation/berkeley-db/d... ), I have tried different options, but I can't recall the effects.
Anyway, after testing MemcacheDB and Tokyo Tyrant in production my conclusion was to use Tokyo Tyrant instead of MemcacheDB. Tokyo Tyrant implements the memcached protocol and performs really well under load (and has TONS of features such as master-master replication, Lua scripting, different types of engines [hash, b-tree or memory]). You can also check LightCloud, which is a distributed key-value database built on top of Tokyo Tyrant.
Thanks for the response - I'm glad to hear I'm not the only one experiencing these issues. I've read a lot of articles recommending MemcacheDB but none mention these issues. I wish the authors had actually tested MemcacheDB before writing about it. I wasted a day working on this.
Have you tried TokyoCabinet? It serves a similar purpose (a persistent key/value store which speaks the memcache protocol) and I've only heard good things about its performance.
Basically, I've found that under high read/write loads I get occasional socket timeouts (tested on linux/osx). I think the underlying reason for the timeouts are Berkeley DB locks when data gets flushed to disk. Beyond the hassle of getting intermittent socket timeouts the real problem is that the memcache client API fails gracefully because it was specifically designed to be fault tolerant. You can check for socket errors and retry queries but you'll still get unpredictable query times.
I'm coming to the conclusion that it's an architecture issue - MemcacheDB is a persistent database abstracted behind an interface specifically designed for non-persistence. The abstraction leaks when the database locks.