Memcached can count – pylibmc can’t

In my recent talk to Big Data London I said that memcached can’t count. Turns out that’s not really true, and the problem lies somewhere else.

Here’s the original evidence memcached has counting issues, using Python & pylibmc:

In [2]: cache.set('wibble', 0)

In [3]: cache.get('wibble')

Out[3]: 0

In [4]: cache.decr('wibble', 1)
Out[4]: 0L

In [5]: cache.incr('wibble', 1)
Out[5]: 1L

In [6]: cache.incr('wibble', -1)
Out[6]: 4294967296L

Incrementing by -1 has resulted in memcached incrementing by 2^32-1 – a classic signed/unsigned conversion problem.

But hang on – according to the memcached spec, incr takes unsigned 64bit integers – and it’s an error to send a signed number.

Sure enough, using the ASCII protocol (with memcached 1.4.9):

get wibble
VALUE wibble 0 1
incr wibble -1
CLIENT_ERROR invalid numeric delta argument

It seems memcached can count after all, or at least refuse to do operations it might get wrong. It’s another question why memcached won’t deal with signed numbers…

So where’s the weird behaviour above coming from? It’s a couple of bugs in pylibmc – bug 74 and bug 73.

The good news? Both bugs are now fixed in the latest github version, so very soon memcached and pylibmc will be able to count. Three cheers – hooray, hooray!



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s