Categories
Posts

Redis Hash Efficiency

Instagram on switching from plain key => value storage to hashes in Redis:

The size difference was pretty striking; with our 1,000,000 key prototype (encoded into 1,000 hashes of 1,000 sub-keys each), Redis only needs 16MB to store the information. Expanding to 300 million keys, the total is just under 5GB—which in fact, even fits in the much cheaper m1.large instance type on Amazon, about 1/3 of the cost of the larger instance we would have needed otherwise. Best of all, lookups in hashes are still O(1), making them very quick.

via Instagram Engineering • Storing hundreds of millions of simple key-value pairs in Redis.

The original key => value storage method used 70MB for 1,000,000 entries. Making their 300,000,000 target usage in the neighborhood of 21GB. Going from 21GB to 5GB is a huge win.