Welcome to End Point’s blog

Ongoing observations by End Point people

Raw Caching Performance in Ruby/Rails

Last week, I set up memcached with a Rails application in hopes of further improving performance after getting a recommendation to pursue it. We're already using many Rails low-level caches and fragment caches throughout the application because of its complex role management system. Those are stored in NFS on a NetApp filer, and I was hoping switching to memcached would speed things up. Unfortunately, my http request performance tests (using ab) did not back this up: using file caching on NFS with the NetApp was about 20% faster than memcached from my tests.

I brought this up to Jon, who suggested we run performance tests on the caching mechanism only rather than testing caching via full http requests, given how many layers of the stack are involved and influence the overall performance number. From the console, I ran the following:

$ script/console   # This app is on Rails 2.3
> require 'benchmark'
> Rails.cache.delete("test")
> Rails.cache.fetch("test") { [SomeKlass.first, SomeKlass.last] }
> # to emulate what would potentially be stored with low-level cache
> { |x|"times:") { 10000.times do; Rails.cache.fetch("test"); end } } 

We ran the console test with a few different caching configurations, with the results shown below. The size of the cached data here was ~2KB.

cache_store avg time/request
:mem_cache_store 0.00052 sec
:file_store, tmpfs (local virtual memory RAM disk) 0.00020 sec
:file_store, local ext4 filesystem 0.00017 sec
:file_store, NetApp filer NFS over gigabit Ethernet 0.00022 sec


I also ran the console test with a much larger cache size of 822KB, with much different results:

cache_store avg time/request
:mem_cache_store 0.00022 sec
:file_store, tmpfs (local virtual memory RAM disk) 0.01685 sec
:file_store, local ext4 filesystem 0.01639 sec
:file_store, NetApp filer NFS over gigabit Ethernet 0.01591 sec



It's interesting to note here that the file-system caching outperformed memcached on the smaller cache, but memcached far outperformed the file-system caching on the larger cache. Ultimately, this difference is negligible compared to additional Rails optimization I applied after these tests, which I'll explain in a future blog post.


Ethan Rowe said...

It's worth noting that memcache is about scalability, not raw speed. This is something the docs are pretty explicit about. The time for a simple cache lookup may be roughly analogous to that of a simple SELECT on MySuql. With that in mind, one expects small, local file lookup to do better compared to small, individual memcache lookups.

You'll get greater benefit from memcache by adapting your access patterns to play to its strengths. For instance, refactor your loops so rather than performing iterative cache reads, you do a single multiget to retrieve all the items in one blocking call, then iterate over the result. That brings down the wait state overhead, while at the same time encouraging the use of small cached items rather than big, aggregated caches -- an approach that allows for better cache reuse and hopefully better value overall.

And of course, you'll see additional benefits over files when running at scale, with multiple servers. The shared cache will make for a happier database, when caches need only be rebuilt once, rather than once per server as would be necessary for local files. you could of course use nfs for shared files, but if you're not already tied to a file-oriented solution, you'll probably find memcache easier to deploy and more flexible in its use.

And of course, we don't want to overlook redis.

Anyway, thanks for the data points, it was interesting reading.

Steph Skardal said...

Ethan - Thanks for the comments. I was expecting & hoping that you'd opine.

Jon and I discussed the benefit here on using memcache on multiple servers. I'm not sure we'll be making the final call on this, but it will certainly be our recommendation.

Redis came up as an option also. This work was a small part of optimizing the entire app, and like I mentioned in the article, the improvement here from caching is negligible compared to other bits of optimization I've done on the app to eliminate database lookup & Ruby object instantiation.