Welcome to End Point’s blog

Ongoing observations by End Point people

Spree Performance Benchmarking

I see a lot of questions regarding Spree performance in the spree-user group, but they are rarely answered with metrics. I put together a quick script using the generic benchmark tool ab to review some data. Obviously, the answer to how well a site performs and scales is highly dependent on the host and the consumption of the web application, so the data here needs to be taken with a grain of salt. Another thing to note is that only two of the following use cases are running on Rails 3.0 — many of our current Spree clients are on Spree 0.11.2 or older. I also included one non-Spree Rails ecommerce application, in addition to a few non-Rails applications for comparison. All of the tests were run from my home network, so in theory there shouldn't be bias on performance tests for sites running on End Point servers.

ab -n 100
-c 2 homepage -c 20 homepage -c 2
product page
-c 20
product page
Client #1
Spree: 0.11.2
Hosting: 4 cores, 512 GB RAM
# Products: <100
7.49 24.75 6.49 19.87 Requests per second
266.889808.041307.9971006.552 Time per request (ms)
Client #2
Spree 0.11.2
Hosting: Engineyard, medium instance
# Products: 100s
5.3220.285.3618.03 Requests per second
375.713986.309373.2891109.524 Time per request (ms)
Client #3
Spree: 0.9.0
Hosting: 4 cores, 1 GB RAM
DB: PostgreSQL
# Products: <100
4.9125.391.986.54 Requests per second
407.135787.7821011.8753060.062 Time per request (ms)
(Former) Client #4
Spree: 0.11.2
Hosting: Unknown
DB: PostgreSQL
# Products: >5000
20.698.8410.1519.28 Requests per second
96.6732262.105196.9961037.146 Time per request (ms)
Client #5
Spree: 0.11.2
Hosting: EngineYard, small instance
# Products: 1
12.2816.23N/AN/A Requests per second
162.9091231.945N/AN/A Time per request (ms)
Client #6
Spree: 0.40
Hosting: 4 cores, 1 GB RAM
# Products: 50-100
3.618.932.963.06 Requests per second
553.5692240.657675.3066539.433 Time per request (ms)
Spree: Edge
Hosting: Heroku, 2 dynos
DB: Unknown
# Products: 100s
8.1712.794.75.48 Requests per second
244.8311563.642425.273652.927 Time per request (ms)
Client #7
*custom Rails ecommerce app
Hosting: 1.0 GB RAM
# Products: 1000s
5.4329.84.4523.14 Requests per second
368.409671.082448.962864.24 Time per request (ms)
Interchange Demo
Hosting: 4 cores, 2 GB RAM
# Products: >500
7.4155.277.513.93 Requests per second
269.942361.875266.4921435.51 Time per request (ms)
Client #8
*PHP site,
serves fully cached pages
with nginx with no app server or db hits
Hosting: 4 cores, 4 GB RAM
10.8130.546.059.87 Requests per second
184.994654.858330.7272027.092 Time per request (ms)
Magento Demo
Hosting: Unknown
DB: Unknown
# Products: 100s
4.2644.852.6836.29 Requests per second
469.831445.931745.472551.11 Time per request (ms)

Here's the same data in graphical form:

Requests per Second

Time Per Request (ms)

We expect to see high performance on some of the sites with significant performance optimization. On smaller VPS, we expect to see the the server choke with higher concurrency.

No comments: