News

Welcome to End Point’s blog

Ongoing observations by End Point people

A caching, resizing, reverse proxying image server with Nginx

While working on a complex project, we had to set up a caching reverse proxying image server with the ability of automatically resize any cached image on the fly.

Looking around on the Internet, I discovered that Nginx has a neat Image Filter module capable of resizing, cropping and rotating images. I decided to try and combine this with Nginx's well-known caching capabilities, to create an Nginx-only solution.

I'll describe here a sample setup to achieve a similar configuration.

Prerequisites and requisites


We obviously need to install Nginx.

Note that the Image Filter module is not installed by default on many Linux distributions, and we may have to install it as a separate module. If we're using Nginx's official repositories, it should just be a matter of installing the nginx_module_image_filter package and restarting the service.

What we want to achieve in this example configuration is to have a URL like:

http://www.example.com/image/<width>x<height>/<URL>

...that will retrieve the image at:

https://upload.wikimedia.org/<URL>

...then resize it on the fly, cache it and serve it.

Cache Storage configuration


First of all, we need to set up the cache in our main http section:

proxy_cache_path /tmp/nginx_cache levels=1:2 keys_zone=nginx_cache:10M max_size=100M inactive=40d;

This will provide us with a 10MB storage space for keys and 100MB for actual images, that will be removed after not being accessed for 40 days. These values can be tuned as needed.

Caching Proxy configuration


Next, we'll configure our front facing virtual host.

In our case, we needed the reverse proxy to live within an already existing site, and that's why we chose the /image/ path prefix.

  server {
      listen       80;
      server_name  www.example.com;
  
      location /image/ {
          proxy_pass http://127.0.0.1:20000;
          proxy_cache nginx_cache;
          proxy_cache_key "$proxy_host$uri$is_args$args";
          proxy_cache_valid 30d;
          proxy_cache_valid any 10s;
          proxy_cache_lock on;
          proxy_cache_use_stale error invalid_header timeout updating;
          proxy_http_version 1.1;
          expires 30d;
      }
  
      location / {
          # other locations we may need for the site.
          root /var/www/whatever;
      }
  
  }

Every URL starting with /image/ will be server from the cache if present, otherwise it will be proxied to our Resizing Server, and cached for 30 days.

Resizing Server configuration


Finally, we'll configure the resizing server. Here, we use a regexp to extract the width, height and URL of the image we desire.

The server will proxy the request to https://upload.wikimedia.org/ looking for the image, resize it and then serve it back to the Caching Proxy.

  server {
      listen 127.0.0.1:20000;
      server_name localhost;
  
      resolver 8.8.8.8;
  
      location ~ ^/image/([0-9]+)x([0-9]+)/(.+) {
          image_filter_buffer 20M; # Will return 415 if image is bigger than this
          image_filter_jpeg_quality 75; # Desired JPG quality
          image_filter_interlace on; # For progressive JPG
  
          image_filter resize $1 $2;
  
          proxy_pass https://upload.wikimedia.org/$3;
      }
  
  }

We may want to tune the buffer size and jpeg quality here.

Note that we may also use image_filter resize and crop options, should we need different results than just resizing.

Testing the final result


You should now be able to fire up your browser and access an URL like:

http://www.example.com/image/150x150/wikipedia/commons/0/01/Tiger.25.jpg

...and enjoy your caching, resizing, reverse proxying image server.

Optionally securing access to your image server


As a (simple) security measure to prevent abuse from unauthorized access, you can use the Secure Link module.

All we need to do is update the Resizing Server configuration, adding some lines to the location section:

  server {
      listen 127.0.0.1:20000;
      server_name localhost;
  
      resolver 8.8.8.8;
  
      location ~ ^/image/([0-9]+)x([0-9]+)/(.+) {
          secure_link $arg_auth;
          secure_link_md5 "$uri your_secret";
          if ($secure_link = "") {
              return 403;
          }
          if ($secure_link = "0") {
              return 410;
          }
  
          image_filter_buffer 20M; # Return 415 if image is bigger than this
          image_filter_jpeg_quality 75; # Desired JPG quality
          image_filter_interlace on; # For progressive JPG
  
          image_filter resize $1 $2;
  
          proxy_pass https://upload.wikimedia.org/$3;
      }
  
  }

To access your server you will now need to add an auth parameter to the request, with a secure token that can be easily calculated as an MD5 hash.

For example, to access the previous URL you can use the following bash command:

echo -n '/image/150x150/wikipedia/commons/0/01/Tiger.25.jpg your_secret' | openssl md5 -binary | openssl base64 | tr +/ -_ | tr -d =

...and the resulting URL will be:

http://www.example.com/image/150x150/wikipedia/commons/0/01/Tiger.25.jpg?auth=TwcXg954Rhkjt1RK8IO4jA

1 comment:

Josh Williams said...

Marco,

That's pretty slick, cool to see that module in action! Especially now that it can be dynamically loaded, no more having to recompile nginx.

Have you looked at the performance of it, compared to proxying the connection back to something else that performs the image resize?