Image Thumbnailer

Objective

Provide an on-the-fly image re-sizing web service. Thumbnailiing here is defined as the process of taking an image and rendering it in a smaller viewport or resolution (downscaling).

Scope

This post highlights only the functions of a thumbnailer.
Authentication and other things around it are out of scope.
Transcoding and image codecs are also out of scope.

Principles

Given an image and some target dimensions, produce an image fitting in those target dimensions.
An advanced feature also requires to tell whether the same thumbnail changed since a given date.
Target dimensions and other parameters for thumbnailing must be part of the request.

Properties

A thumbnailer is a perfect "standalone" application requiring very little configuration and interactions with other systems.
It's actually composed of 2 components:

1. Request handler component which first deals with some URI format to understand what it has to thumbnail, then deals with "If-Modified-Since" headers for example, and specifying response headers (cache control, content type, etc.).
2. Actual Thumbnailer component responsible for taking an image as input, some parameters (e.g. target dimensions) and render an image as output.

Thumbnailer should not require databases in order to scale easily when necessary.
It can be "embedded" in the main web application (e.g. for small web sites for example) but it's best to keep it outside of the main web app.
A different (sub?) cookie-less domain should be used (to speed up page load time) and it will even facilitate deployments and performance testing.

Caching should also be implemented some way or another. The best approach is to have a CDN and use the thumbnailer service as origin.

Implementation

The thumbnail parameters are constructed when parsing the URL sent to request the thumbnail image.
Parameters include but are not limited to:

• URI (partial or absolute) of the original image to thumbnail
• Dimension(s): width and height but it's possible that only one can be specified (and let the thumbnailer figure out the missing dimension)

Using the URI, the thumbnail will check its last modified date against the "If-Modified-Since" header of the request if present.
The thumbnailer will respect that header and return a 304 Not Modified if it is the case (see diagram D2 below).

If the original image changed since the "If-Modified-Since" date, or if the "If-Modified-Since" header is missing, or if we can't honor that same header for any reason, the thumbnailer will create an image according to the parameters sent (see diagram D1 below).

Viewport & Aspect Ratios

In order to keep the original aspect of an image, we can crop it when the ratio of the target dimensions is different.
Example: Original Image in 16/9 ratio vs. Tall thumbnail

Thumbnailing Algorithms

The table below present some algorithms to use when thumbnailing.
The Time column here contains data of tests against the same image for each algorithm, repeated multiple times and taking the average.
It does not mean thumbnailing will always take that same amount of time, it's only to compare algorithms between them.
Also, no native acceleration were used for any of those algorithms.

Algorithm	Description	Time
Nearest	Poorest quality of all but with very good performance	0.225ms
One-step bilinear	About the same quality as Nearest/	0.75ms
Bicubic	About the same quality as Nearest again.	1.925ms
Area Average	Best quality but might look a bit blurry. Poor performances however (vs the rest)	31.25ms
Multi-step Bilinear	More blurry again than Area Average.	5.25ms
Trilinear mipmip	Even more blurry than Multi-step Bilienear.	10ms

At Tripfilms.com, we opted for the one-step algorithms (Nearest, One-step bilinear or Bicubic).

Filters & Masks

Designers are accustomed to using filters to improve the quality of an image.
A very common one is called unsharpen masking. The idea is to sharpen the image using first a blur version of it as a mask, then combining it to the negative version of the image. This results in a crisper image (i.e. more contrast).

For performance and costs reasons, we apply this unsharpen filter on all thumbnails except Nearest. We wanted to keep Nearest as fast (and cheap) as possible.

Example: Nearest, no unsharpen filter vs One-step bilinear with unsharpen filter.

Resources

[1] Header Field Definitions, http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
[2] The Perils of Image.getScaledInstance(), http://today.java.net/pub/a/today/2007/04/03/perils-of-image-getscaledinstance.html
[3] Unsharpen masking, http://en.wikipedia.org/wiki/Unsharp_masking