Thursday, September 24th, 2009

Content Aware Image Resizing… in JavaScript

Category: Canvas

Content aware image resizing is a nifty technique where you can make an image smaller by literally understanding the content better and remapping it.


Stéphane Roucheray has put together a fancy demo (Firefox 3.5 only) that uses JavaScript and the Canvas tag plus createImageData to do all of this in the browser versus C++:

For this demo, a sub part of the Content Aware Image Resizing algorithm has been implemented. The width of the image can be reduced interactively without modifying its height. This implementation uses seam carving to re-size the image, subtracting the less visible vertical lines. It is a four step iterative algorithm. One iteration is one pixel width re-size. First an image is loaded into the Canvas context and then the iteration starts :

  1. A grayscale version of the image has to be calculated
  2. The edges of the image (Sobel convolution is used in our case) and its energy matrix is computed
  3. The seam of least energy (1 pixel vertical line from the bottom to the top of the energy matrix) is detected
  4. Then the pixel of the detected seam is removed from the original image and the result is re-injected as a source image to step 1

Each of the previous steps stores a whole matrix of data at the source image size. While these matrices are not all images but actually artifacts of the algorithm, storing them in an ImageData object is more convenient than using simple Arrays. This is why the createImageData method of the Canvas context is used. One of the benefits of this process is to allow showing the intermediate computations made under the hood.

As Chris Blizzard mentions in the blog post, this opens up lots of interesting possibilities:

This demo shows that’s possible to do more intelligent image resizing than just flattening an image’s pixels with CSS. Having computational and image manipulation capabilities directly in the browser opens up a new range of possibilities of how image data can be displayed to users. This is only one small demonstration of that.

Posted by Brad Neuberg at 1:30 pm

4.4 rating from 44 votes


Comments feed TrackBack URI

Cool technique. Also the dragging cursor is messed up and the guy on the video sounds vaguely like Kermit the Frog.

Comment by Darkimmortal — September 24, 2009

If the script uses createImageData, it should have also used web worker threads to distribute the ImageData computations among threads.

Comment by EliGrey — September 24, 2009

@Jadet: Photoshop already supports this technique. Adobe hired the guy that developed the original algorithm.

Comment by CheloXL — September 24, 2009

Directly in the browser would be nice… perhaps with a Firefox plugin to start…

Comment by sroussey — September 24, 2009

Just watched the you tube video, the way this algorithm works is quite incredible. Nice demo too!

Comment by McDaid — September 25, 2009

@Jadet It’s hard to use Web workers along with Canvas computation due to Web wrokers limitations, especially no access to the DOM

Comment by sroucheray — September 27, 2009

Leave a comment

You must be logged in to post a comment.