A bit of a departure from the norm today, I've just spent the afternoon having a whale of a time with a good mate of mine, talking guitars, web programming, a bit of home automation, and at some point the subject of time lapse photography came up. But wait, isn't this a coding blog? What the heck's this got to do with code!?
Okay, some background first. My friend is fortunate enough to have a view from his window over the beautiful countryside surrounding Preston, out towards the coast. For a year now, he's had a digital camera trained on said countryside, taking a photo every minute. 12 months and half a million photos later, he's discovered a problem. Some of his photos need touching up, and by some I mean a LOT. Here's where the coding comes in.
Some Background and Some Requirements
We get a lot of cloud in Preston, which is great for this kind of time lapse photography, it's beautiful stuff to watch. But the occasional moments where it's not cloudy have played hell with the CCD on his (relatively) cheap camera. Where we have a cloudy sky the picture's great, the colours are spot on. As soon as the sky clears, everything's washed out and colourless.
Now, the work required to fix each photo is dead simple, bring it into Photoshop and apply a curves adjustment to do some gamma correction, but I'm not sure Photoshop's geared up to process half a million images, and to work out which images need fixing and which don't!
So the obvious answer is to write a little console app in .Net that will do the job. At the moment, I can see the following requirements:
- Read a JPG image into a 2-dimensional array of brightness values.
- Examing a configurable portion of the image to determine whether the image needs fixing or not, and how severely.
- Apply a curves adjustment to the image with configurable input and output values
- Save the image again
This sounds fairly straightforward, reading a JPG is no problem, examining the brightness of a pixel is no problem, there's a curves tool in Paint.Net that hopefully I can borrow the code for, and saving JPGs is again pretty straightforward stuff.
The Inevitable Twist - Editing JPEGs Without Recompressing
But of course, there's a twist - I'd like to be able to do all of this, without recompressing the image! The forementioned (relatively) cheap camera has been capturing these images at standard VGA (640 x 480) resolution, so every pixel counts.
How to edit the JPEGs without the recompression? Let's first of all have a quick recap how JPEG compresses images:
- The image is changed from the RGB colour space that we all know and love into a YCrCb colour space (see Wikipedia's discussion on YCrCb for more info, including a cool little animated GIF)
- The Cr and Cb channels are downsampled by 50% on each axis, since the human eye is most sensitive to the Y (brightness) component. This halves our payload.
- Each colour channel is broken up into 8x8 pixel blocks
- Each block has a good ol' DCT (Discrete Cosine Transformation) applied, this converts our 64 individual pixels into 64 frequencies, describing the patterns of pixels in that block.
- The big compression win now comes in as quantisation is applied to the output of the previous step. Certain frequencies are stored in high fidelity, other frequencies with less fidelity or not at all, depending on the compression quality indicated when the image is compressed.
- A final lossless compression of the resulting frequency data, and we're done.
So the question is, after all of the above mangling of an image, how easy is it to directly manipulate the Y values without otherwise affecting the image?
Time will tell!