Tuesday 21 October 2014

Forensics - An Introduction

We are used to analysing images based on patterns that are familiar to us.  We recognise that the subject in a photo is a bird, that it is flying and that it has a subtle pattern of light and shade on it's wings.  We can observe that one wing appears to be a little more in shadow than the other.  At some point we may be tempted to over-stretch our analysis and perhaps draw conclusions that are based more on guesswork than on sound evidence.

Digital image processing tools allow us to dig deeper and perhaps reveal properties about the image and the subject that are very often simply not presented in the original JPEG image produced by the camera.  But there are two sides to that coin.  All forms of image manipulation have the potential to lead us astray.  So, we need some structure or methodology to guide us.  We also need to understand the benefits and limitation of each of the tools we apply.

Poultry 101

There are different ways to cook chicken.  At the local chipper, the chicken may have a distinctive and comforting flavor.  It may not be the most exciting, nor the healthiest option.  It is JPEG chicken!  At home, the same piece of chicken can be cooked from raw to create a masterpiece, or perhaps something fit for the trash bin.  Unlike raw chicken however, the same RAW image file can be reworked ad infinitum until the desired results are achieved.  We know that RAW image processing is a powerful tool, not just for creating the best images, but for interrogating image files for identification purposes.  For more see HERE and HERE.

The original JPEG (left) is overexposed.  The JPEG on the right has been created from RAW, and demonstrates the power of RAW editing software, both for improving images and as a forensic tool.

In the absence of a RAW data file we can still, to a very limited extent, use image editing tools to improve the quality of JPEG and other file formats.  We can also, to an extent, interrogate JPEG and other files using image editing software.  Often, editing for the sake of image quality and editing for the sake of image interrogation are mutually exclusive.  In other words, improving the quality of an image may not always improve the resolution of fine detail or subtle colours, and visa versa, as illustrated HERE.  Also, image manipulation may often come at a cost.  It is easy for example to lose direction and end up further away from an objective than ever before.  We may be reluctant to take a chance with a form of analysis that we may not readily understand.  Sometimes, it is safer to put ones trust in the local chipper than to take a chance on the newly-opened Thai restaurant next door, or on the Thai cookbook gathering dust on the shelf for that matter!

The JPEG on the left is typical of a very bright day.  The harsh light challenges the camera's dynamic range.  Though, far less effective than RAW editing software, a standard image editing package like Adobe Elements makes a reasonable stab at retrieving details from this poor image.   Note how image quality dis-improves yet the ability to pick out plumage detail is much improved.

Learning a new skill takes much more effort than the willingness to simply 'have a go'.  Whether it is, 'how to use spices to cook the perfect Thai curry', or 'how to use software to analyse digital images', it all takes patience, time and practise.  At least, for the former we have a cook book to guide us. We do not have an equivalent text for what we are looking for here.  In many ways, what I am writing about here is based on pure trial and error on my part.  Some techniques will be handy in some cases but not in others.  And, I am starting close to the bottom of the learning curve, perhaps like most of you.  So, please bare with me.  One of the goals here is to work on a Forensics Manual and I will have a blog page devoted to the subject in time.

Firstly, the Ingredients

Each digital image is made up of millions of pixels, often consisting of tens if not hundreds of thousands of distinct colours.  Each pixel has a Hue value, a Saturation value and a Luminance value (for more see HERE).  But none of these parameters are fixed.  Image processing and editing tools allow these values to be changed.  These tools are simply complex equations or algorithms that modify these parameters across some or all of the pixels in an image.

As with cooking, where different processes and different seasoning and spices help enhance or mask flavors, image editing tools draw out different patterns in digital images.  Whether our goal is to be a master chef or a master forensicist we need to try and gain a good understanding of all the tools and processes as well as their impacts.  However as with all my postings I will try and avoid getting too technical, and will use illustrations to convey concepts and demonstrate results.


One would never throw all of the ingredients into a bowl, mix them together and then decide at that point whether the meal is going to be bread or a pie!  We need to know what our objectives are from the outset.  No doubt we will have some idea what the species might be and which field marks or plumage colours need expressing in the images we scrutinise.  We also hopefully will have some idea of the image quality issues that are facing us, be it underexposure, defocus, a white balance error, etc.  The objective determines what tools we will choose to use and in what particular order we might use them.  Once again, that cooking analogy.  For consistent results it is best to work to a recipe.


Image editing and processing tools can be very powerful.  They can reveal hidden detail and colours, or they can just as easily ruin a good photo or send analysis off on a tangent.  As with cooking, less is more!  Generally alterations should only be made if they contribute to the objective, and only to the extent that they continue to reward clear results.  Unnecessary modification is always a bad thing.

The left hand image is the JPEG produced by the camera.  Greyscale is one of the simplest image processing tools we have.  By reducing Saturation values to zero we also eliminate Hue from the equation, leaving only Luminance values.  Though the image quality hasn't suffered we have a much simpler pallet of colours, or rather shades to analyse.  There are in fact just 256 luminance values (called levels) in sRGB colour space (213 expressed in this image).  Working in greyscale can give us a clearer appreciation of subtle lighting and shade.  Saturation is another simple tool.  Here I use it to demonstrate how effective a 400% increase in Saturation can be at highlighting edge distortions like JPEG compression artefacts and sharpening halos around the bill of this tern.  It is also interesting to note that, for this image an increase in saturation has brought about a near four fold increase in uniquely-coloured pixels.  This might imply that that there is a lot of hidden colour detail in the original JPEG image.  Of course, the flip side of that is that many of these extra colours are false and are merely signal noise (a digital artefact).  All forms of image modification carry a sting in their tail!


Having created the perfect meal, presentation is key.  A chef might also want to take notes, make alterations to the recipe and file away for future reference.  I think it is really important to carefully record the tools and methods of analysis used so that the analysis can be replicated.  Ideally modified images should be stamped in such a way that they can't be mistaken for an original camera output file.  As part of my work in this area I will be providing some log sheets to accompany analytical recipes  and forensic tool specification sheets.  That way, hopefully everything I analyse and put up on the blog can be subject to peer review and scrutiny.

No comments:

Post a Comment