In the last few postings I have explored a range of different human cognitive biases and arranged them into a certain order which we will hopefully find useful here. The lists include distraction biases, evaluation biases, memory biases, biases of the self and the group and finally experimental bias. This is not an entirely exhaustive list. We also have various social biases and others (see HERE) which I have not looked at because we are only interested in a narrow subject and context.
The Context
Essentially our context is typically one where we are quietly examining one or more digital images, possibly with some additional supportive notes. Whereas during field observation, where we may find ourselves making split-second decisions, we should not be under any great time pressure to make a critical evaluation from a set of digital images (unless perhaps the photos are part of a table quiz). Our main distractions are likely to lie within the evidence itself. We find ourselves evaluating the available evidence, drawing on our own knowledge and memory to do so (hopefully with access to research materials if needed). We may be feeling we need to justify our conclusions to our peers, possibly even publishing them to the world over the internet. So a great many of the common cognitive biases will still apply, but often in a setting where if we wish we can apply control over bias. So what are we looking for and how do we avoid bias?
Distraction
In earlier postings I have made some comparisons between the human visual system and the camera (HERE and HERE). During critical observation our focus is extremely narrow. If I allow my judgement to guide my line of observation I can very easily miss something right under my nose. We know that we have a tendency to anchor on certain details. Consider how differently we would view the below image (mystery diver, Gavia sp.) if we had first read just one of these two observer notes.
Observer 1
A medium-sized diver with a prominent dark neck strap, clearly visible throughout the observation...
Observer 2
A large diver, viewed distantly in choppy waters. Appeared to have a neck strap but difficult to make out due to the strong shadow and low light...
If we want to avoid being distracted we must try and clear the mind and quell any prior expectations we may have. We must also try and approach the puzzle in a logical and systematic fashion, reserving all judgements until we have reviewed all of the available evidence. With images we have the opportunity to look at an identification with a fresh pair of eyes. I would advise pushing all the other information aside and concentrating on identification from first principals from the images before introducing additional information from the available notes. A full analysis of a photo could take some time and objective analysis deserves that investment, because once additional information is introduced to the analysis it begins to alter our perceptions. I would take each photo as though it were in isolation and extract every available detail from it before moving on. I would also recommend taking detailed notes for each image. Memory is fickle and an important observation can be forgotten or twisted by bias to suit a new perception.
Evaluation
To evaluate anything we need to understand the limits of the subject, so we need a set of assessment criteria. When it comes to identification of a bird from photos we are not just referring to the criteria used to separate one species from another. We also must consider the criteria for accurately judging details and colour from images. This requires a good understanding of how images are created and the reliability of different criteria when viewed in digital images. As we have seen, there are a lot of variables to consider so we must take our time and challenge each of our own conclusions very carefully.
Continuing with the above diver example we must remember that we begin to make evaluations from the very moment we receive sensory information. As we have seen, a range of biases actually help guide our judgement and can quickly lead us astray. We need to keep an open mind and try to avoid investing in one answer at the expense of another. We must also try and be as critical as we can be of our own findings and try and approach each question equally from both sides of the argument. It might be worth imagining having to debate the more convenient side of the argument and then having to present the alternative, apparently less-likely side with even greater conviction, as though it were the correct answer. In order to do so, it is likely every conclusion will have to be rigorously challenged.
This mystery diver is a tough call. With only one available image all attempts to improve on the original JPEG (left) have only yielded a slightly better quality image (right). If we have avoided all distractions and resolved all the available evidence from this image we should have concluded that the image is ambiguous at best.
We know that as humans we all suffer from cognitive bias so we cannot apply the same level of objectivity to written descriptions as we can to digital images. We have to allow for the possibility that a written description is factually inaccurate, no matter how careful the observer has been. If we make the conscious decision not to accept what we read as fact, then what is our starting position? I would advocate that we create two opposing positions and argue both of them based on the evidence. If we cannot satisfy ourselves that one argument wins out firmly over the other then it is probably safe to say that the evidence is inadequate to support a firm identification.
Memory
As someone evaluating the information we must draw on our own memory and understanding of the limits of the subject. There are a lot of variables to consider so we may forget to apply certain tests or come at the puzzle misinformed. It pays to have lists as an aide-mémoire. We might list the identification criteria for a particular species and the sources where those criteria can be found. We might use a standard list of image quality parameters and a list of image tests and modifications we can choose from to test or extract evidence from images.
Biases of the Self and the Group
It is possible to be influenced by the group and influenced by various ego-driven biases. The more structured the analytical process is the greater the chance of avoiding these pitfalls. It is best to conduct one's own analysis prior to reviewing the analysis of others.
Experimental Bias
If analysis involves conducting an experiment or experimenting with images to try and resolve a question, it is important to set objectives and maintain an open mind. It is possible to be overly selective and to only apply direct testing, thereby narrowing the focus and ignoring other possible solutions.
Ten Tips
Here are some general findings from this initial analysis of different forms of cognitive bias.
(1) Try and approach all identifications with an open mind, free from preconceptions and distractions.
(2) Start with the images first. Treat each image as though it were the only image available and give it the full attention it requires. Take notes as memory can be fickle.
(3) When evaluating evidence consider a counter-argument for each conclusion and try and build a strong case for both sides of an argument.
(4) When reviewing observer notes always assume that they are biased and therefore factually may be full of inaccuracies.
(5) Start from the most conservative position and work up to an answer based on a logical foundation of evidence.
(6) Avoid relying too much on one's own memory. Work systematically using lists of identification criteria, lists of image quality parameters and artefacts.
(7) Try not be get too invested in the outcome. Focus on the analysis.
(8) You are under no pressure to publish your findings or put them up for peer review.
(9) Wait until you have completed your own analysis before reviewing the analysis of others.
(10) If you are experimenting with images or generating experiments to test a theory try to avoid narrow, direct forms of testing.
It is my intention to develop these ideas further and start to produce some general tools to minimise bias during analysis.
No comments:
Post a Comment