Combatting Confirmation Bias

            Confirmation bias is a term psychologists use to describe the state of mind where an individual, or group persists in a belief that they hold, where they only attend to data that confirm their beliefs, but ignore data to the contrary.   This often happens in political discourse, but as Paul Levy has blogged, it can infect diagnostic assessments of patients.   I’ve written about this in the context of way finding, where people will “bend the map”  - try to get landmarks indicated on a map to line up with their perceptions, even though the correspondence doesn’t match.  

            How can we combat confirmation bias?   Part of the battle relates to the problem of the unconscious origins of the biases.   Mahzarin Banaji is a social psychologist at Harvard.  She gave a convincing demonstration of this to a group of faculty members.   Mahazrin showed a movie of several people tossing balls to each other.   We were instructed to watch the clip and count how many times the balls were tossed back and forth.   The audience dutifully counted, and when the clip was over, she asked us how many tosses there were.   Most of us answered around the “right” number of twenty-two, as I recall.   She then asked if we noticed anything unusual in the movie.   No one was aware of anything amiss.   The clip was played again and we watched. This time we looked for something out of the ordinary.   On top of the image of the balls and people, the image of an actor walking through the scene in a gorilla suit carrying a fancy parasol was superimposed in black and white.   No one noticed the image of the gorilla the first time around when we were counting the tosses, but once we were alerted to the presence of something else, we immediately saw it. 

            I suspect that our ability to “see” things has to do with our attentiveness.   If we don’t expect to see something, we will ignore its presence.  A few years ago, I taught myself animal tracking, and had to retrain my attention to different clues in the environment: a disturbance in leaves on the ground, animal scat, and sometimes the tracks themselves.   Suddenly the secret life of all sorts of mammals in my suburban neighborhood became as clear as day, while for years I was oblivious to their presence.

 

            I think of navigation as a laboratory of human cognition.   Part of this is a reductionist exercise.   Medical diagnoses may be complicated by the emotional feelings a physician has toward a patient.  Pronouncements about weapons of mass destruction in Iraq were fraught with emotions, politics, power, and culture.  Navigation, on the other hand, involves how we compare observations to something simple: our position and the identity of landmarks.   When way finding, I often try to form an initial hypotheses about a landmark, and then test it against other signs.   Often, I just let my attentiveness float, never focusing too much on one thing and try to remain flexible in my interpretations, wondering if my first assumption is wrong, and how could it be so?

            While I have already described one case of confirmation bias in this blog, I’ve slowly been collecting other examples and analyzing the failure modes.   More often than not, Mazharin’s description of unconscious bias, or my notion of the lack of attentiveness, is the culprit. 

            A number of times, my brother-in-law Ken has been tripped up by both false assumptions and problems of attentiveness.   I don’t mean to pick on Ken, at all.   Ken’s a great guy, very intelligent, and thoughtful.  The main issue is that I’ve had the opportunity to interact with him and observe his pronouncements on our position and landmarks while backpacking – a perfect laboratory where many extraneous elements are removed.   

            One time we were hiking along a ridgeline in Olympic National Park.   The trail was covered in snow, and we were looking for a side trail that forked off.    I was trying to figure out our progress against features in the valleys on either side of the ridge.   Ken kept trying to pair the names of lakes or ponds on the map with features we’d see below the ridge.   There was one called “Lake Eight” indicated on the map.   We passed some glacial ponds, and he decided each one was Lake Eight in turn.   After some distance we saw a lake further down in a cirque below the tree line. He decided that this must be Lake Eight, as the constricted waist made it look like a figure 8.   After consulting the map and looking at the nearby ridges, I identified the lake as one called Morganroth Lake from the location and the topography.

            As it turned out, there was an unusually heavy snowpack that year, and Lake Eight was frozen and covered with a thick layer of snow, so it was not visible.   Mistake number one was assuming that we could see Lake Eight, when it was actually invisible.   Mistake number two was bending the map and interpreting the shape of Morganroth Lake to be Lake Eight from its shape.  The name “Lake Eight” probably comes from its proximity to a depression called Seven Lakes Basin.   Lake Eight was an ‘eighth lake’ – a play on words.  Ken kept trying to make some feature fit into his conception that he should be seeing Lake Eight and kept finding ‘facts’ to back up this notion.  

            If he’d had more open attentiveness, he would have realized that the heavy snowpack might have obscured Lake Eight, and also that the topography surrounding Morganroth Lake made it unsuitable as Lake Eight.  I hadn’t formed any strong opinions on the subject until I finally hit on what was going on.   In other words, I had a kind of open attentiveness and didn’t work too hard to make things line up.    

Morganroth Lake and Lake Eight in snow-free conditions.   When we passed through, Lake Eight was frozen and buried under a snowfield.

This summer, I used myself as a subject. While I was paddling up the Sheepscot River with Dan Carr, he asked me where I thought an island called Powderhorn was located.  This was our destination for the evening.   I looked about a mile up the river and identified a white sandy island that seemed to be a likely candidate.    I also knew that Powderhorn was at a bearing of 80 degrees magnetic from the northwest point of another island called MacMahon Island.    I took a bearing to the sandy island and told Dan that I thought it was Powderhorn.    

            As we reached MacMahon, and paddled along its eastern edge, it became apparent that the sandy island I spotted wasn’t Powderhorn at all, but a much smaller island called Middle Mark.   Although Middle Mark Island was directly in line between the previous position and Powderhorn, I completely overlooked this on the map and misidentified it.

            We paddled on to the northeast point of MacMahon Island.   I took a bearing of 80 degrees magnetic (the local variation is 18 degrees, so the true bearing was more like 60 degrees).   I could not discern Powderhorn – it blended into the background of other islands and the mainland.   We started off on that bearing, and I knew a buoy (buoy 1 in the figure) was almost directly in line with our heading to Powderhorn, but I only saw a large buoy (buoy 2) further to the north.   I stopped and puzzled why the buoy seemed off.   Just then, Dan spied a much smaller buoy that was consistent with the heading to Powderhorn.   Finally, as we approached Powderhorn, we saw the red buoy (buoy 3) and knew we were nearly on top of the island. 

            Now there was some confirmation bias going on the whole time – my mistake of Middle Mark Island as Powderhorn and my misidentification of one of the buoys, but by making a tentative identification of these landmarks, and then abandoning them when the facts didn’t suit the first hypothesis, we made progress with little loss of time.   Again, this was a case of having an open attentiveness, and I avoided traps by being willing to abandon the initial hypothesis when the facts didn’t fit. 

The route up the Sheepscot toward Powderhorn, including landmarks mentioned in the text. 

Another case of  a successful outcome was reported by Larry and Lin Pardey.   They were sailing from Sri Lanka across the Bay of Bengal.   The tried to sail southwest to avoid shipping lanes, but unexpectedly kept seeing large ocean going vessels.   Their sextant readings showed them over a hundred miles north of where they expected to be, and thought that they had made some error in their math.   The combination of both the sights and the presence of vessels didn’t square with their dead reckoning.    What was the explanation?   

Larry figured that the only plausible answer was an unexpected large current that dragged them north.   They began to see very long period swells, consistent with a typhoon.   Larry concluded that there was a typhoon off in the Bay of Bengal that had set up an anomalous current.    They steered again to the southwest, and the shipping traffic disappeared below the horizon to the north.   They rode out the storm successfully, although a Canadian yacht sunk.

 

            In the case of the Pardeys the disagreement between the celestial observations and the dead reckoning led them to disbelieve the celestial calculations, but they had a kind of open attentiveness to other clues and eventually figured out what was going on. 

            Now, I’m well aware that these stories may be subject to my own meta-confirmation bias in cherry picking them.  On the other hand, if they are indicative of how we can avoid the problems created by the bias, it shows that a kind of floating attentiveness and a willingness to examine alternative explanations is a plausible way of addressing the problem.   In the meantime, I’m  collecting more navigation stories that involve confirmation bias.   Contributions welcome!