How AI might complement what human intelligence lacks and act as a counterweight to humans’ cognitive biases
Image source: photo by geralt from Pixabay
From the book “Thinking, Fast and Slow” by the Nobel prize winner Daniel Kahneman, we all know human brains are far from perfect in what they are supposed to do. In addition to emotional impulses, obstinate addictions, and moral struggles since the dawn of human civilization, Kahneman has comprehensively described the nuances of our cognitive biases in his book. More frustratingly, many permeate every walk of our lives, organizations, and society. Given this, we now face two key questions. First, how can we identify the reality of such biases in our decisions? Second, how can we compensate for or prevent these biases in our decision-making process? In this article, I will address these two questions from a data science perspective by focusing on one of the most prevalent biases — confirmation bias. Given the advances of machine learning and AI, with their help, we now see the light of detecting, overcoming, and preventing these biases.
What is Confirmation Bias?
Confirmation bias is the tendency to interpret and look for information to confirm or support existing beliefs or prior conclusions. Because of confirmation bias, people tend to confirm their beliefs or assumptions by testing ideas one-sided, focusing on the supporting/positive evidence but ignoring the alternative/contradictory facts that could disapprove of their views. Confirmation bias is unintentional by nature, as opposed to deliberate deception. This bias has widespread implications across many areas of human life and society, including scientific research, medicine, politics, law enforcement, social psychology, and organizational decisions.
The English psychologist Peter Cathcart Wason first termed and studied confirmation bias systematically in the 1960s. In his experiments, Wason gave the participants a 4-card puzzle, also called the Wason selection task, to solve. The puzzle can have any variations, but the results are highly repeatable. Let’s take a look at one example:
Four cards are placed on a table, each with a number on one side and a color on the other. The rule is that if a card shows an even number on one face, its opposite side should be blue. Now, on the table, you see four cards: 3, 8, blue, and red. Which card(s) must you turn over to test the rule to be true or not?
An example of the Wason selection task: which card(s) must be turned over to test the rule that if a card shows an even number on one face, its opposite face is blue? Image source: Wikipedia
Everyone knows to turn the 8 card; some would choose 8 and blue. The correct answer is to turn the 8 and red cards, whereas most people miss turning the red card. If you turn over the 3 card, blue or not blue on the opposite side is irrelevant to the rule. Likewise, if you turn over the blue card and find an odd number on the other side, it does not have an impact because the rule dictates what even number may have on the opposite side. On the other hand, if you turn over the red card and find an even number on the opposite side, you would prove the rule is violated.
Surprisingly, participants repeatedly performed poorly on various forms of this test. People focus on the positive support of the rule (the opposite side is blue) but ignore information that could potentially falsify the specified rule (the opposite side is not blue).
The rules of Wason Selection Tasks are all straightforward with a logical condition and consequence: if P, then Q. To fully prove if the rule is valid, two criteria listed below need to be confirmed:
If P, then QIf not Q, then not P
The fact that, on average, only 10% of the Wason selection task participants were entirely right by including the 2nd choice shows human brains automatically focus on the positive evidence to ratify a conclusion but have difficulty checking evidence that may refute the rule.
Interestingly, most people get it right quickly if we add social context to the puzzle, mainly concerning permissions or duties. A popular example is like this: the rule is that if you are under 21, you cannot drink alcohol. Suppose there are four cards — Beer, 28, Coke, 15. Which card(s) must you turn over to test the rule to be true or not? Most people would give the correct answers quickly by intuition: turn over the Beer card to see if the age on the opposite side is above 21, and turn over the 15 card to find out if the other side lists an alcoholic beverage.
What Causes Confirmation Bias?
What do the results from the Wason selection tasks imply? Scientists are still researching what neural circuits and mechanisms can explain confirmation bias. But we can derive two things:
The human brain is not a logical operator to solve this type of logic problem using symbols and tokens.The bias can be overcome with social context when humans have previous experience with the rules in social situations.
Given the learning mechanism of neural networks in both biological brain and artificial learning (see the article “From Biological Learning to Artificial Neural Network: What’s Next?”), the confirmation bias may be a by-product of how neural networks work for pattern recognition. For an existing belief or rule, a neural network has learned by strengthening the involved neural connections with the input precedent condition. Similar evidence would activate the same network that leads to the same conclusion while reinforcing the neuronal connections simultaneously. On the contrary, to approve the opposite side, the network needs to be trained separately by different input data (not P) to get a different conclusion (not Q). In other words, the network likely involves different neuronal connections to learn separately. Because of the barrier and the effort required to establish another neural output for understanding the opposite effects, human brains are predisposed to the existing brain circuitry.
When people obey a social rule, they know they will get punished or pay a certain cost if they do not follow it. This opposite scenario had been thought of and built into the brain’s circuitry, which explains why humans have no difficulty seeing the other side when solving a puzzle in the social context. In other words, the brain has learned the scenarios of both sides by empirical data in the first place.
However, there is another way to prevent the confirmation bias. It is to use a tool beyond our native brain capacity. We usually call these mental tools. For Wason selection tasks, the device is the simple logic:
If P, then QIf not Q, then not P
Suppose we plug the precondition (P) and the consequence (Q) of each rule into both scenarios above; we will get the puzzle 100% correct, whether related or unrelated to social context. In other words, a mental tool as simple as using both positive and negative logic can help us to think clearly without overlooking the opposite side by intuition.
In the real world, outside the labs, however, the rules are more complex and implicit. It is where data science and AI can help humans overcome confirmation biases by leveraging the same principle and strategy.
How can Data Science and AI Overcome Confirmation Biases?
Given the similarities in learning between biological neural networks and artificial neural networks, we do not want AI to repeat the same bias from humans in the first place. While it is difficult for humans to overcome the confirmation biases, AI and machine learning have the following advantages to overcome them:
The models and algorithms interpret the training data in a pre-designated way. Therefore, they do not have the interpretation or favor problems over opposite facts as humans do.The collaborative teamwork from data scientists and engineers against AI makes it more objective, contrasting with the human bias from each individual’s point of view.It is flexible for AI to add additional statistical and logical tools and processes to prevent biases from happening.
On the other hand, because AI and machine learning depend on human-curated training data, extra precautions should be given to prevent biases from being introduced to the models. Therefore, we should focus on three areas to overcome the confirmation biases: training data, model testing and monitoring, and interpretability of the results.
Ensure Training Data is not Biased
Since the beginning of Data Science, one of our slogans has been to make “data-driven decisions.” But as we all know, data can be incomplete, junky, or downright wrong, which is a major peril that could lead AI to make bad or biased decisions. Making the training data correct and complete with confirming and disconfirming facts is the prerequisite to eliminating confirmative bias.
For example, suppose we build a model to forecast the growth of subscribers. In addition to searching for the features relevant to the subscribers, have those related to non-subscribers been explored? Could some features have contributed to subscribing and non-subscribing at the same time? Is the training data limiting or straying our forecast and decisions?
Ensuring training data has all-sided facts equally represented is one of the most critical steps to ensure AI does not inherit biases from humans. Because AI models depend on the data that humans have collected and curated and humans tend to have confirmation bias, when designing a training model, ensuring the data has both positive and negative scenarios is a must to ensure the model is not biased. However, it usually involves different data sources and requires multiple data collection and curation methods. In some cases, if the opposite data does not exist or is costly to gather or collect, data synthesis might be the solution to simulate contrasting scenarios.
2. Prevent Biases by Thorough Testing and Validations
ML and AI already have the automated test process to validate a model. The purpose of the validation, however, usually centers around the repeatability of predictions, ensuring model generalization without overfitting, and removing outliers from statistical distributions. Preventing confirmation bias would require extra steps to validate a training set and a model’s behaviors and outputs. For example, can the model both confirm and disconfirm a hypothesis? Are there any fallouts or anomalies due to small samples of negative cases? Have any features been under-represented due to human interference deemed unimportant?
Identification of confirmation bias is not a one-time task. In the ever-changing world, new negative or contradictory facts could emerge unexpectedly. A recent exception could become the new norm in the future. An exception-handling routine in data curation should be examined regularly to identify if it deletes actual opposite cases. In addition, regular auditing and testing should be done to ensure biases are not introduced after a model is launched.
3. Demonstrate the “Thought” Process
From human experience, our thought processes and mental models are critical for making the right decisions. We should assess and understand how an AI model reaches the decision or conclusion. An obvious advantage of AI is that it can have many data scientists and engineers work together to assess its decision-making process at the same time, while humans can only do so individually in an individual’s mind.
However, neural networks and deep learning models are notorious for being uninterpretable. Given this, deduction of the process and hybrid approaches may be needed to understand if a decision or conclusion is biased:
Thorough understanding of where the training data comes from and how it is processed and used for the models.Improve models’ interpretability using ad hoc processes and available libraries (e.g., LIME — Local Interpretable Model-agnostic Explanations, SHAP — SHapley Additive exPlanations)Leverage visualizations (e.g., graphs and charts) to illustrate not only the final results but also the end-to-end process from source data to training and model executions, such as the quality of training data, the parameter instances supporting each decision-making, the consistency of the output categories over time, any fallouts or outliers during the process, etc. In this way, it is easier for data engineers and data scientists to identify at which step the model could have been biased and what data or training is still needed.
Conclusion
Throughout history, humans have been good at inventing tools to overcome their limitations and constraints. Given the similarities and differences between human and AI intelligence, we should focus on how AI can complement what human intelligence lacks and prevent humans’ cognitive biases. While it is difficult for humans to overcome those biases, data science and AI could help us identify and minimize them while making the process more visible. Even though we have focused our discussions on confirmation bias in this article, similar principles and methods may also be applied to tackle other cognitive biases.
Can AI Overcome Human’s Confirmation Bias? was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.
