Education resources › Blog › 9 cognitive biases to look out for when engaging with research

9 cognitive biases to look out for when engaging with research

9 cognitive biases to look out for when engaging with research

5 min read
  • Becoming evidence-informed

We’re living in exciting times, with more and more educators seeking out evidence-informed practices to elevate their teaching and enhance their students’ learning experience and outcomes.

But this journey isn’t without its obstacles. Once you’ve managed to find relevant research, get past paywalls and understand what all the psych jargon means, you still need to reckon with the cognitive biases that influence our decision making, ability to identify reliable sources, and even the very research findings we’re looking at.

Unravelling and acknowledging these biases is key when engaging with research. This blog explores just nine of them, shedding light on those you will commonly face when interacting with Cognitive Science research – and providing you with tips to navigate them. So, read on to learn more about:

  1. The Sharpshooter Fallacy
  2. Selection Bias
  3. Observer Expectancy
  4. Publication Bias
  5. Response Bias
  6. Researcher Bias
  7. Recall Bias
  8. Social Desirability Bias
  9. Funding Bias

1. The Sharpshooter Fallacy

The Sharpshooter Fallacy is a fallacy based on the metaphor of a gunman shooting the side of a barn at random, then circling clusters of bullet holes to create the illusion of hitting the target accurately. This illustrates the act of cherry-picking data after the fact to fit a particular pattern or hypothesis, overlooking contradictory evidence.

How can you avoid the Sharpshooter Fallacy?

You can proactively combat the Sharpshooter Fallacy by establishing clear criteria for data analysis before collecting results. This approach encourages a comprehensive and unbiased interpretation of findings, fostering a more accurate assessment of teaching effectiveness.

2. The Selection Bias

Selection Bias occurs when the sample used in a study is not representative of the broader population, leading to skewed or inaccurate conclusions. An example of this bias may be if only high-performing students are included in a study on the effectiveness of a teaching intervention. This ultimately leads to a false perception of the intervention’s success.

How can you avoid Selection Bias?

When encountering research, assessing for Selection Bias is crucial to ensure you’re not misled by skewed findings. Critically examining the methodology section of studies can help you understand how participants were chosen. Scrutinise whether the sample size is sufficient and representative of the broader student population.

3. The Observer Expectancy Bias

The Observer Expectancy Effect, also known as the Experimenter Expectancy Effect, refers to the phenomenon where the expectations of a researcher or observer can significantly influence the outcome of an experiment or study. This bias arises when a researcher’s conscious or unconscious expectations about the results of their study affect their behaviour towards participants, which in turn influences the participants’ performance or responses.

How can you avoid Observer Expectancy Bias?

You can mitigate the impact of Observer Expectancy Bias when engaging with research by cultivating a practice of critical reflection and cross-examining studies by comparing them with other research outcomes, particularly those with conflicting results. This will help you gain a more well-rounded understanding of the topic at hand.

4. The Publication Bias

Publication Bias, commonly referred to as the “file drawer problem,” occurs when the outcome of research influences the likelihood of its publication. Studies that yield positive results are more likely to be published, while those with negative or inconclusive outcomes tend to remain unpublished.

This bias is particularly problematic in research because it can skew the perceived effectiveness of interventions, treatments, or methodologies, leading to an incorrect understanding of the subject matter.

How can you avoid Publication Bias?

You can avoid Publication Bias by actively seeking out diverse sources of information, including unpublished studies and grey literature. Emphasising the importance of considering positive, negative and inconclusive results fosters a more comprehensive understanding of educational research.

5. The Response Bias

Response Bias occurs when people fail to answer questions truthfully. In studies, participants sometimes provide inaccurate, false or biased responses, often due to social desirability or misunderstanding the survey question. This bias can distort study results, which lead to inaccurate conclusions.

How can you avoid Response Bias?

By checking and rechecking the methodology used in studies, particularly the design of surveys and questionnaires, you can assess whether they were structured in a way that minimises the possibility of biased responses.

High-impact CPD made easy. Develop evidence-informed CPD at your school, using our exclusive online collection of courses and resources.

6. The Researcher Bias

Researcher Bias involves researchers unconsciously influencing the outcomes of a study due to preconceived notions or personal beliefs. This can affect the interpretation of findings because the researcher’s expectations may lead to subjective interpretations or selective reporting of results.

How can you avoid Researcher Bias?

When reading research, try to pay close attention to the language used in the study to identify any emotive or biased terminology that could indicate a lack of objectivity. Seeking out studies that provide a balanced view, including the limitations of their findings, is crucial.

7. The Recall Bias

Recall Bias occurs when participants inaccurately remember past events or experiences, leading to biased data as the information received is incomplete or lacking in detail. Overall, this bias has an impact on the validity of study findings.

How can you avoid Recall Bias?

You can explore studies that use diverse data collection methods such as observations or longitudinal studies. These approaches can help alleviate some of the limitations associated with interviews or questionnaire-based research, which rely on participants’ memory of past events.

8. The Social Desirability Bias

Social Desirability Bias occurs when participants respond to questions in a way that they perceive as socially acceptable or favourable, rather than providing genuine responses. This bias can lead to over-reporting of positive findings or under-reporting of negative findings and affects the accuracy of self-reported data.

How can you avoid Social Desirability Bias?

You can prioritise studies that implement a mixed-methods approach, combining quantitative data with qualitative insights. This can offer a more nuanced understanding that goes beyond surface-level responses.

9. The Funding Bias

Funding Bias arises when financial interests or sponsorship influence the design, conduct or reporting of research. The integrity of research can be compromised if the funding source has a vested interest in positive outcomes, potentially leading to partial or misleading conclusions.

How can you avoid Funding Bias?

When reading research, you can look for indications of financial conflicts of interest or disclosures of funding sources in published papers. Additionally, seeking out independent and unbiased sources can provide a more objective perspective on the topic.

Final thoughts

Research will inevitably face various biases. It is crucial for educators to recognise and acknowledge these biases when engaging with research to avoid being misled by inaccurate interpretations.

By approaching research with a critical and open mind, we can effectively identify and tackle any cognitive biases present, ultimately enhancing the quality and reliability of teaching practices.

For more useful insights to help develop Teaching & Education at your school, join our Teacher CPD Academy today!

About the editor

Bradley Busch

Bradley Busch

Bradley Busch is a Chartered Psychologist and a leading expert on illuminating Cognitive Science research in education. As Director at InnerDrive, his work focuses on translating complex psychological research in a way that is accessible and helpful. He has delivered thousands of workshops for educators and students, helping improve how they think, learn and perform. Bradley is also a prolific writer: he co-authored four books including Teaching & Learning Illuminated and The Science of Learning, as well as regularly featuring in publications such as The Guardian and The Telegraph.

Follow on XConnect on LinkedIn