Publication bias refers to the selective publication of research studies based on their results. Here, studies with positive findings are more likely to be published than studies with negative findings.
Positive findings are also likely to be published quicker than negative ones. As a consequence, bias is introduced: results from published studies differ systematically from results of unpublished studies.
Example: Publication biasIn 2014, Franco et al. studied publication bias in the social sciences by analysing a sample of 221 studies whose publication status was known. The sample was drawn from an archive called Time-sharing Experiments in the Social Sciences (TESS).
Because TESS proposals undergo rigorous peer review, the sample studies drawn from the archive were all considered to be of high quality. Additionally, researchers could see in this archive whether the studies were eventually published or not.
The authors found that only 10 out of 48 null results were published, while 56 out of 91 studies with strongly statistically significant results made it into an academic journal.
In other words, there was a strong relationship between the results of a study and whether it was published, a pattern that indicates publication bias.
Publication bias can affect any scientific field, leading to a biased understanding of the research topic.
Recall bias refers to systematic difference in the ability of participant groups to accurately recall information. Observational studies that rely on self-reporting of past behaviors or events are particularly prone to this type of bias.
Example: Recall biasParents whose children have developed asthma are likely to be quite concerned about what may have contributed to their child’s condition.
As a result, if asked by a researcher, these parents are more likely to recall relevant details, such as changes in their children’s breathing when active or resting, than parents of children without any health issues.
They’ve also already associated possible triggers, such as certain foods, environments, or other allergens, with their child’s asthma. This difference in the ability to recall information results in recall bias.
Recall bias threatens the internal validity and credibility of studies using self-reported data.
Response bias refers to several factors that can lead someone to respond falsely or inaccurately to a question. Self-report questions, such as those asked on surveys or in structured interviews, are particularly prone to this type of bias.
Example: Response biasA job applicant is asked to take a personality test during the recruitment process. One of the questions is ‘Do you like meeting new people?’
The applicant thinks that, since this is a customer service job, the company is probably looking for someone who enjoys meeting new people. Despite being an introvert at heart, the applicant answers ‘yes’ in an attempt to increase their chances of being hired.
Because respondents are not actually answering the questions truthfully, response bias distorts study results, threatening the validity of your research. Response bias is a common type of research bias.
The placebo effect is a phenomenon where people report real improvement after taking a fake or nonexistent treatment, called a placebo. Because the placebo can’t actually cure any condition, any beneficial effects reported are due to a person’s belief or expectation that their condition is being treated.
Example: Placebo effectYou participate in a double-blind clinical trial on a new migraine medication. For the next month, each time you experience a migraine, you are instructed to take a pill and rate the pain intensity.
You feel that the pill relieves the symptoms, but at the end of the month you find out that you were given a placebo—and not the new medication. The perceived improvement you experienced was due to the placebo effect.
The placebo effect is often observed in experimental designs where participants are randomly assigned to either a control or treatment group.
Ascertainment bias occurs when some members of the target population are more likely to be included in the sample than others. Because those who are included in the sample are systematically different from the target population, the study results are biased.
Example: Ascertainment biasSuppose you are investigating the ratio of people who identify as male or female in a certain area. You draw your sample from a housing project for elderly people. Because, statistically speaking, women tend to live longer than men, your results could be biased in favor of women, with women overrepresented in your sample.
Ascertainment bias is a form of selection bias and is related to sampling bias. In medical research, the term ascertainment bias is more common than the term sampling bias.
Regression to the mean (RTM) is a statistical phenomenon describing how variables much higher or lower than the mean are often much closer to the mean when measured a second time.
Regression to the mean is due to natural variation or chance. It can be observed in everyday life, particularly in research that intentionally focuses on the most extreme cases or events. It is sometimes also called regression toward the mean.
Example: Regression to the meanRegression to the mean can explain the so-called ‘Sports Illustrated jinx’. This urban legend claims that athletes or teams that appear on the cover of the sports magazine will perform poorly in their next game.
Players or teams featured on the cover of SI have earned their place by performing exceptionally well. But athletic success is a mix of skill and luck, and even the best players don’t always win.
Chances are that good luck will not continue indefinitely, and neither can exceptional success.
In other words, due to RTM, a great performance is more likely to be followed by a mediocre one than another great one, giving the impression that appearing on the cover brings bad luck.
Regression to the mean is common in repeated measurements (within-subject designs) and should always be considered as a possible cause of an observed change. It is considered a type of information bias and can distort research findings.
Generalisability is the degree to which you can apply the results of your study to a broader context. Research results are considered generalisable when the findings can be applied to most contexts, most people, most of the time.
Example: GeneralisabilitySuppose you want to investigate the shopping habits of people in your city. You stand at the entrance to a high-end shopping street and randomly ask passersby whether they want to answer a few questions for your survey.
Do the people who agree to help you with your survey accurately represent all the people in your city? Probably not. This means that your study can’t be considered generalisable.
Survivorship bias occurs when researchers focus on individuals, groups, or cases that have passed some sort of selection process while ignoring those who did not. Survivorship bias can lead researchers to form incorrect conclusions due to only studying a subset of the population. Survivorship bias is a type of selection bias.
Survivorship bias exampleA hospital is conducting research on trauma patients admitted to the ER, seeking to find out which procedures work best. However, researchers can only begin their studies if a patient is stable enough to give consent. Because the trial excludes everyone who didn’t survive their injuries or is too sick to consent, survivorship bias may occur.
In addition to being a common form of research bias, survivorship bias can also lead to poor decision-making in other areas, such as finance, medicine, and business.
The Pygmalion effect refers to situations where high expectations lead to improved performance and low expectations lead to worsened performance. Although the Pygmalion effect was originally observed in the classroom, it also has been applied to in the fields of management, business, and sports psychology.
Example: Pygmalion effectYou want to research the influence of two storytelling methods on the vocabulary size improvement of children. To test this, the children are either given 20 minutes of storytelling from their teacher or 20 minutes of computerised storytelling.
You strongly believe the human aspect is needed to aid in the vocabulary development of children. You encourage the children in that group to pay attention and be excited, whereas you don’t show this behaviour to the computer group.
The children in the first group are now paying more attention and feeling better about themselves than children in the other group, potentially leading to a Pygmalion effect.
The Pygmalion effect is also known as the Rosenthal effect, after the researcher who first observed the phenomenon.
Selection bias refers to situations where research bias is introduced due to factors related to the study’s participants. Selection bias can be introduced via the methods used to select the population of interest, the sampling methods, or the recruitment of participants. It is also known as the selection effect.
Example: Selection biasHealth studies that recruit participants directly from clinics miss all the cases who don’t attend those clinics or seek care during the study.
Due to this, the sample and the target population may differ in significant ways, limiting your ability to generalise your findings.
Selection bias may threaten the validity of your research, as the study population is not representative of the target population.
