Internal vs External Validity | Differences & Examples
When testing cause-and-effect relationships, validity can be split up into two types: internal validity and external validity.
The validity of a study is largely determined by the experimental design. To ensure the validity of the tools or tests you use, you also have to consider measurement validity.
Trade-off between internal and external validity
Better internal validity often comes at the expense of external validity (and vice versa). The type of study you choose reflects the priorities of your research.
A solution to this trade-off is to conduct the research first in a controlled (artificial) environment to establish the existence of a causal relationship, followed by a field experiment to analyse whether the results hold in the real world.
Threats to internal validity
There are eight factors that can threaten the internal validity of your research. They are explained below using the following example:
They set up an experiment with two groups: 1) a control group of employees with fixed working hours, and 2) an experiment group with employees with flexible working hours.
The experiment will run for six months. All employees fill in a survey measuring their job satisfaction before the experiment (pre-test) and after the experiment (post-test).
| Threat | Explanation | Example |
|---|---|---|
| History | Unanticipated events change the conditions of the study and influence the outcome. | A new (better) manager starts during the study, which improves job satisfaction. |
| Maturation | The passage of time influences the dependent variable (job satisfaction). | During the six-month experiment, employees become more experienced and better at their jobs. Therefore, job satisfaction may improve. |
| Testing | The pre-test (used to establish a baseline) affects the results of the post-test. | Employees feel the need to be consistent in their answers in the pre-test and post-test. |
| Participant selection | Participants in the control and experimental group differ substantially and can thus not be compared. | Instead of a randomly assigning employees to one of two groups, employees can volunteer to participate in an experiment to improve job satisfaction. The experimental group now consists of more engaged (more satisfied) employees to begin with. |
| Attrition | Over the course of a (longer) study, participants may drop out. If the dropout is caused by the experimental treatment (as opposed to coincidence), it can threaten the internal validity. | Really dissatisfied employees quit their job during the study. The average job satisfaction will now improve, not because the ‘treatment’ worked, but because the dissatisfied employees are not included in the post-test. |
| Regression towards mean | Extreme scores tend to be closer to the average on a second measurement. | Employees who score extremely low in the first job satisfaction survey probably show greater gain in job satisfaction than employees who scored average. |
| Instrumentation | There is a change in how the dependent variable is measured during the study. | The questionnaire in the post-test contains extra questions compared to the one used for the pre-test. |
| Social interaction | Interaction between participants from different groups influences the outcome. | The group of employees with fixed working hours are resentful of the group with flexible working hours, and their job satisfaction decreases as a result. |
Threats to external validity
There are three main factors that might threaten the external validity of our study example.
| Threat | Explanation | Example |
|---|---|---|
| Testing | Participation in the pre-test influences the reaction to the ‘treatment’. | The questionnaire about job satisfaction used in the pre-test triggers employees to start thinking more consciously about their job satisfaction. |
| Sampling bias | Participants of the study differ substantially from the population. | Employees participating in the experiment are significantly younger than employees in other departments, so the results can’t be generalised. |
| Hawthorne effect | Participants change their behaviour because they know they are being studied. | The employees make an extra effort in their jobs and feel greater job satisfaction because they know they are participating in an experiment. |
There are various other threats to external validity that can apply to different kinds of experiments.
Frequently asked questions about internal and external validity
- What is the difference between internal and external validity?
-
Internal validity is the degree of confidence that the causal relationship you are testing is not influenced by other factors or variables.
External validity is the extent to which your results can be generalised to other contexts.
The validity of your experiment depends on your experimental design.
- What are threats to internal validity?
-
There are eight threats to internal validity: history, maturation, instrumentation, testing, selection bias, regression to the mean, social interaction, and attrition.
- What are the two types of external validity?
-
The two types of external validity are population validity (whether you can generalise to other groups of people) and ecological validity (whether you can generalise to other situations and settings).
- What are threats to external validity?
-
There are seven threats to external validity: selection bias, history, experimenter effect, Hawthorne effect, testing effect, aptitude-treatment, and situation effect.
- What is the definition of an experimental design?
-
Experimental designs are a set of procedures that you plan in order to examine the relationship between variables that interest you.
To design a successful experiment, first identify:
- A testable hypothesis
- One or more independent variables that you will manipulate
- One or more dependent variables that you will measure
When designing the experiment, first decide:
- How your variable(s) will be manipulated
- How you will control for any potential confounding or lurking variables
- How many subjects you will include
- How you will assign treatments to your subjects
Cite this Scribbr article
If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.
Streefkerk, R. (2022, October 10). Internal vs External Validity | Differences & Examples. Scribbr. Retrieved 22 November 2024, from https://www.scribbr.co.uk/research-methods/internal-vs-external-validity/
