Research Design
Understanding the basics of within-subjects and between-subjects designs is crucial for any decision-maker who is conducting research. Participant design is a core concept, yet even experienced researchers sometimes have difficulty. Google search histories can surely confirm this.
Here are the essentials, in a between-subjects design, two or more subject groups experience their own unique condition. If we wanted to compare the desirability of apples to oranges, one group of participants would eat an apple and the other group would eat an orange. In a within-subjects design, all participants experience each condition; each participant would eat an apple and an orange.
That’s pretty easy; if you just needed a refresher on which one is which, that did it for you. But when should I use each? This is a much more important and practical question to answer.
Between-subjects
When it comes to non-academic research, between-subjects designs are beneficial because they offer more control and can save you vast amounts of time if you run multiple sessions simultaneously. However, because each subject experiences only one condition, either apples or oranges, the number of participants required to compare the two fruits increases; you need more participants.
Within-subjects
Within-subjects designs, conversely, not only serve well when you want to make a more direct comparison between two or more options but also when the number of participants is limited. Because each participant is exposed to each condition, the experimenter is getting more bang for her buck.
Order Effects and Counterbalancing
Now you’re saying, “Okay, great. Then I’ll just implement a within-subjects design every time.” Be careful; there’s a catch. If you decide to save participants by having everyone try an apple and an orange, you may run into order effects. It could be that trying an orange first may influence how an apple tastes. In this case, participants who try oranges first may perceive the taste of apples differently from those who do not, causing unwanted manipulations of your data.
Order effects and testing effects are factors to consider when implementing a within-subjects design. These effects typically occur when exposure to one condition changes how a participant perceives or reacts to another condition. Order and testing effects are quite common in research and there is a very reasonable way to deal with them: counterbalancing. If we counterbalance the conditions in the apples and oranges experiment, half the subjects would eat an apple and then an orange, whereas the other half of the subjects would eat an orange then an apple. By doing this you can reduce order effects.
An example to consider
In Study 1, Car Company A wanted to evaluate the usability of various in-vehicle infotainment systems in order to gain insight on which to use in their vehicles. In Study 2, Car Company B sought to conduct an ethnographic study and interviews regarding their Wheelchair Accessible Vehicles (WAVs).
The goal was to generate the highest quality data possible, in a reasonable amount of time. In Study 1, Car Company A provided four systems to test, each system taking roughly an hour to evaluate. Because of this, a between-subjects design made the most sense; each participant interacted with just one system. We could do this because our participant pool was large; we only needed licensed drivers (which nearly everyone is). Implementing a between-subjects design also enabled us to run multiple sessions at once, speeding things up. Had we implemented a within-subjects design, each participant would have had to endure roughly four hours of tasks and interviewing. Four hours straight of anything isn’t fun, and when participants are fatigued, data quality can suffer.
In Car Company B’s project, we went with a within-subjects design. There were a couple of reasons for this decision: First of all, unlike those required for Study 1, the participants available to us were at a premium. The desired subjects were families who used Wheelchair Accessible Vehicles, and there simply aren’t many people who fit that demographic. Using a within-subjects design helped us conserve participants; each family looked at both vehicles. Additionally, as a major goal was to gather preference between two WAVs, it made sense to have all participants see both. Had we implemented a between-subjects design, it would have been difficult to determine which vehicle the participants preferred as everyone would have only seen half of the options.
Summary
So there you have it. Two examples: one used a between-subjects design and the other a within-subjects design. These two studies are different, but both scenarios examine various goals and constraints in order to make appropriate decisions. An essential point to keep in mind is that each study is unique in its own way. Deciding between between-subjects and within-subjects depends on you’re goals, but we outline a couple factors to consider:
- Between-subjects designs typically serve the researcher well when time is at a premium.
- Or testing/order effects want to be avoided when there are plenty of participants available.
Within-subjects designs help to conserve participant resources and are helpful when the goal is to directly compare multiple products.
Contact us to learn more about how we can help with your human factors research.
