Research Design: Understanding the basics of within-subjects and between-subjects designs is crucial for any decision-maker who is conducting research. Participant design is a core concept, yet even experienced researchers sometimes find difficulty in defining them off the cuff. I admit it. A quick meaning confirmation with our good friend, Google, can be found in the depths of my search history more than once.
Here are the essentials: in a between-subjects design, two or more subject groups experience their own unique condition. If we wanted to compare the desirability of apples to oranges, one group of participants would eat an apple and the other group would eat an orange. In a within-subjects design, all participants experience each condition; each participant would eat an apple and an orange.
That’s pretty easy; if you just needed a refresher on which one is which, that did it for you. But let’s be honest, what you’re interested in is when should I use each? This is a much more important and practical question to answer.
Please forgive us in advance, for we are about to embark on a critically important, hypothetical journey of evaluating apples and oranges:
When it comes to non-academic research, between-subjects designs are beneficial because they offer more control and can save you vast amounts of time if you run multiple sessions simultaneously. However, because each subject experiences only one condition, either apples or oranges, the number of participants required to compare the two fruits increases; you need more participants.
Within-subjects designs, conversely, not only serve well when you want to make a more direct comparison between two or more options, but also when the number of participants is limited. Because each participant is exposed to each condition, the experimenter is getting more bang for her buck.
Now you’re saying, “Okay, great. Then I’ll just implement a within-subjects design every time.” Be careful; there’s a catch. If you’ve decided to save participants by having everyone try an apple and an orange, you may run into order effects. It could be that trying an orange first may influence how an apple tastes. In this case, participants who try oranges first may perceive the taste of apples differently from those who do not, causing unwanted manipulations of your data.
Order effects and testing effects are factors that must be considered when implementing a within-subjects design. As can be seen in the apples and oranges example, these effects typically occur when exposure to one condition changes how a participant perceives or reacts to another condition. Order and testing effects are quite common in research and there is a very reasonable way to deal with them: counterbalancing. If we counterbalance the conditions in the apples and oranges experiment, half the subjects would eat an apple and then an orange, whereas the other half of subjects would eat an orange then an apple. By doing this, order effects are reduced.
To help convey these ideas further, let’s consider a few examples that Research Collective has encountered in the past. In Study 1, Car Company A wanted to evaluate the usability of various in-vehicle infotainment systems in order to gain insight on which to use in their vehicles. Meanwhile in Study 2, Car Company B sought to conduct an ethnographic study and interviews regarding their Wheelchair Accessible Vehicles (WAVs).
The goal was to generate the highest quality data possible, in a reasonable amount of time. In Study 1, Car Company A provided four systems to test, each system taking roughly an hour to evaluate with a participant. Because of this, a between-subjects design made most sense; each participant interacted with just one system. We could do this because our participant pool was large; we only needed licensed drivers (which nearly everyone is). Implementing a between-subjects design also enabled the ability to run multiple sessions at once, speeding things up. Had we implemented a within-subjects design, each participant would have had to endure roughly four hours of tasks and interviewing. Four hours straight of anything isn’t fun, and when participants are fatigued, data quality can suffer.
In Car Company B’s project, we went with a within-subjects design. There were a couple reasons for this decision: First of all, unlike those required for Study 1, the participants available to us were at a premium. The desired subjects were families who used Wheelchair Accessible Vehicles, and there simply aren’t many people who fit that demographic. Using a within-subjects design helped us conserve participants; each family looked at both vehicles. Additionally, as a major goal was to gather preference between two WAVs, it made sense to have all participants see both. Had we implemented a between-subjects design, it would have been difficult to determine which vehicle the participants preferred as everyone would have only seen half of the options.
So there you have it. Two examples: one used a between-subjects design and the other a within-subjects design. These two studies were largely different from one another, but both scenarios were examined with various goals and constraints in order to make the most appropriate decisions. An essential point to keep in mind is that each study is unique in its own way. Deciding between between-subjects and within-subjects often just depends on what you’re after, but this post helps to outline a couple factors considered in typical studies: Between-subjects designs typically serve the researcher well when time is at a premium or testing/order effects want to be avoided, but only when there are plenty of participants available.
Within-subjects designs help to conserve participant resources and are helpful when the goal is to directly compare multiple products.