Research Design: Between-Subjects vs Within-Subjects

Research Design: Understanding the basics of within-subjects and between-subjects designs is crucial for any decision-maker who is conducting research.  Participant design is a core concept, yet even experienced researchers sometimes have difficulty. Google search histories can surely confirm this.

Here are the essentials: in a between-subjects design, two or more subject groups experience their own unique condition.  If we wanted to compare the desirability of apples to oranges, one group of participants would eat an apple and the other group would eat an orange.  In a within-subjects design, all participants experience each condition; each participant would eat an apple and an orange.

That’s pretty easy; if you just needed a refresher on which one is which, that did it for you. But when should I use each?  This is a much more important and practical question to answer.  

Between-subjects

When it comes to non-academic research, between-subjects designs are beneficial because they offer more control and can save you vast amounts of time if you run multiple sessions simultaneously.  However, because each subject experiences only one condition, either apples or oranges, the number of participants required to compare the two fruits increases; you need more participants.

Within-subjects

Within-subjects designs, conversely, not only serve well when you want to make a more direct comparison between two or more options but also when the number of participants is limited.  Because each participant is exposed to each condition, the experimenter is getting more bang for her buck.  

Order Effects and Counterbalancing

Now you’re saying, “Okay, great. Then I’ll just implement a within-subjects design every time.”  Be careful; there’s a catch.  If you decide to save participants by having everyone try an apple and an orange, you may run into order effects.  It could be that trying an orange first may influence how an apple tastes.  In this case, participants who try oranges first may perceive the taste of apples differently from those who do not, causing unwanted manipulations of your data.  

Order effects and testing effects are factors to consider when implementing a within-subjects design.  These effects typically occur when exposure to one condition changes how a participant perceives or reacts to another condition.  Order and testing effects are quite common in research and there is a very reasonable way to deal with them: counterbalancing.  If we counterbalance the conditions in the apples and oranges experiment, half the subjects would eat an apple and then an orange, whereas the other half of the subjects would eat an orange then an apple.  By doing this you can reduce order effects.

An example to consider

In Study 1, Car Company A wanted to evaluate the usability of various in-vehicle infotainment systems in order to gain insight on which to use in their vehicles.  In Study 2, Car Company B sought to conduct an ethnographic study and interviews regarding their Wheelchair Accessible Vehicles (WAVs).  

The goal was to generate the highest quality data possible, in a reasonable amount of time.  In Study 1, Car Company A provided four systems to test, each system taking roughly an hour to evaluate.  Because of this, a between-subjects design made the most sense; each participant interacted with just one system.  We could do this because our participant pool was large; we only needed licensed drivers (which nearly everyone is).  Implementing a between-subjects design also enabled us to run multiple sessions at once, speeding things up.  Had we implemented a within-subjects design, each participant would have had to endure roughly four hours of tasks and interviewing.  Four hours straight of anything isn’t fun, and when participants are fatigued, data quality can suffer.  

In Car Company B’s project, we went with a within-subjects design.  There were a couple of reasons for this decision:  First of all, unlike those required for Study 1, the participants available to us were at a premium.  The desired subjects were families who used Wheelchair Accessible Vehicles, and there simply aren’t many people who fit that demographic.  Using a within-subjects design helped us conserve participants; each family looked at both vehicles.  Additionally, as a major goal was to gather preference between two WAVs, it made sense to have all participants see both.  Had we implemented a between-subjects design, it would have been difficult to determine which vehicle the participants preferred as everyone would have only seen half of the options.

Summary

So there you have it.  Two examples: one used a between-subjects design and the other a within-subjects design.  These two studies are different, but both scenarios examine various goals and constraints in order to make appropriate decisions.  An essential point to keep in mind is that each study is unique in its own way.  Deciding between between-subjects and within-subjects depends on you’re goals, but we outline a couple factors to consider:

  • Between-subjects designs typically serve the researcher well when time is at a premium.
  • Or testing/order effects want to be avoided when there are plenty of participants available.

Within-subjects designs help to conserve participant resources and are helpful when the goal is to directly compare multiple products.

Related Posts

1 Comment. Leave new

[…] The big bonus associated with interval data is that we can now calculate all the primary measures of central tendency, which is something crucial in conducting usability research. Specifically, interval data allows us to calculate average scores — something that is of particular relevance in within and between subject designs. […]

Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

Fill out this field
Fill out this field
Please enter a valid email address.

Menu