No matter the product—an inhaler, a surgical robot, a medication pump, or a smartphone application—following a human factors engineering process will use usability studies. Usability testing is a research method designed to evaluate a product’s ability to be used correctly by its intended users. It is worth highlighting that the purpose of a usability study is not to determine if intended users can use medical devices as intended (and without errors). Although the two definitions are nearly identical, the former keeps the focus of the evaluation on the product whereas the latter implies that the intended users (study participants) are the subject of the test.
Usability study participants are obviously a critical component of the usability study. Without them, we would be left on our own to evaluate the design of the product. While we have methods such as expert reviews and heuristic evaluations at our disposal, human factors practitioners know the real test of the product’s design is placing it in the hands of real people, sitting back, and watching them figure out how to use it.
Human factors studies evaluate the product, not the user.
Study participants represent a sample of the population of people intended to use a product. It should be expected, or even celebrated, that they come in all shapes and sizes, with varied abilities, limitations, life experiences, strengths, and weaknesses.
Participants that use the product correctly, refer to the instructions when they get confused, and answer our questions accurately are not “good” participants. They are simply participants.
Meanwhile, participants that toss the instructions in the garbage, make mistake after mistake, and do not notice the bolded/highlighted/circled warnings on the box are not “bad” participants. They are simply participants.
If we were evaluating people, we might claim participants that did everything correctly are better. However, we are not, and will never, evaluate people in a usability study. Both types of participants provide valuable information regarding the quality of the product’s design.
Blaming the user for errors accomplishes nothing.
It is easy for blaming the user to creep into a usability study or the subsequent analysis of results. After all, participants will often blame themselves for errors they made. “Oh, I am such an idiot, I can’t believe I forgot to press ‘Confirm’.”
Maybe the participant IS an idiot, but if our goal isn’t to make products idiot-proof then what is it? Are we not all idiots sometimes? I sure know I am.
If you are involved in designing products, you know how difficult it is to design a product that is useful, usable, and safe. If we want a mere opportunity to truly make a usable product, we cannot afford to blame participants for the errors they make in usability studies. We are evaluating products, not people. Use errors, difficulties, and close calls are the result of a flaw in the design of the product. Period.
No, seriously.
I highly recommend a thorough and regular reading of Medical Device Use Error: Root Cause Analysis by Michael Wiklund, Andrea Dwyer, and Erin Davis. They posit that blaming the user for an error should be your last recourse. They also suggest that only 1 in 100 use errors is purely the user’s fault (Wiklund, et al., 2016).
Another great book with rationale for not blaming participants is Usability Testing of Medical Devices by Michael Wiklund, Jonathon Kendler, and Allison Strochlic. They write, “Why not promote ‘forgetfulness’ as a use error’s root cause if the participant who committed the use error said, ‘It was my fault…I just forgot to do it’? The answer is that forgetfulness is a known human trait and one that should be considered in medical device development. Sure, it might seem that the user was at fault… However, anticipating the potential for such a failure, the [user interface] design could have averted failure by several means” (Wiklund, et al., 2016).
Conclusion
In conclusion, it is important to ensure that the subject of usability testing is always the product’s design, not the people who use the product. When analyzing study results to determine the potential root causes of use related issues, always consider first the likelihood of a shortcoming in the design of the product before attributing blame to the user. If you find that you have listed a user-focused root cause of an error (meaning that it blames the user), consider asking yourself if there is anything the product could have done to prevent that error from happening.
We recently created a video on this topic. Check it out here:
Contact us to learn more about how we can assist with your human factors needs.
References
Nielsen, J. (1994). Usability engineering. Morgan Kaufmann.
Norman, D. (1986). User-centered system design. New perspectives on human-computer interaction.
Sanders, M., and McCormick E., Human Factors in Engineering and Design. New York: McGraw Hill; 1993.
Wiklund, Michael; Dwyer, Andrea; Davis, Erin. Medical Device Use Error: Root Cause Analysis (p. 63). CRC Press.
Wiklund, M., Kendler, J., Strochlic, A. (2016) Usability Testing of Medical Devices (2nd ed.). Boca Raton, FL: Taylor & Francis/CRC Press.
