Usable Human Factors Reports
When it comes to producing usable results, consistently embracing the study’s goals is critical. A project’s findings are only as useful as the questions it strives to answer, so delivering results that answer trivial questions just won’t be helpful. With this in mind, let’s talk about the two main types of usability study. Each has unique objectives: formative usability studies and validation usability studies.
Formative vs. Validation
Formative studies are typically completed in the beginning and middle portions of the product development cycle; the goal is to learn as the product is developed. We observe users’ interactions with the product in order to identify strengths, weaknesses, and potential use errors.
The beauty of formative studies really come to the surface when they are used iteratively. That is to say, formative studies can be (and should be) completed many times over the course of the development lifetime; each formative study’s findings subsequently inform present design decisions, so on and so forth. This also allows engineers and designers to see the product change and grow, from concept wireframes to final product.
Formative studies can be resource-intensive and results can produce unfavorable feedback for the team members whose concepts didn’t do so well. At the end of the day, however, disappointing formative results are worth it; the most important part of a formative study is to prepare the product’s design for human factors validation testing.
Validation usability studies also strive to identify use errors, but for different reasons. While formative testing helps inform future design decisions, validation testing assumes that the little things have been figured out already. The goal is to validate, or confirm that the product is easy, comfortable, and safe to use for all users involved.
For medical devices in the U.S., validation testing hopes to demonstrate to the FDA that neither the user nor the patient will be harmed during use. In summary, one type of test helps create a good product (formative) while the other confirms the product is good (validation).
Formative and Validation Reports
Formative studies and validation studies both typically yield a report of some kind, but how do the reports differ? To start, we must refer back to the project’s goals. A project’s findings are only as useful as the questions it strives to answer. Of equal importance is consistently referring back to the original research questions when writing up the report. A project may be only as strong as its research questions, but the degree to which the report actually addresses those questions also matters. We can use the original research questions to ensure the results are on target.
Knowing the audience is also an enormous factor when constructing usable results. While the audience of a medical device’s validation test is usually the Food and Drug Administration (FDA), formative study audiences can vary. From small-group development teams, to consultant-client contracts, and even CEOs, a great number of people may see a report.
Fortunately, it is usually possible to define who the audience is early enough to tailor the report accordingly. If the product development team is the main target, you can assume that detail and specifics will be required. Alternatively, if the primary target is the CEO of the company, an executive summary may be sufficient, given that the proper steps for moving forward are clearly communicated. It can be difficult to determine some of these details if communication is poor, so it may be beneficial to simply ask (ahead of time, preferably).
As mentioned, validation study reports almost always go to the FDA. Thankfully, we can make our results usable for FDA reviewers by consulting their Guidance on Applying Human Factors and Usability Engineering to Medical Devices. For the most part, FDA’s Human Factors Guidance document does a great job outlining what they expect for a successful submission.
Conclusion
What type of study is your team delivering results for? What is the goal for the study in the first place? Knowing the answers to these questions will guide the format of the report and provide increased usability for its audiences as you continue to write it.
In order to view the other parts of this blog mini-series, please follow the links below!
Part 1: Formative vs. Validation
What are formative and validation studies? How do they differ from one another? What are their goals? How do those topics impact how the results are reported?
Part 2: Using Task Analysis
What is a human factors task analysis? What are its contents? How do you define critical tasks and task success criteria? How does its quality impact how usable results will be?
Part 3: Usability Studies
How do you collect high-quality data? How will that impact how usable results will be later in the research process.
