Introduction: FDA Human Factors FAQs
When the FDA released its guidance document Applying Human Factors and Usability Engineering to Medical Devices in 2016, the intent was to provide a clear picture of the human factors and usability methods that medical device manufacturers should implement into their product development processes. A few of the guidelines include:
- The manufacturer should conduct a simulated-use validation usability study
- Validation testing should evaluate all of the device’s critical tasks
- Studies should include a minimum of 15 users per user group
- The results of validation testing should demonstrate that intended users can use the device safely and effectively
Although the Guidance intends to provide clarity, there are often a number of questions that arise when a manufacturer attempts to implement them into practice. Consequently, Research Collective is routinely asked a variety of clarifying questions, such as:
- How realistic does a simulated-use usability study need to be?
- What makes a task critical?
- Can we include nurses and physicians in one user group?
- What is the minimum number of task failures that the FDA will accept?
- Can changes be made to the device or instructions after validation testing?
Although the answers to these uncertainties largely depend on the specifics of each device, we have put together a list of some frequently asked questions (FAQs) and our frequently answered responses.
How do you define critical tasks?
The FDA’s critical task definition is a good place to start. A critical task is a user task which, if performed incorrectly or not performed at all, would or could cause serious harm to the patient or user. Additionally, the definition of harm includes compromised medical care.
Critical tasks should be determined from severity scores given in a risk analysis or failure modes and effects analysis (FMEA). While the risk analysis or FMEA traditionally gives each user task two scores, severity and probability, critical tasks do not concern probability. In short, any task that could lead to harm, regardless of likelihood of occurrence, should be listed as a critical task.
Manufacturers often miscategorize tasks related to successful delivery of therapy as non-critical tasks. Generally, this is not appropriate. Above all, the FDA expects to see evidence that a device can be used for its intended purpose, otherwise what is the value of that device?
What is the purpose of identifying “known issues” and how do we identify them?
Known issues refer to usability issues that have already been documented from previous versions of the device, predicate devices, or devices that are similar. This is the FDA’s way of saying, “don’t make the same mistake twice.” By identifying usability issues with current and previous devices, you can ensure the design of the new device avoids those same problems.
There are multiple places to look when searching for known issues related to usability. These might include:
- Internal customer feedback and complaints
- Sales representatives and clinical support staff
- Observing or interviewing current users
- Searching external databases like the FDA’s Manufacturer and User Facility Device (MAUDE) database, FDA’s MedSun Network, FDA’s Safety Communications, and FDA’s Medical Device Recalls website
How many use errors will the FDA accept?
To quote the FDA directly, “it depends.” Frustratingly, it’s an unsatisfying answer, but it’s true. The answer really depends on a number of factors:
- Type of device and potential harm
- The severity of the potential consequences of use errors
- The number of tasks that have use errors
- The number of use errors on a given task
- Whether a design change can be made to correct the problem
- Whether a design change is made to correct the problem
- The reviewer’s background
- Whether you already attempted a submission
Each use error, and difficulty or close call, require an in-depth analysis to determine the root cause. From there, you will need to decide if the residual risk requires additional mitigation, or explain why the residual risk is acceptable as is.
How realistic does a simulated-use usability study need to be?
The goal of a simulated-use usability study is to observe users interacting with the device interface independently and naturally. Accordingly, the FDA guidance recommends that testing take place in realistic, but simulated use scenarios. There are three human factors considerations that interact to produce “device use”: the user, the device interface, and the use environment. Ultimately, the simulated-use environment should be realistic enough to understand how the device, user, and use-environment all interact.
The design of simulated-use usability studies should consider all aspects of the use environment that may influence the user’s interaction with the device. Some of these aspects may include:
- Lighting and ambient noise levels
- Presence of multiple models of the same device (forcing users to identify the correct one)
- Distractions
- Busy or cluttered environments
- Pace of the environment
Can we include nurses and physicians in one user group?
The FDA recommends that validation usability studies include 15 participants per user group. In an effort to save time and resources, medical device manufacturers often attempt to simply combine user groups. While there are situations where this can be appropriate, the answer to the question is probably not.
Nurses and physicians are actually quite different from a human factors perspective. In addition to having different training and experience, the way nurses and physicians use a medical device are usually different, too. The tasks a nurse must complete do not typically align with those completed by physicians. Therefore, we usually separate nurses and physicians for validation usability testing.
Can we make changes to the device or instructions after the validation usability study?
Yes. Importantly, however, it is likely the FDA would ask for a reevaluation of the tasks that were affected by the change. To be clear, that does not mean a complete reevaluation; only the tasks affected by the manufacturer’s change would need further evaluation. For example, if you changed the wording of only one step in the instructions, the FDA would likely ask for only that step to be reevaluated.
Completing a reevaluation of only a few tasks is obviously a simpler and quicker affair, but it is important to note that a full 15 new participants would have to be included. Additionally, in order to reevaluate Step 4 of the instructions, you would ask the new participants to complete Steps 1-9 so each user has the proper context.
We hope that you find our FAQ responses to be helpful! Please contact us if any of the answers listed here need further clarification. We would be happy to discuss your human factors needs with you.
What other questions might you have? Leave a comment! We would love to hear from you.