The COVID-19 pandemic has put a serious damper on the world of in-person usability research. For good reason, a huge portion of the U.S. population is under stay-at-home orders. It is therefore very difficult to ask users to interact with your product in person. Much of the UX, usability, and human factors community has therefore shifted heavily towards conducting research in a socially-distant way. While many of us have conducted remote research before, experience levels may be questionable. There are many factors to consider when implementing the transfer.
Below are a few important steps to consider as you navigate conducting remote research:
Define which type(s) of study you want to conduct
The first thing you’ll need to do is identify what kinds of projects are coming down the pipe. Do you need in-depth interviews (IDIs) which require a live moderator? Did you have a focus group in mind? Can the participant be recorded when they’re available and reviewed a later time? Of course, an important variable is whether the product under evaluation is entirely digital or if parts of the interface require physical interaction.
Define the requirements for the testing platform
A large portion of remote studies will use an online testing or virtual meeting platform. Identify what your studies need from these platforms in order to be successful. This will be most efficient if you can identify all the studies slated to occur during the next several months because you can define requirements based on each of these studies. It might be that one study needs transcription services while another requires the ability to control the participant’s screen. Many platforms require a full year commitment; planning ahead will allow you to make informed investment decisions that are more likely to pay off.
Here are some common capabilities research teams like to have. Keep in mind that common meeting platforms such as Zoom, GoToMeeting, Skype, and Teams have some of these (budget saver!)
- Audio and video — Can you hear and see the participant?
- Recordings — Is your session being reliably recorded?
- Transcription — Are transcription services included in the subscription/license?
- Screenshare — Can you share your screen with the participant?
- Control share — Can you take control of the contents of the participant’s screen?
- Share securely — Will the participant have access to the testing materials after the session is over?
- Markup — Can the participant draw or annotate on the screen?
- Virtual backroom — Does the platform allow for observers? Are they hidden from the participant?
- Timestamped notes — Are the notes you take timestamped?
- Recruiting and participant management services — Do you want recruiting help from the platform or would you like to take care of it by other means?
Find a remote research platform (or two)
Once you’ve identified what you need out of a remote testing platform, you can begin your search for one. There are several resources available online to help you generate a preliminary list, but we strongly recommend signing up for free trials and live demos.
Budget at least a week to perform this search. Setting up demos and practicing with free trial versions is time-consuming. It can also require help from coworkers or friends to really try things out, because it is important to understand how the virtual session will work from both the moderator and participant points of view.
Depending on your needs, you may not find a platform that provides all the capabilities you’re looking for. In that case, it’s a matter of prioritizing which capabilities are most important, then comparing costs and benefits of each option. It is not uncommon to use standard teleconferencing platforms simply because of the bang for their buck.
Modify the study protocol and recruiting procedures
When recruiting participants for a remote study, ensure the screener is compatible with the new study format. For example, it is standard to exclude people who do not have access to a computer, webcam, or stable internet connection. It may also be wise to ask potential respondents if they have access to a quiet area, away from interruptions and distractions.
Of course, the study itself will be very different from an in-person format. Especially if remote testing is new to you, prioritize familiarization of your platform to account for the various modality changes. Writing a protocol for remote research is an opportunity to get creative, but don’t force it. Evaluate what you can and save the unrealistic portions for another time.
Think about accessibility for your research participants
Not all users are tech-savvy. Not everyone knows how to join a virtual meeting. Make it as easy as possible for participants to be involved in your study.
Consider adding training appointments prior to the session so that you can be sure the participant is able to get online. Ensure their audio, video, and screen sharing capabilities are functional, and if they aren’t, help them troubleshoot the issue. It is also smart to simply increase the session length to account for technological difficulties.
Rather than written instructions, it may also be worthwhile to explore creating tutorial videos for participants. If your setup is overly complex, consider getting participants setup on a control-share platform such as Team Viewer so that you can get your participants set up the rest of the way.
Be flexible and have fun with it!
This is all new to a lot of us. Nothing can replace understanding, patience, and a good attitude. At the end of the day, remote usability testing is a fantastic option even when in-person studies are possible. It is a great way to gather the insights you’re after and learn new research techniques at the same time. In many cases, remote research is actually much faster, efficient, and more powerful than traditional research methods. Have fun adding it to your repertoire and good luck!
- The Conference, Reimagined: HFES Virtual Healthcare Symposium
- Remote Research: Conducting Research During COVID-19
- In-Person Research Methods: Conducting Research During COVID-19
- Virtual Tour of RC: What are you doing to stay productive working from home?
- Coronavirus: Tips for Converting to Remote Research