Usability Test

The full process of preparing and running usability tests may vary in time and scope, depending on how many scenarios you want to test and with how many user segments, among others. However, the required steps to take remain almost the same in every situation.

Briefing

Often, the Product Manager and UX Designer will brief the UX Researcher about what the project is about and which are the most flows and activities that require user validation. Try to ask enough questions to get a good overview of the product and the business requirements of the user research.

Step 1 – Prepare testing scenarios (30 MIN)

The UX Researcher and UX Designer will create concrete scenarios that the test participants should complete during the test. The scenarios should result in user feedback on the most important flows and features that you want to test.

Step 2 – Recruit test users (1-2 DAYS)

For every usability project, you should try to recruit between 5 and 10 test participants per user segment. Often, recruiting users is outsourced to external call centers, but obviously you can take up this task as well.

Step 3 – Build a prototype (1-2 DAYS)

In case the flows that will be tested are not live yet, the UX Designer (sometimes together with a UI Designer) will create new screens and make them into a prototype that will be used during the usability tests.

Step 4 – Prepare for the usability tests (2 HOURS)

• Create a testing script with the scenarios and any additional questions you want to ask. This script will be used during the tests so you can write down your observations in a structured way.
• Print out the scenarios so you can show them to the participants.
• Set up the camera, so stakeholders have the possibility to observe the tests from an observation room.

Step 5 – Moderate the usability tests (1 DAY)

Invite the test participants one-by-one to the testing location. Give them the device with the prototype and ask them to complete the scenarios. Note everything down what they say and, most importantly, do. The users’ unconscious behavior will often inform you most about what people like and what makes their experience good or bad. Always try to make the participants feel at ease. A good way is to tell them that you are testing the prototype and not them, so that they can’t do anything wrong. The users should not have the feeling that their behavior is closely analyzed. Otherwise they may potentially change their behavior, which leads to invalid observations.

Step 6 – Report on the results (2 DAYS)

Make a list of all your observations and count how many test participants showed this behavior. Try to group the observations in larger themes in order to define the key issues you want to communicate to the product team and other stakeholders. Assign a priority level to each issue and rank the issues from critical to unimportant. Create a report on all the issues using screenshots, explanatory text and, if needed, video recordings.

How to use it

When you start to analyze your test observations, make a list of all the issues you encountered. Count how many times you observed the issue and evaluate the priority level of the issue. This will help you to see greater themes among the issues and to communicate the key findings in a prioritized way to stakeholders.

Pro tips to run the template

Do not report on every observation. Group your findings in larger themes.

• Stick to validated priority levels, like the ones from Nielsen Norman.
• Priority level is more important than the frequency of occurrence. Even if every test participant encountered the same usability issue, this issue may only be cosmetic and not blocking. Meanwhile, another issue that is observed by only one user may be blocking his experience and causing him to drop out.