U.S. flag

An official website of the United States government

Government Website

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Safely connect using HTTPS

Secure .gov websites use HTTPS
A lock () or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Approaches to Usability Testing

Usability Testing Lesson | 15 Minutes

You have decided your materials or services are ready for usability testing but can't decide which approach might work best to meet your needs. Well, let's take a look at four different approaches to determine which one may work best for your usability team.

The approach you choose will depend on your goals, budget, timeframe, and what resources you have.

Image
4 Approaches to Usability Testing. There are many different approaches to usability testing, here are 4 that can be chosen depending on what you want to learn. Explorative – quick review of the material, used during task-based or scenario-based testing. Comparative - compare 2 or more drafts/finished services to see which the participants like best. Assessment - observe participants’ interaction with the service, used at any stage of dev. Validation – test the service, focus on quantitative data.

Explorative Usability Testing

What It Is: Explorative usability testing focuses on assessing what is and what is not working in the design of your materials. This approach is often task-based or scenario-based to establish empathy towards future users of the materials.

Best For: Early stages of development for a quick review of your material.

Example: The Secret Service (USSS) may conduct explorative usability testing on an early prototype for an internal facing web-based application to obtain feedback from their colleagues on the applications contexts and performance. The USSS usability testing team would be looking for initial thoughts and feedback on the early prototype to further develop the web-based application to be more aligned with the future users' needs.

Comparative Usability Testing

What It Is:  Comparative usability testing focuses on comparing and contrasting materials or services to see which one is more effective and efficient for the participant to use. During testing you want to obtain both quantitative and qualitative data points to compare services.

Best For: Comparing two or more drafts of materials or services to see which one works best for the participants.

Example: The Federal Law Enforcement Training Center (FLETC) may conduct comparative usability testing to determine which of two training courses is more effective to those using it. The FLETC usability testing team would be looking to test efficiency, effectiveness, and satisfaction by using both comments and metric data about each training course to determine which course to include in required law enforcement training.

Assessment Usability Testing

What it Is: Assessment usability testing focuses on observing participants' interaction with the service through task-based scenarios to obtain qualitative data based on the participants ability to complete tasks while using the service.  

Best For: Testing at any stage of a service's development to gain future user feedback. This is the most common approach to usability testing.

Example: Customs and Border Protection (CBP) may conduct assessment usability testing on a prototype midway through development to evaluate a new regulatory document for user comprehension. The CBP usability testing team would be looking to gain feedback and recommendations on the language used in the document and the clarity of the information to ensure the content was correctly understood.

Validation Usability Testing

What It Is: Validation usability testing focuses on obtaining quantitative data, such as completion rates and time taken to complete a task, to establish benchmarks to help set a standard for future designs and iterations.

Best For: Testing the final version or updated version of your service.

Example: The Coast Guard (USCG) may conduct validation usability testing on a prototype close to completion for an online supply request form. The USCG usability testing team would be looking to gain feedback on completion rate, time of completion, and an ease-of-use rating scale for the online supply request form to ensure the form can be completed with ease and in a timely manner.

In-Person... Remote... Which Testing Method Should I Use?

Your usability team's goals, time, funds, and resources will impact your decision to conduct usability testing in-person or remotely. It is important to remember that there isn’t a best method to conduct usability testing, just a more beneficial one depending on your needs.  

Image
Remote and In-Person Testing. UT can be done remotely or in-person, consider what you want to learn and the amount of time and resources available when selecting a method. Remote testing – Test moderator and participate are NOT in the same location. Typically requires less resources and planning to conduct because there is no need for travel. In-person testing – Test moderator and participant are in same location. Easier for the moderator to capture physical and emotional responses to material being tested.

Benefits of In-person Usability Testing  

  • Better insights into a participant's non-verbal communication.
  • Easier to recognize when to probe and ask questions based on participant’s body language.
  • Able to recruit participants in lower socioeconomic areas that may not have access to technology.
  • Better controlled environment to limit distractions during the testing session.
  • Testing can be conducted at earlier stages in development without running into prototype dead ends.

Benefits of Remote Usability Testing

  • Participants may feel more comfortable testing in their own space and using their own technology.
  • You can test participants outside of your geographic location.
  • Easier to recruit participants based on their availability.
  • More flexible with time and budget friendly.

Additional Testing Considerations

Image
Quantitative and Qualitative Data. Info gathered during your usability testing session will take the form of quantitative and/or qualitative data. Quantitative data focuses on collecting metrics that describe the user’s experience. Example: tracking completion rate and time for a task within the service. Qualitative data focuses on collecting insights, findings, and anecdotes about how a participant uses the service. Example: capturing participant’s pain-points by observing their frustrations while testing

Quantitative and Qualitative Data: Information gathered during usability testing sessions will take the form of quantitative and/or qualitative data.

Quantitative data focuses on collecting metrics that describe the user's experience. For example, tracking the completion time of a task within the service and success rate of the service.

Qualitative data focuses on collecting insights, findings, and anecdotes about how a participant uses the service. For example, discovering a participant's pain-points by observing and listening to their frustrations or confusions while they test a service.

Recruiting Participants: It is not required to pay your participants; however, you can give a small token of appreciation. (EX: a promotional item from your agency)

Number of User Groups: Depending on the material or service you are testing; you may choose to have multiple user groups to collect enough user feedback to make iterations.

In Summary

No matter which approach you choose, conducting a usability test brings your DHS Component one step closer to creating a more customer-focused service that will lead to a better customer experience.

Usability testing gives you important information about how real users interact with your materials and services, helping you to make informed decisions about how to create and update your services to ensure your customers' voices are heard.

  • What Did You Learn?

    Take our short knowledge check to find out how much you know about approaches to usability testing.

    Disclaimer: Responses and results are anonymous and may be used to improve the course offerings.

    Check Your Knowledge

 

  • User Acceptance & Usability Testing

    10 Minutes

    Learn the difference between user acceptance testing and usability testing, and how they can be used together to help improve service delivery.

  • Moderating Usability Testing

    10 Minutes

    Learn about the role of a moderator and how to create a step-by-step process to ensure you gain meaningful feedback during usability testing.

Last Updated: 01/24/2023
Was this page helpful?
This page was not helpful because the content