Oliver Latouf

The art & design of Oliver Latouf

Ad example
You are here: Home / User Testing

User Testing

  • What is Usability Testing?
  • Types of testing
  • What to test for
  • Methods
  • Books and resources

This article is based off the following books and sites:

What is usability testing

The process of watching users interact with a product you’re designing, and using the results of that interaction to improve the usability of the product. Usability testing forms part of the holistic User Experience offering, but has tangible measurable results which helps stakeholders and designers made rapid decisions, resulting in fast iterative designs.

Types of usability testing

  • Qualitative
  • Quantitative
  • Summative: large user base (25-50 users) to gather metrics for analysis and benchmarking in a more formal format (eg. Remote testing)
  • Formative: smaller sample size (approx 5 users) for informal testing
  • Formal, informal, large sample, small sample, do-it-yourself, benchmarking tests, and more

Metrics

  • Quantitative metrics
    • The number one metric to test for is Ease-of-Use or Success Rate – how many users were able to successfully complete a task. Do the testers “get the task”?
    • Time on task – how long did users take to complete the set of predetermined tasks.
    • Number of clicks to success
    • Number of errors – the number of errors the user made while completing a task.
    • Failure rate – the number of times a user could not complete a task.
  • Qualitative metrics
    • Stress responses – when the user appears stressed
    • Confusion – when the user appears confused or expressed confusion
    • System Usability Survey (or similar survey taken after the test)
  • Specific to mobile
    • How many interruptions in a flow (can the interface accommodate interruptions seamlessly?)
    • Is navigation apparent at all times (visibility of controls)
    • Does the interface sustain orientation changes?
    • Is one-handed operation possible?

How do we measure these metrics

One method is by using the HEART method.

Created by Telepathy in collaboration with Google Ventures, it groups the important metrics into 5 categories.

  • Happiness: measures of user attitudes, often collected via survey. For example: satisfaction, perceived ease of use, and net-promoter score.
  • Engagement: level of user involvement, typically measured via behavioral proxies such as frequency, intensity, or depth of interaction over some time period. Examples might include the number of visits per user per week or the number of photos uploaded per user per day.
  • Adoption: new users of a product or feature. For example: the number of accounts created in the last seven days or the percentage of Gmail users who use labels.
  • Retention: the rate at which existing users are returning. For example: how many of the active users from a given time period are still present in some later time period? You may be more interested in failure to retain, commonly known as “churn.”
  • Task success: this includes traditional behavioral metrics of user experience, such as efficiency (e.g. time to complete a task), effectiveness (e.g. percent of tasks completed), and error rate. This category is most applicable to areas of your product that are very task-focused, such as search or an upload flow. 1

Software options for Usability Testing

1Taken from the article: How to choose the right UX metrics for your product