Top 10 Tools to Measure User Experience

By Giovanni Calabro
October 27, 2009

The one fundamental truth vital to web product development success is to know the mind of the user. Why?


Pragmatic Marketer Magazine Volume 7 Issue 5Because there is a direct correlation between techniques used to measure an online customer’s experience and the ability to persuade him or her to act. As more of our daily activities, both personal and professional, are managed online, most organizations are responsible for multiple websites and web-based tools that serve a broad range of audiences: customers, prospects, employees, channel partners, regulatory agencies, etc. That means online success depends on the ability to accurately measure a website or web application’s performance for unique target audiences on multiple levels across multiple dimensions.

However, relying on assumptions drawn from rudimentary measurements of user-centered design can’t provide sufficient data to tailor a website or web application to your users’ unique needs.

What’s more, when determining whether a website or web-based application is meeting the needs of its audience(s), it’s important to separate aesthetics from function. For example, your product may be very well-designed and visually appealing to your users but at the same time, can be functionally “unusable.”

We’ve all experienced unusable sites and applications. Whether it’s the online help site provided by your cable or satellite provider, or the online banking system you use to pay bills. Beautiful design can be painfully frustrating for users. Alternatively, poorly designed sites and applications may be quite functional, but fall far short in reinforcing the brand. Finding the appropriate balance between aesthetics and function is a difficult task. Understanding and measuring the user experience is the first step to achieving that balance. To remain competitive and retain customers, product managers need tools to optimize both aesthetics and function.

This article provides a three-staged User Experience Measurement Hierarchy and also outlines a simple approach you can use to determine the best user experience measurement tools and techniques for your web product management needs.  

User Experience Measurement Hierarchy

The majority of organizations sit at the bottom of the pyramid, having only achieved Stage One: General Knowledge of their users’ online experience. Due to the perceived cost and complexity of measuring user engagement, the widespread availability of user-centered design best practices, and significant cost pressures, this basic level is understandable. However, leading organizations in industries such as insurance, manufacturing, healthcare, travel/hospitality and financial services, have made the migration to Stage Two: Understand User Behavior, and are seeing significant return on investment  by way of increased revenue, reduced support costs and improved customer satisfaction. Even rarer are firms that have achieved Stage Three: Influence Your Users. While we’ve yet to see these tools widely adopted, they represent the future of user experience measurement.


Stages Need Recommended Tools Ideally Suited For...
Stage One
General Knowledge
Provides a basic sense of site or web application performance
  • Hueristics
  • Expert Review
  • Web Hits/Usage Analysis
Getting a big picture sense of performance and major weaknesses
Stage Two
Understand User Behavior
Identify what users are doing and where problems exist
  • User Testing
  • Session Analysis
  • Online Surveys
  • A/B and Multivariate Testing
Documenting user behavior and understanding why users are not completing tasks
Stage Three
Your Users
Determine whether a website or application is compelling
  • Eye Tracking
  • Emotion/Trust Measurement
  • Neuro-Marketing
Measuring user thinking to compel and persuade users to act

Stage One

General Knowledge

A basic sense of site or web application performance

The first stage of user experience measurement involves very little measurement at all. At Stage One, organizations are typically working with “heuristic analyses” or best practices to create assumptions about users. Whether performed by an internal usability expert or a usability consultant, the result is an “informed guess” as to what users need.

Tools to find out general knowledge about the user include:

1. Heuristics

Heuristics are de facto web standards and research from  the cognitive sciences that reveal a number of best practices in interaction styles, page layout, and visual design. Usability problems found through a heuristic evaluation are typically linked to aspects of the interface that are reasonably easy to demonstrate. For example, use of colors, layout and information structure, consistency of the terminology, or consistency of the interaction mechanism.

2. Expert Review

An expert review is a rapid survey of a website/application from specialists in user-centered design. Expert reviews can take you to the next level of user experience measurement by systematically rating the performance of your product on several relevant dimensions: navigation effectiveness, content selection, visual presentation, branding, and interaction simplicity. An expert review can often be done with a short turnaround and can be used to develop a measure of all your web assets as well as identify areas that need the most attention.

3. Web Hits/Usage Analysis

Web hits/usage analysis uses tools like Google Analytics, Omniture and WebTrends to study the hit pattern of your site or application. They can give you a sense of the number of pages visited and viewed, the typical navigation flow, and key drop-off locations. You can also track statistics such as bounce rates and conversion metrics. When combined with customer relationship management  data, web hits/usage analysis can be highly effective in mapping and understanding user behavior.

It’s important to remember, web analytics/usage metrics rely on assumptions about user behavior and how those assumptions correspond to your web goals.  So while these tools can help you determine the relative “success” of your website in meeting business goals, it’s difficult to know whether you are truly capturing the needs of your key target audience(s). The knowledge you gain through web analytics tools may be too general to understand individual user’s thinking and ultimately persuade them to act or change their online behavior.

Page 1 / 2

About the Authors

  • Giovanni Calabro has more than 10 years of experience leading interactive research and design efforts for a wide range of business sectors. Giovanni currently serves as Vice President of User Experience at Siteworx, a Reston, Virginia-based interactive agency and Web strategy consultancy.

    Giovanni is an avid writer and presenter on all topics related to the business aspects of user experience and user interface design. Contact Giovanni at, or visit  the Siteworx blog for deeper insights into user experience design and measurement techniques.

Categories:  Requirements Personas

Post a Comment


Allowed HTML: <b>, <i>, <u>