CN19: Tips and Techniques for Measuring the User Experience

Wednesday, 8 April 2009, 14:30

1 unit

Instructors

Thomas S. Tullis, Senior VP of User Insight, Fidelity Investments; Adjunct Professor, Bentley College
William Albert, Director of User Experience, Fidelity Investments

Benefits

This course is designed for anyone who wants to learn more about measuring the user experience. Many aspects of the user experience can be quantified, including task completion rates, task times, and various types of subjective ratings. This course will highlight ten of the most useful tips or techniques for measuring the user experience that we’ve learned from conducting hundreds of usability studies. These are practical techniques that can be immediately applied. Real examples are given, including how to do the analyses using Excel.

Origins

An in-depth, full-day version of this course was given at the at the Nielsen Norman Group Conference in 2005 and annually at the UPA conference since 2006. In addition, we taught an expanded version as part of the Master’s program in Human Factors in Information Design at Bentley College. In May 2008 we gave a presentation at the Boston UPA Conference of the material we plan to cover in this course. The very positive reception that this “tips and techniques” presentation received inspired us to submit this proposal

Features

This course is structured around a “Top 10 List” of tips and techniques for measuring the user experience:

  1. Know your data (and what you can & can’t do with it)
  2. Show your confidence (intervals)
  3. Deal with binary task success data (appropriately)
  4. Compare means
  5. Consider using expectation measures
  6. Use the System Usability Scale (SUS)
  7. Show frequency distributions
  8. Combine different metrics
  9. Use appropriate tools
  10. Present data appropriately

Audience

Usability and user experience practitioners who want to take a more quantitative approach to understanding the user experience. Prior experience with statistical analyses is not required, but basic familiarity with usability evaluations is helpful.

Presentation

Lecture with lots of examples and discussion.

Instructors' background

Thomas S. Tullis is Senior VP of User Insight at Fidelity Investments. He was instrumental in starting the User Experience group at Fidelity and setting up their Usability Labs. His team is responsible for continuous improvement of the user experience for Fidelity’s customer-facing applications. He is also an Adjunct Professor at Bentley College where he teaches classes on accessibility and usability metrics. He has over 30 years of experience in conducting HCI studies and more than 50 publications.

William Albert is Director of User Experience at Fidelity Investments, where he manages the team responsible for the user experience of Fidelity’s workplace investing applications. Over the past decade, he has utilized nearly every type of usability metric. He has more than 20 publications, and has presented his research at many professional and academic conferences.

Tullis and Albert co-authored the new book, Measuring the User Experience: Collecting, Analyzing, and Reporting Usability Metrics, published in April 2008 by Morgan Kaufmann. As of July 2008 it has already sold more than 1,000 copies.