Measuring User Experience

Online Usability Testing Tools

by Tom Tullis
Posted on February 15, 2009
Last updated on December 30, 2009

Online usability testing is not that different from traditional lab testing: both involve representative users doing realistic tasks using a prototype or live site. The difference is that with the traditional test you're directly observing each participant, while with an online test you're indirectly observing via automated data-collection methods (task timing, clickstream recording, etc). Automated data collection allows you to collect data from far more participants than you ever could in a lab setting. Check out our forthcoming book, Beyond the Usability Lab: Conducting Large-scale Online User Experience Studies.

We routinely do online usability studies where we get data from over 1,000 participants in just two days. Why would you want data from these large numbers of participants? Mainly because it allows you to easily make comparisons between alternative designs. We just finished an online study comparing two subtly different designs for an online account-opening wizard. Participants were randomly assigned to one design or the other. With over 400 participants per condition, we were able to reliably detect that one design resulted in a 2.5% higher completion rate than the other. Even just a 2.5% improvement in the completion rate can have a major impact on the bottom line.

The following tools support some type of online usability testing:

Keynote's Webeffective


Loop 11

Comments? Contact

Measuring UX Homepage