Meta-Analysis of Correlations Among Usability Measures

Kasper Anders Søren Hornbæk, Law Effie Lai Chong

Research output: Chapter in Book/Report/Conference proceedingArticle in proceedingsResearchpeer-review

171 Citations (Scopus)

Abstract

Understanding the relation between usability measures seems crucial to deepen our conception of usability and to select the right measures for usability studies. We present a meta-analysis of correlations among usability measures calculated from the raw data of 73 studies. Correlations are generally low: effectiveness measures (e.g., errors) and efficiency measures (e.g., time) has a correlation of .247 ± .059 (Pearson's product-moment correlation with 95% confidence interval), efficiency and satisfaction (e.g., preference) one of .196 ± .064, and effectiveness and satisfaction one of .164 ± .062. Changes in task complexity do not influence these correlations, but use of more complex measures attenuates them. Standard questionnaires for measuring satisfaction appear more reliable than homegrown ones. Measures of users' perceptions of phenomena are generally not correlated with objective measures of the phenomena. Implications for how to measure usability are drawn and common models of usability are criticized.
Original languageEnglish
Title of host publicationConference on Human Factors in Computing Systems : CHI 2007, Reach beyond. Conference Proceedings
EditorsBo Begole, Stephen Payne, Elizabeth Churchill, Rob St. Amant, David Gilmore, Mary Beth Rosson
PublisherAssociation for Computing Machinery
Publication date2007
Pages617-626
ISBN (Print)9781595935939
DOIs
Publication statusPublished - 2007
EventConference on Human Factors in Computing Systems - San Jose, Calif., United States
Duration: 28 Apr 20073 May 2007

Conference

ConferenceConference on Human Factors in Computing Systems
Country/TerritoryUnited States
CitySan Jose, Calif.
Period28/04/200703/05/2007

Cite this