Ana Isabel Martins, Alexandra Queirós, Anabela G. Silva et al. · Advances in systems analysis, software engineering, and high performance computing book series · 2014
This chapter aims to identify, analyze, and classify the methodologies and methods described in the literature for the usability evaluation of systems and services based on information and communication technologies. The methodology used was a systematic review of the literature. The studies included in the analysis were classified into empirical and analytical methodologies (test, inquiry, controlled experiment, or inspection). A total of 2116 studies were included, of which 1308 were classified. In terms of results, the inquiry methodology was the most frequent in this review, followed by test, inspection, and finally, the controlled experiment methodology. A combination of methodologies is relatively common, especially the combination of test and inquiry methodologies, probably because they assess different but complementary aspects of usability contributing to a more comprehensive assessment.
Cited by 30
Terence S. Andre, Margaret A. Schurig · 2007
Abstract : The Behavioral Sciences and Leadership Department at the United States Air Force Academy (USAFA) developed a low-cost human-computer interaction (HCI) laboratory in 2004. Since that time, the lab has grown into a teaching laboratory introducing HCI concepts to cadets in the Behavioral Sciences-Human Factors option as well as cadets in the Systems Engineering-Human Systems concentration. The HCI lab exposed cadets to contemporary methods and tools used in usability evaluation. The purpose of this final report is to document two studies recently conducted in the HCI laboratory. The first study examined the use of eye tracking as an advanced technique in determining the attentional focus of an evaluator watching a recorded usability highlight video. Current usability evaluation recording technology provides the usability practitioner with the capability to record audio, video of the user, and desktop screen activity in a picture-in-picture (PIP) format, allowing the evaluator to observe the interface screen and the human user simultaneously. The research question in the first study focused on how best to present the PIP video that is often displayed along with the desktop screen capture. In the second study, instructors in the Behavioral Sciences and Leadership Department were interested in examining the changes in a student's technique of identifying usability problems while using the HCI laboratory. Practitioners in the usability field have noted that experience contributes to the quality of usability problem reports, especially when that experience includes exposure to a framework for doing usability evaluation. Thirteen students in an undergraduate HCI course participated in this study during the Fall 2006 semester.
Cited by 1
Patrick W. Jordan · 2020
Cited by 0
Dominique L. Scapin, Tomas Berns · Behaviour and Information Technology · 1997
(1997). Guest editorial Usability evaluation methods. Behaviour & Information Technology: Vol. 16, No. 4-5, pp. 185-187.
Cited by 8Open access
Calvin Kalun Or, Alan H. S. Chan · Human-Computer Interaction · 2024
Usability is an important attribute that influences a product’s success. Users are likely to stop using a product, particularly in a voluntary use context, if it is not easy to use or does not support task performance (Or, Holden, & Valdez, 2023; Or et al., 2011; Xie & Or, 2020). Given that organizations typically invest substantial resources in the development of products, users’ abandonment of products means that organizations lose their return on investment. Moreover, usability can influence how individuals interact with or perceive a design, which subsequently affects their effective behaviors and decision-making (Chan & Chan, 2013; Coldewey, Diruf, Röhrig, & Lipprandt, 2022; Jung, Labriola, Krebs, & Sawyer, 2022; Liu et al., 2016).
Cited by 0
Riihiaho, Sirpa · 2000
This thesis presents methods for usability evaluation and experiences in applying the methods. The work aims to give background knowledge for selecting evaluation methods in various situations. The thesis presents experiences in usability evaluations gathered in 72 evaluations during years 1993-1999. In these evaluations, both software and embedded systems have been studied. Usability evaluation is an essential step in human-centred design. A variety of usability evaluation methods are needed in a development process, because usability is a complex multidimensional concept that should be looked at in many ways. Different methods serve different evaluation purposes and reveal different problems. Therefore, several methods should be used as a complement to each other. In this thesis, usability evaluation methods are divided into empirical user testing and usability inspections without user involvement. The most common user testing method is a usability test in which a participant does given tasks with the evaluated system. The most common usability inspection method is a heuristic evaluation in which an inspector evaluates a user interface against a set of usability guidelines. The thesis also presents some new evaluation methods modified from usability testing and contextual inquiry. The evaluations that we have conducted at Helsinki University of Technology have shown that user testing is possible even with moderate resources. For example, usability tests with cardboard mock-ups and pluralistic usability walkthroughs with paper prototypes have given valuable results that have affected the product development. On the other hand, inspections have helped to find potential usability problems even before any prototypes have been built.
Cited by 11
Calvin Kalun Or, Alan H. S. Chan · 2024
Product usability can affect product success; therefore, conducting usability inspections to inspect product design flaws and allow them to be corrected in the early stage of a product development cycle is crucial. This chapter provides an overview of three classical usability inspection methods and their applications: heuristic evaluation, cognitive walkthrough, and pluralistic walkthrough. Several established usability design principles are also presented, allowing practitioners to understand how they can be applied for inspecting and improving product usability. Moreover, two real-life case studies that provide an in-depth analysis of the inspection methods are included. Other less common usability inspection methods, including persona-based inspection, feature inspection, action analysis, and consistency inspection, are also summarized. In sum, product usability is an important aspect of user–product interaction, which can be enhanced through usability inspections.
Cited by 12
Jennifer M. Hocko · 2008
As one of their “three measures for examining an evaluation method,” Hartson, et al. describe reliability as a measure whereby “results [are] consistent...[and]... independent of the individual performing the usability evaluation” (pp. 388-389). Muller, et al. define the following terms in their comparison of usability evaluation methods, which can be used to further qualify this notion of reliability: Raw yield—the number of unique classes of usability problems that are uncovered by an evaluation method. (Note that this is different than the total number of usability problems uncovered.) Refined yield—the proportion of severe problems that are uncovered by an evaluation method (pp. 185). Thus, if an evaluation method is to be considered reliable, it will consistently produce good results in terms of both raw and refined yield, regardless of who uses it. While reliability is a “desirable ” quality of any evaluation method (Hartson, et al. pp. 396), the research on this topic indicates that some of the most popular evaluation methods are actually not very reliable. This paper discusses this disappointing fact with
Cited by 0
Iordanis Kavathatzopoulos · 2008
Cited by 0
Benedyk, R, Edwards, H · UCL Discovery (University College London) · 2005
Cited by 0
Helen Edwards, Rachel Benedyk · 2023
This study assesses three usability evaluation methods (Active Intervention, Peer Tutoring and Cross-Age Tutoring) with children aged 6–8 years old within a school setting, using an interactive, educational multimedia product. Cross-Age Tutoring elicited significantly fewer comments than the other two methods, and “plan” comments were significantly rarer than “action” and “perception and cognition” comments. In terms of the suitability of these evaluation methods for child participants, and context of use in this particular setting, Peer Tutoring appears to have the most potential. Methods need to better reflect children’s motivations and the variety of contexts in which they use multimedia products. The implications for HCI practitioners are explored.
Cited by 0
Maristella Matera, Francesca Rizzo, Giovanni Toffetti · 2006
Cited by 148
H. Rex Hartson, Terence S. Andre, Robert C. Williges · International Journal of Human-Computer Interaction · 2001
The current variety of alternative approaches to usability evaluation methods (UEMs) designed to assess and improve usability in software systems is offset by a general lack of understanding of the capabilities and limitations of each. Practitioners need to know which methods are more effective and in what ways and for what purposes. However, UEMs cannot be evaluated and compared reliably because of the lack of standard criteria for comparison. In this article, we present a practical discussion of factors, comparison criteria, and UEM performance measures useful in studies comparing UEMs. In demonstrating the importance of developing appropriate UEM evaluation criteria, we offer operational definitions and possible measures of UEM performance. We highlight specific challenges that researchers and practitioners face in comparing UEMs and provide a point of departure for further discussion and refinement of the principles and techniques used to approach UEM evaluation and comparison.
Cited by 406
Regina Bernhaupt · IGI Global eBooks · 2010
Usability and user experience are two important factors in the development of mass-customizable personalized products. A broad range of evaluation methods is available to improve products during an user-centered development process. This chapter gives an overview on these methods and how to apply them to achieve easy-to-use, efficient and effective personalized products that are additionally fun to use. A case study on the development of a new interaction technique for interactive TV helps to understand how to set up a mix of evaluation methods to cope with some of the limitations of current usability and user experience evaluation methods. The chapter concludes with some guidelines of how to change organizations to focus on usability and user experience.
Cited by 6
Estelle De Kock, Judy Van Biljon, Marco Pretorius et al. · 2016
All in-text\treferences\tunderlined\tin\tblue\tare\tlinked\tto\tpublications\ton\tResearchGate, letting you\taccess\tand\tread\tthem\timmediately.
Cited by 0
Regina Bernhaupt, Kristijan Mihalic, Marianna Obrist · IGI Global eBooks · 2008
Evaluating mobile applications and devices is particularly challenging given the variability of users, uses, and environments involved. This chapter introduces usability evaluation methods (UEMs) for mobile applications. Over the past decades various usability evaluation methods have been developed and implemented to improve and assure easy-to-use user interfaces and systems. Since most of the so-called ‘classical’ methods have demonstrated shortcomings when used in the field of mobile applications, they were broadened, varied, and changed to meet the demands of testing usability for mobile applications. This chapter presents a selection of these ‘classical’ methods and introduces some methodological variations for testing usability in the area of mobile devices and applications. It argues for a combination of both field evaluation methods and traditional laboratory testing to cover different phases in the user-centered design and development process.
Cited by 22
Anneli Folkesson · 2011
Cited by 0
Rebecka Janols · 2012
Cited by 0
Sugandha Gupta · International Journal of Computer Trends and Technology · 2015
Cited by 20
Jie Lu · OSF Preprints (OSF Preprints) · 2021
This systematic review focuses on literature of usability evaluation methods designed and applied to evaluate educational/learning technologies.
Cited by 0Open access