Combining experimental and inquiry methods in software usability evaluation: The paradigm of LvS educational software

Abstract

Purpose

The purpose of this paper is to present a methodology combining experimental and inquiry methods used for software usability evaluation. The software product of LeViS project funded by the European Commission (Socrates/Lingua II) is used as an evaluation paradigm. The aim of the paper is twofold: to present the results of the usability evaluation using this software as an example and to suggest a number of improvements for the next version of the software tool; and to portray the advantages of combining methods from different evaluation approaches and the experiences from their application.

Design/methodology/approach

The evaluation for this experiment combined different usability methods, both experimental and inquiry ones. More specifically, the methods employed were the Thinking Aloud Protocol and the User Logging, which were performed in a usability evaluation laboratory, as well as the inquiry methods of Interview and Focus Group.

Findings

In this study, usability problems regarding the Learning via Subtitling (LvS) educational software were revealed as well as issues regarding the use of Thinking Aloud Protocol method and involving users with a specific profile. The research findings presented in this paper constitute an innovative and effective methodology for software usability evaluation and are useful for laboratories aiming to conduct similar evaluations.

Research limitations/implications

Although this methodology has been successfully applied for over 20 software products, due to practical purposes related to this paper's extent, only one software is used as an example.

Originality/value

Through the evaluation process, apart from discovering certain usability problems related to the software, there are a number of important conclusions drawn, regarding the methods used and the methodology followed in software usability evaluation.

Citation

Kostaras, N., Stavrinoudis, D., Sokoli, S. and Xenos, M. (2010), "Combining experimental and inquiry methods in software usability evaluation: The paradigm of LvS educational software", Journal of Systems and Information Technology, Vol. 12 No. 2, pp. 120-139. https://doi.org/10.1108/13287261011042921