Ready, Set…? End-User Testing of an AR Treasure Hunt Learning Game

Samuel Taylor & Adam Stone, Kyushu Sangyo University


Formative user testing, aimed at obtaining feedback from students, instructors, and administrators at a higher education institution in Japan, was undertaken on an AR treasure hunt learning game app. It involved the collection of usage data from data-mining tools, anecdotal evidence from surveys and interviews, and descriptive evidence from observations of tester behavior to investigate aspects of app learning, engagement, usability. Analysis indicated that testers (N=21) rated learning significantly higher than both engagement (p<0.01) and usability (p<0.01), and that engagement was rated significantly higher than usability (p<0.05). Observations noted indicators of engagement, frustration, and unanticipated patterns of usage. Interviews with testers raised issues concerning usability and engagement, and provided suggested changes to the app. These were used to make improvements that, while impacting learning, also facilitated the classroom implementation of the app in pedagogy. In conclusion, it is suggested that user testing is a vital part of educational technology impact evaluations, and that key aims of the design and development of educational apps should be the easy access, enabled through high usability and engagement, of learning content situated in contexts that provide opportunities for further or applied use.


Download this article here.