Kahoot! Predicts exam scores and promotes student engagement

Abstract

Purpose

Kahoot! is a free e-learning tool that employs game-based learning which is often considered a best practice in education. The aim of the current study is to assess the effectiveness of Kahoot! in a child development course.

Design/methodology/approach

Sections of child development were randomized in terms of review format prior to exams one and two. All sections had a Kahoot! review prior to the final exam. Regression analyses were employed to determine the optimal set of predictors of exam scores. Students also completed a survey assessing their opinions of Kahoot! reviews.

Findings

Kahoot! scores accounted for 31.3% of the variability in exam 1 scores, 11.1% of the variability in exam 2 scores and 19.9% of the variability in final exam scores. Students reported that Kahoot! made class more interactive and that Kahoot! helped their learning of course concepts. These findings indicate that Kahoot! is an effective review tool. Furthermore, students reported that the use of Kahoot! was a positive experience that added to their understanding of the topics taught in the child development classroom.

Practical implications

To foster student engagement, instructors should consider incorporating game-based learning in their courses.

Originality/value

This study demonstrates that Kahoot! is an effective review tool in a sample of undergraduate students. Furthermore, this study indicates that Kahoot! promotes student engagement.

Citation

Figuccio, M.J. and Johnston, M. (2022), "Kahoot! Predicts exam scores and promotes student engagement", Journal of Research in Innovative Teaching & Learning, Vol. 15 No. 2, pp. 170-177. https://doi.org/10.1108/JRIT-07-2021-0051

Introduction

Game-based learning has long been considered a best practice in education. Among the benefits, research indicates that gamification in the classroom increases student participation, attentiveness, motivation and satisfaction (i.e. Koile and Singer, 2006; Zarzycka-Piskorz, 2016; Stachowski and Hamilton, 2019; Wang et al., 2019). Although the use of games in the classroom is not new, with advances in technology, the various digital learning platforms continue to expand and the degree of interaction and feedback to the student vary with each gaming option. In today's classrooms, game-based student response systems are more likely to incorporate graphics, audio and the use of scores to create competition (both within and between students) versus other basic student response systems that are potentially less engaging (Wang and Lieberoth, 2016). Students have shown positive responses to games that allow them to receive immediate feedback on their work as well as actively engage in lecture topics with their instructor and peers. Using technology has been shown to be an effective way to achieve this goal (Johns, 2015). Ultimately, regardless of the format, the pedagogical end-goal of any game-based learning system is to maximize student learning. As more gaming options are created, their individual effectiveness needs to be both explored and quantified.

Despite the growing use of gamification in the classroom setting, Plump and LaRosa (2017) state that “within higher education, the use of these tools is often limited due to lack of time, insufficient experience, or doubts regarding the scholarly merits of such activities” (p. 151). A main objective of the present study was to add to the body of the literature substantiating that game-based learning is an excellent classroom tool that not only enhances the student experience but solidifies student learning and retention. Previous research has shown that for gaming platforms to be the most useful, they should motivate intrinsic learning, incorporate appropriate and challenging cognitive activities, and provide feedback (Kapp, 2012). For these reasons, not all review platforms or strategies are created equal. In addition, instructors may often fall back on more traditional forms of class review as a function of experience and time constraints. Such traditional in-class review sessions often lack in their ability to create positive student motivation to engage the course materials, and these basic reviews of content potentially may not provide immediate feedback to the learner. Although game-based reviews appear to provide more student and student/teacher interaction, not all game-based technologies provide the same level of opportunity for further class discussion, teacher explanation and essential feedback. Overall, when there are time limits for classroom review, quick, easily accessible and useful, game-based platforms can become essential tools to enhance both student engagement and ultimately, achievement. Discovering which platforms lead to the best student outcomes is therefore a worthy exploration.

One of the relatively newer options for game-based learning is Kahoot!. Kahoot! is a free game-based platform in which instructors can create Kahoot! quizzes in minutes. Players then use their own devices (phones, tablets, etc.) to respond while questions are projected on a shared screen. Students receive instant feedback and are scored based on both accuracy and speed. Moreover, once all students have made their selections, the correct answer is displayed on the shared screen along with a graph illustrating how many students selected each of the possible answer choices. This illustration provides tremendous benefit for the instructor as it diagrams the areas of content where students might be struggling. In addition to these fun and useful tools, Kahoot! also displays the names of the top performers, thus creating a sense of friendly competition. In a commentary on Kahoot!, Dellos (2015) states, “Kahoot! encourages student curiosity and involvement, which provides opportunity for the educator to identify ‘gaps’ or areas of weakness in content understanding” (p. 51). Not only does the real-time feedback of Kahoot! allow instructors to potentially modify their lectures based on the students' understanding of course material but this instant information also allows students to see how many others in the class got a question wrong. When students realize that they are not the only one struggling with one learning objective, they may be more comfortable asking questions, therefore leading to a better understanding of course material for all involved (Plump and LaRosa, 2017).

Among the emerging literature surrounding the study of the use of Kahoot!, most data rely on students' self-reports. One such study done by Lee et al. (2019) indicated that the use of Kahoot! decreased student anxiety towards learning the subject in question. Although done with middle school students and not college students as in the present study, the students in Lee et al.'s (2019) study also found that Kahoot! increased their concentration in class and that they were much more willing to preview and review the course material as a result. Kahoot! naturally fosters classroom discussions and allows all students, even those who are more naturally introverted, to actively engage with the material. Despite Kahoot! launching in 2013 and having more than 1.6 billion players, research on its effectiveness is rather limited. Like the Lee et al. (2019) study, there are several reviews in the literature that overwhelmingly indicate positive student self-reports, but fewer studies examine the direct impact of Kahoot! on performance measures, as well. In addition, the effectiveness of Kahoot! versus other game-based review strategies has received little attention in the gaming literature.

Among the limited research, Stachowski and Hamilton (2019) examined various game-based review formats and their relationship to exam performance (among other outcomes). Their study with students in an introductory psychology course compared the effects of Jeopardy, Kahoot! and the immediate feedback assessment technique (IF-AT) to how these review types related to student motivation, conflict/group dynamics during the review and, ultimately, test performance. Unlike the Stachowski and Hamilton (2019) study, however, the current study compared technology-enhanced review sessions (Kahoot!) to a traditional review process (Jeopardy) to explore how students' performance in technology-enhanced review sessions differ from more frequently used traditional exam review sessions (Jeopardy). In addition, the student experience of game-based learning was assessed in the present study as research indicates that students report greater motivation and positivity to activities that allow them to have greater interaction with both fellow students and their instructor (Good and Brophy, 2000, as cited in Johns, 2015). Students who enjoy the review experience are more likely to engage with the course materials and potentially retain important information needed to perform well on exams.

The second review, and commonly used game-based review, examined in the present study was Jeopardy. Designed after the long-running television game show, Jeopardy is a review game where students give a short reply to a statement/definition presented by their instructor. Unlike Kahoot!, students receive the same number of points if they answer the question correctly. In addition, students present their answer on a sheet of paper. Like Kahoot!, Jeopardy allows students to recognize areas of weakness prior to an exam. Previous research has found that students report overall positive experiences with both Jeopardy and Kahoot! with no measurable differences in performance (Stachowski and Hamilton, 2019). It may be that overall game-based learning is inherently motivating and generally allows students to identify areas they need to review. It should be noted however, that Jeopardy and Kahoot! potentially lead to different levels of depth in class discussion and explanation. The direct feedback provided by Kahoot! may allow for instructors to immediately identify areas of content the entire class may need clarified rather than simply leaving it to the student to see their own individual areas of weakness as would be the more likely scenario with Jeopardy.

Kahoot! also appeals to students due to its highly engaging nature and some of the nuances of the game may provide greater benefit than those of playing Jeopardy. The default settings of Kahoot! include scoring and audio that keeps students motivated and excited. Wang and Lieberoth (2016) specifically looked at the benefits of the audio and points settings in Kahoot!. Their study found that the use of audio and points increased students' concentration during the lecture and that the students reported having more fun in the class. Overall, their findings suggest that removing audio feedback limited students' questions, discussion and celebration while playing Kahoot!. Removing points still allowed for open engagement, but it was considerably more muted. In addition, removing both audio and points from the game resulted in low energy and no discussion or questions in the classroom. The conclusion was that the audio in Kahoot! had the largest impact on classroom engagement, and that the most effective use of Kahoot! incorporated both points and audio (Wang and Lieberoth, 2016). This finding may suggest that other game-based platforms that lack audio feedback like Kahoot! may potentially be less effective learning tools for exam review, despite student enjoyment of the gaming process (i.e. such as Jeopardy).

Overall based on previous literature of both Kahoot! in general and other game-based learning, the aim of the current study was to assess the effectiveness of Kahoot! in an undergraduate child development course within a psychology department. Specifically, this study hypothesized that: (1) Kahoot! sessions are effective methods of review in the undergraduate psychology classroom and (2) students will report that Kahoot! is promotes student engagement.

Methods

Participants

The current study was approved by the Institutional Review Board at a four-year public institution in the Mid-Atlantic region, Farmingdale State College (FSC). A total of 190 undergraduate students enrolled in sections of Child Development (PSY 232) participated in the study. A total of 144 students identified as female, 45 students identified as male and one student identified as transgender male. The mean age of participants was M = 19.83 years, SD = 1.60. A total of 21.58% of participants identified as Hispanic. A total of 72.11% of participants identified as White, 12.63% of participants identified as Black or African American, 2.11% of participants identified as American Indian or Alaska Native, 1.58% of participants identified as Asian and 11.58% of participants did not report their race. A total of 39.47% of participants reported sophomore standing, 30.52% of participants reported freshmen standing, 19.47% of participants indicated junior standing, 7.37% of participants indicated senior standing and 2.63% of participants reported non-matriculated standing. Participants reported they were enrolled in 14 distinct majors. The most frequent majors that students reported were: 36.32% science, technology and society, 17.37% liberal arts and sciences, 11.58% nursing and 7.89% applied psychology.

Course

PSY 232 explores human development from conception through middle childhood. The course includes historical and modern concepts of attitudes toward children, theories and models of child development, research methods in the study of children, genetics, prenatal development and influence, pregnancy and birth. Moreover, within each age range, the course emphases factors influencing the physical, cognitive, social and emotional development of the child. Developmental disorders are also examined. The course was a traditional face-to-face course that met in person for two one-hour and 15-min sessions each week.

Institutional profile

FSC is a four-year public institution located in Long Island, New York. FSC is dedicated to student success and is the largest applied science and technology college in the State University of New York (SUNY) system. FSC has over 10,000 undergraduate students and offers 41 undergraduate degree programs. FSC offers a 20:1 student to faculty ratio, and the average class size consists of 25 students. FSC has a freshmen retention rate of 83%, and a six-year graduation rate of 55%.

Study design

Participants were enrolled in sections of PSY 232. All sections were taught by the same instructor to reduce variability in teaching style. Review sessions were conducted the class meeting prior to the exam. Review sessions were either Jeopardy (traditional) or Kahoot! (technology-enhanced). It is important to note that the questions in the traditional and technology-enhanced review sessions were identical; the only difference was the method of delivery. Traditional and technology-enhanced review sessions were counterbalanced for exams 1 and 2. All participants had a Kahoot! review for exam 3. For example, Section 1 had a Jeopardy review for exam 1, a Kahoot! review for exam 2 and a Kahoot! review for exam 3. In contrast, Section 2 had a Kahoot! review for exam 1, a Jeopardy review for exam 2 and a Kahoot! review for exam 3.

Data collection and analysis

An array of classroom data was collected, including attendance, demographics, Kahoot! performance and exam results. IBM Statistical Package for Social Sciences (SPSS) version 25 was employed for data analysis. Independent measures t-tests were run to compare exam scores of students who participated in Kahoot! and Jeopardy review sessions. Stepwise multiple regression analyses were conducted to determine the optimal set of predictors of exam performance. In addition, students completed a questionnaire at the end of the semester which assessed their views of Kahoot! and Jeopardy review sessions. Participants reported their responses on a five-point Likert scale. Since Likert scale ratings consist of ordinal data, nonparametric statistics were employed to examine questionnaire data (Nanna and Sawilowsky, 1998). Medians and ranges were assessed for questionnaire items.

Results

Independent measures t-tests

Independent measures t-tests were conducted to compare exam scores in Kahoot! and Jeopardy conditions. Exam scores did not significantly differ between review methods. There was not a significant difference in exam 1 scores for Kahoot! (M = 74.21, SD = 12.04) and Jeopardy (M = 72.85, SD = 12.03) conditions; t(188) = 0.75, p = 0.452. There was not a significant difference in exam 2 scores for Kahoot! (M = 73.97, SD = 12.28) and Jeopardy (M = 71.95, SD = 13.01) conditions; t(186) = 1.05, p = 0.297. There was not a significant difference in exam 3 scores for Kahoot! (M = 78.01, SD = 11.55) and Jeopardy (M = 79.27, SD = 13.67) conditions; t(184) = −0.66, p = 0.514.

Stepwise multiple regression

A stepwise multiple regression was carried out to investigate whether Kahoot! scores, attendance and sex could significantly predict participants' exam 1 scores. The results of the regression indicated that the model with Kahoot! scores as a predictor variable explained 31.3% of the variance, and that the model was a significant predictor of exam 1 performance, F(1,97) = 44.28, p < 0.001. While Kahoot! scores contributed significantly to the model (β = 0.560, p < 0.001), attendance (β = 0.124, p = 0.143) and sex (β = 0.043, p = 0.626) did not.

A stepwise multiple regression was also carried out to investigate whether Kahoot! scores, attendance and sex could significantly predict participants' exam 2 scores. The results of the regression indicated that the model with Kahoot! scores as a predictor variable explained 11.1% of the variance, and that the model was a significant predictor of exam 1 performance, F(1,83) = 10.41, p = 0.002. While Kahoot! scores contributed significantly to the model (β = 0.334, p = 0.002), attendance (β = 0.057, p = 0.595) and sex (β = 0.001, p = 0.995) did not.

A stepwise multiple regression was also carried out to investigate whether Kahoot! scores, attendance and sex could significantly predict participants' exam 3 scores. The results of the regression indicated that the model with Kahoot! scores as a predictor variable explained 19.9% of the variance ,and that the model was a significant predictor of exam 1 performance, F(1,145) = 35.94, p < 0.001. While Kahoot! scores contributed significantly to the model (β = 0.446, p < 0.001), attendance (β = 0.190, p = 0.069) and sex (β = 0.098, p = 0.368) did not.

Qualitative nonparametric statistics

Students completed a questionnaire to assess their experience with Kahoot! on a five-point Likert scale. Students reported that they enjoyed playing Kahoot! (Mdn = 5, Ra = 4). Students also indicated that Kahoot! was easy to use (Mdn = 5, Ra = 4). In addition, students reported that Kahoot! made class more interactive (Mdn = 5, Ra = 4). Students also reported that Kahoot! helped with my understanding of course concepts (Mdn = 5, Ra = 4). Students also indicated that they prefer reviewing with Kahoot! than with Jeopardy (Mdn = 5, Ra = 4). Lastly, students reported that they felt better prepared for the upcoming exam after using Kahoot! (Mdn = 5, Ra = 4) (see Table 1).

Discussion

The present study hypothesized that Kahoot! sessions would be an effective method of review in the undergraduate Psychology classroom. In addition, the present study sought to add further evidence to the body of literature surrounding game-based learning as an effective classroom tool that both solidifies learning and retention as well as adds to a positive student experience in the classroom. Results indicate that Kahoot! is indeed an effective review tool that predicts exam scores for students regardless of other factors such as gender and even attendance. As predicted, students also reported that the use of Kahoot! was a positive experience that added to their understanding of the topics at hand and their overall grasp of concepts taught in the child development classroom.

When compared to review methods such as Jeopardy, both review tools resulted in similar exam score results. It is important to note that the questions were identical in the technology-enhanced (Kahoot!) and traditional review sessions (Jeopardy). This finding is consistent with research by Stachowski and Hamilton (2019), who had also discovered no measurable differences in performance between Jeopardy and Kahoot!. It appears that the game-based nature of both review platforms provide enough feedback to help students improve their scores and engage with the course materials. Even though there was no quantitative difference in performance, students reported a preference for technology-enhanced (Kahoot!) review sessions over traditional (Jeopardy) review sessions. Kahoot! may be a better platform due to the immediacy of the feedback for the student as well as the audio Kahoot! provides when compared to other games such as Jeopardy. This idea stemmed from research by Wang and Liebroth (2016), who found that the audio in Kahoot! had the largest impact on classroom engagement. Despite showing no difference between Kahoot! and Jeopardy in the current study, we still believe the nuances of different game-based reviews deserve more attention in future research in order to further examine what factors of the game lead to the best student outcomes, enjoyment, learning and retention. It would be interesting, for example, to explore if the competitive nature of Kahoot! versus Jeopardy makes a difference in performance for students who are more competitive in general. It is possible that there are student characteristics yet to be explored that may make one review platform better for a given student over another. Future research should consider examining such personality differences as a factor that could potentially influence the impact of different types of game-based reviews and their overall usefulness on an individual level.

Finally, the present study also hypothesized that students would report that Kahoot! was a more positive experience compared to other forms of review. Despite no discernible test score differences compared to the use of the Jeopardy review, the results do indicate that students report that Kahoot! made the class more interactive and overall, they prefer reviewing with Kahoot!, and they felt they understood concepts better when using Kahoot! versus Jeopardy. It is possible that students prefer the characteristics of Kahoot! versus Jeopardy due to inherent technology differences. For instance, Kahoot! allows students to respond to questions on their mobile devices. Students also receive positive reinforcement when presented with trophies. The use of technology and reinforcement may explain why Kahoot! is reportedly more enjoyable by students. Although there may be differences in student game preferences, it may be that both games highlight to students' what concepts they need to review and focus on when studying prior to an exam, thus leading to better exam scores regardless of game type. Interestingly, Braad et al. (2020) examined game-based learning as it relates to metacognition. They sought to explore differences in game-based learning environments, and how the design of games can better inform metacognitive processes in learners. An understanding of student study techniques and metacognitive processes could also serve to better inform how game-based reviews may be more useful for some students versus others depending on both the students' own study and learning style and the gaming characteristics.

Overall, the present study revealed that Kahoot! is an effective review tool predicting test scores in a social science classroom setting. Although this game-based review resulted in gains similar to the Jeopardy review, there are demonstrated differences in the enjoyment level experienced by students as well as their beliefs about exam preparedness and understanding core concepts. Future research should continue to explore how differences in learning style and personality characteristics might influence the gains experienced by different types of gamification in the classroom on an individual level. It is worth examining how individual students may make gains in intrinsic motivation, performance and enjoyment based on game type and gaming characteristics.

Table 1

Summary of qualitative results

Item	N	Median	Mode	Range
Enjoyed	190	5	5	4
Easy to use	190	5	5	4
Interactive	190	5	5	4
Understanding	190	5	5	4
Preferred	190	5	5	4
Better prepared	190	5	5	4