Evaluating the impact of students' generative AI use in educational contexts

Abstract

Purpose

The purpose of the study was to evaluate the impact of generative artificial intelligence (GenAI) on students' learning experiences and perceptions through a master’s-level course. The study specifically focused on student engagement, comfort with GenAI and ethical considerations.

Design/methodology/approach

The study used an action research methodology employing qualitative data collection methods, including pre- and post-course surveys, reflective assignments, class discussions and a questionnaire. The AI-Ideas, Connections, Extensions (ICE) Framework, combining the ICE Model and AI paradigms, is used to assess students' cognitive engagement with GenAI.

Findings

The study revealed that incorporating GenAI in a master’s-level instructional design course increased students' comfort with GenAI and their understanding of its ethical implications. The AI-ICE Framework demonstrated most students were at the initial engagement level, with growing awareness of GenAI’s limitations and ethical issues. Course reflections highlighted themes of improved teaching strategies, personal growth and the practical challenges of integrating GenAI responsibly.

Research limitations/implications

The small sample size poses challenges to the analytical power of the findings, potentially limiting the breadth and applicability of conclusions. This constraint may affect the generalizability of the results, as the participants may not fully represent the broader population of interest. The researchers are mindful of these limitations and suggest caution in interpreting the findings, acknowledging that they may offer more exploratory insights than definitive conclusions. Future research endeavors should aim to recruit a larger cohort to validate and expand upon the initial observations, ensuring a more robust understanding.

Originality/value

The study is original in its integration of GenAI into a master's-level instructional design course, assessing both the practical and ethical implications of its use in education. By utilizing the AI-ICE Framework to evaluate students' cognitive engagement and employing action research methodology, the study provides insights into how GenAI influences learning experiences and perceptions. This approach bridges the gap between theoretical understanding and the real-world application of GenAI, offering actionable strategies for its responsible use in educational settings.

Citation

Wood, D. and Moss, S.H. (2024), "Evaluating the impact of students' generative AI use in educational contexts", Journal of Research in Innovative Teaching & Learning, Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/JRIT-06-2024-0151

Introduction

Generative Artificial Intelligence (GenAI) has emerged as a transformative technology with wide-ranging applications in various fields, including education. It involves AI systems that can produce content such as text, images, or videos, often by mimicking or generating human-like creative output (Alasadi and Baiz, 2023; Chiu, 2023; Lee et al., 2023; Pavlik, 2023). In the context of education, GenAI can be applied to tasks like creating educational content, generating personalized recommendations, or assisting in instructional design (Bolick and da Silva, 2024). However, the rapid proliferation of GenAI technologies has raised concerns about their effectiveness and ethical use within educational settings (Kostka and Toncelli, 2023; Lim et al., 2023; Pedersen, 2023; Su and Yang, 2023).

The integration of GenAI in education has the potential to enhance teaching and learning by automating tasks, personalizing instruction and expanding the accessibility of educational resources (Adiguzel et al., 2023; Chan and Hu, 2023; Ciampa et al., 2023). Neumann et al. (2023) acknowledge the importance of integrating AI tools into higher education, emphasizing their likely permanence and the consequent necessity for the development of AI skills to prepare for the future. When GenAI is effectively incorporated into the curriculum, it has the potential to transform teaching methods and enrich learning experiences. Nevertheless, this technological advancement also poses ethical, social and pedagogical challenges. There are concerns about plagiarism, bias in generated content, privacy and the need to ensure that students understand the technology's limitations and implications (Abd-Alrazaq et al., 2023; Ariyo Okaiyeto et al., 2023; Chan and Lee, 2023). As a result, the integration of GenAI into educational contexts brings about unique challenges, unlike those posed by conventional educational technologies. Mishra et al. (2023) express that GenAI tools can be seen as a new type of digital technology that requires a nuanced understanding of its integration into educational practices. Integrating GenAI into educational systems involves more than just acquiring a new skill; it likely necessitates a shift in cultural mindset and could also demand adjustments to existing educational practices (Hwang et al., 2020).

The success of GenAI integration hinges on identifying and implementing best practices that not only enhance educational outcomes but also safeguard against pitfalls such as academic dishonesty and an overreliance on AI for tasks requiring critical thinking. Thus, a nuanced understanding is essential to integrate GenAI into education, ensuring that it benefits all students and enhancing educational practices. Students' perceptions of GenAI also play a role in determining their engagement levels and the overall effectiveness of their learning experience. Chan and Hu (2023) found that in an educational setting, students' perceptions and attitudes towards a technological advancement like GenAI, including their opinions, apprehensions and interactions with the technology, can influence their readiness to use the tool. This, in turn, affects how much the tool is incorporated into the learning process. However, By understanding the impact of GenAI on student perceptions, educators can tailor their pedagogical strategies to maximize the benefits of AI, ensuring it serves as a powerful tool for educational advancement.

Hwang et al. (2020) recognize the need for research to focus not just on the efficiency of AI systems but also on how AI-assisted learning design impacts student performance and attitudes. This is echoed by Zawacki-Richter et al. (2019) in their systematic review of AI applications in higher education, noting a significant gap in the literature concerning the pedagogical and ethical implications, as well as potential risks, of deploying AI in this context. Additionally, Kim and Lee (2023) acknowledge that there is a growing demand to understand how AI can support student learning. Laato et al. (2023) advocate for future research through case studies on the real-world application of GenAI, particularly focusing on ChatGPT, within educational settings. The need for structured and responsible integration of GenAI into educational curricula is evident (Dai et al., 2023; Kostka and Toncelli, 2023; Lee et al., 2023). To address these issues, the study seeks to understand the effectiveness of a course designed to integrate GenAI within an educational context. The research will also investigate students' perceptions of the pedagogical implications of using GenAI in their learning experiences. An important objective is to identify strategies and best practices that can be employed to promote responsible and effective use of GenAI in educational contexts.

The research is centered on a university master’s-level course in Instructional Design that incorporates GenAI as an instructional tool. The research will consider the broader context of the use of GenAI technologies, aligning with ongoing efforts to develop GenAI guidelines, principles and educational resources for students. It will contribute to the growing body of knowledge surrounding GenAI practices in educational settings. To address the research goal, the following research questions were developed:

R1.

How do students perceive GenAI effects on their learning experiences?

R2.

How effective were strategies employed to ensure the responsible application of GenAI?

R3.

How does the integration of GenAI into the curriculum influence students' perceptions and attitudes?

The research employs action research as the foundational methodology as a practical and reflective means to improve learning (Crawford, 2022). This study utilizes action research due to its ability to experimentally manipulate interventions and assess their impact on intricately connected dependent variables within a setting, blending experimental and non-experimental methods to uncover relationships between variables and inform future actions (Cunningham, 2008). By engaging in a cyclical process of planning, acting, observing and reflecting, action research enables a systematic investigation of how GenAI affects student learning and engagement within a real-world context. This hands-on approach allows for the direct assessment of students' perceptions of GenAI and its impact on their learning experiences, as well as the identification of effective strategies and best practices for its responsible application.

Fundamentally, action research provides a practical and reflective framework to explore the potential of GenAI in education, ensuring its integration is both effective and aligned with learning objectives, thereby contributing to a more engaging and efficient learning environment. The credibility of the data gathered, analyzed and interpreted is ensured through methods such as corroboration and the researchers’ knowledge of the issue and context (Cunningham, 2008). Action research allows for the comprehension and scrutinizing of the intricate challenges of specific educational contexts through a structured research approach, enabling the extrapolation of insights to broader educational scenarios (Crawford, 2022).

Acknowledging the broad scope of Artificial Intelligence (AI) which encompasses a diverse range of technologies, this study narrows its focus to GenAI, with a special emphasis on technologies akin to ChatGPT. Generative AI, a branch of AI, is particularly known for its capability to create new content, ranging from text to complex interactive dialogs (Paul and Sarkar, 2023). ChatGPT-like technologies represent a significant advancement in this field, demonstrating the ability to generate coherent, contextually relevant and often insightful textual content (Su and Yang, 2023). This focus is strategic, allowing the study to delve deeply into the educational implications, challenges and opportunities presented by advanced text-generating AI systems. This narrower focus is essential for a thorough exploration of the nuanced impacts these technologies have on learning processes and pedagogical strategies within the realm of AI in education.

Defining the responsible use of GenAI is crucial in this study, as the term inherently involves judgment and varies in interpretation. Responsible use of AI is understood here as the conscientious application of AI technologies, emphasizing fairness, transparency, accountability and the well-being of individuals and society (Michel-Villarreal et al., 2023; Paul and Sarkar, 2023). Furthermore, responsible AI use involves raising awareness and understanding of AI's impact, actively working to minimize negative outcomes while maximizing positive contributions in various sectors, including education. This definition sets the foundation for evaluating GenAI's integration and application in educational contexts.

Course design

The course is part of a master’s program in educational technology and instructional design. The course provides students with an exploration of critical aspects of instructional design, focusing on strategic planning and effective technology integration to enhance learning experiences. The course was recently revised to integrate GenAI as an instructional tool, offering students a unique opportunity to explore its applications while fostering critical reflection on use. The course structure incorporates hands-on experiential learning, enabling students to directly engage with GenAI tools for planning and developing educational materials. This direct involvement is intended to help students grasp both these technologies' practical uses and boundaries. Students were informed about the integration of GenAI tools into their coursework. This included providing explanations of the specific tools being used, their intended purposes and the expected outcomes. Transparency in this regard supported students' understanding of the relevance and utility of GenAI in their learning journey, reducing apprehension and promoting informed use.

The design includes project-oriented tasks, prompting students to conceptualize, devise and justify learning experiences. These authentic contexts nurture student creativity and experimentation. An integral part of the learning process is reflection, wherein students critically evaluate their creations and the influence of GenAI on both learning and instructional design. Through this experiential approach, students gain proficiency in integrating GenAI into instructional design, while simultaneously enhancing their critical thinking abilities and deepening their insight into the relationship between technology and education.

Kim (2024) recognizes that integrating GenAI as a collaborative tool in education and the workplace can significantly enhance human capabilities, leading to superior outcomes in data and knowledge creation. Thus, incorporating GenAI into the instructional design course equips students with skills in both GenAI technology and instructional design processes, preparing them for emerging trends and job requirements in the education sector. By actively encouraging the incorporation of GenAI programs, the course aims to reflect the current landscape of technology-enhanced education. AI-driven content generation is rapidly becoming an integral part of instructional design, and education professionals should not only be aware of this trend but also be adept at utilizing these tools effectively (Bolick and da Silva, 2024). In parallel, the course invites students to engage in thoughtful reflection about the responsible use of GenAI in instructional design by exploring questions surrounding bias, privacy and intellectual property rights, fostering a deeper understanding of the potential pitfalls related to AI implementation in education.

Methodology

The primary investigator of the study also served dual roles as both the course author and the facilitator of the course under examination. This positioning allowed for a firsthand perspective on the educational content, instructional methodologies and student engagement strategies employed throughout the course. By designing the curriculum and leading the instructional sessions, the researcher was able to closely monitor the learning outcomes, adjust teaching approaches in real-time and directly observe the interactions and feedback from students. Furthermore, this dual role facilitated a deeper analysis of the course structure and content, allowing for a more informed evaluation of its strengths and areas for improvement.

The study focused on a qualitative research approach to understand the experiences and perceptions of 17 graduate students enrolled in a Master of Science in Education course, emphasizing instructional design. Of the 17 students, 16 were experienced educators. The majority (12 students) were employed as K-12 educators, while two were involved in higher education, two were focused on corporate settings, and one had neither teaching nor instructional design experience. The diversity across the different sectors is appreciated, despite the small sample size, as the use of GenAI is perceived differently among the instructional design sectors. This aspect enriches the study by introducing a multifaceted view of GenAI's impact and acceptance. The diversity not only represents the broad spectrum of attitudes towards GenAI but also adds depth and complexity to the investigation, underscoring the importance of considering sector-specific contexts when evaluating GenAI applications.

The primary aim was to understand changes in students' attitudes and perceptions towards GenAI in education. Data collection was primarily qualitative, including reflective assignments that captured the students' experiences with GenAI applications in their assignments and participation in class discussions, pre and post-course surveys and a questionnaire. Through thematic analysis, the qualitative data were examined to identify emerging themes and patterns, providing an account of how GenAI integration influences educational experiences. This qualitative-centric approach allowed for an interpretation of the impact of GenAI on students, highlighting the complexities and subtleties of its integration into educational settings.

A combined framework, synthesizing the ICE Model and the three AI paradigms in education, was developed and utilized. The AI paradigms provide a structure to categorize the relationship between the learner and AI. The ICE Model, an acronym that stands for ideas, connections and extensions, provides a foundational structure for understanding cognitive processes in learning. When integrated with the three paradigms, this model offers a perspective on how students interact with and utilize AI. This synthesis is referred to as the AI-ICE Framework. Rather than focusing solely on the outputs generated by AI, this perspective emphasizes the cognitive processes and effort students engage in while interacting with AI systems. The cognitive-centered perspective suggests that the true value does not solely lie in the products AI can create but in the process of students learning to use these tools to extend their cognitive horizons, solve complex problems and achieve goals that were previously unreachable.

Imagine a tool that individuals can utilize for a variety of tasks, such as generating art, solving mathematical equations, or composing narratives. Rather than solely marveling at the capabilities of this tool, the focus shifts to the cognitive processes and learning experiences of students as they interact with the tool. This perspective emphasizes the importance of the cognitive engagement that occurs when students collaborate with AI, such as the improvement in learners’ questioning skills or the innovative problem-solving strategies they develop during this collaboration. Thus, when students work alongside AI, the emphasis isn't just on the product of their collaboration. It also encompasses the ways in which this interaction stimulates deeper thinking, fosters creativity and unveils solutions that were previously inconceivable.

The AI-ICE framework developed acts as a lens through which the student-AI collaboration can be characterized, focusing on the cognitive effort students perform rather than just the end products of such collaborations. The framework is grounded in the idea by Bansal et al. (2021) that AI systems augment but humans remain the decision-makers. The achievement of ideal human-AI collaboration occurs when the human-AI team outperforms both AI or human efforts alone. At the core of this framework is the recognition that when students engage with AI, they should not be passively receiving information or outsourcing tasks. Instead, they're actively involved in a dynamic process that challenges and stretches their cognitive capabilities. The framework does not include a category for the passive use of AI, as this would negate the goal of augmenting student learning through cognitive effort. Further, the framework has the potential to contribute concerning Steyvers and Kumar’s (2023) observation of a need to better understand the factors contributing to human-AI collaboration.

The ICE Model provides for a conceptual understanding and application of knowledge (Fostaty et al., 2000). The model consists of three levels, ideas, connections and extensions as shown in Table 1. At the Ideas stage, students focus on understanding basic concepts, vocabulary and fundamental facts within a given context, essentially gathering pieces of information from their learning environment. In the Connections phase, students analyze this information to find patterns and relationships, linking new knowledge with what is already known. At the Extensions level, the emphasis shifts to creatively applying new knowledge in innovative and meaningful ways, involving internal reflection and the extrapolation of this learning to new contexts. The approach, in essence, delineates a learner's progression from being a beginner, advancing through a stage of competence and ultimately achieving expertise.

Ouyang and Jiao (2021) describe three paradigms of AI in education that represent different roles and relationships between learners and AI technologies as shown in Table 2. The AI-Directed paradigm positions AI as the main guide in the learning process, with learners primarily receiving information. The AI-Supported paradigm sees AI as a tool to enhance learning, promoting a collaborative relationship between learners and AI. Lastly, the AI-Empowered paradigm emphasizes the learner’s leadership, using AI as a tool to foster independent and creative thinking.

By combining the three AI paradigms with the ICE model (Table 3), a framework is created to better understand students’ usage of AI. The ICE Model integrates a conceptual understanding of cognitive processes with the three paradigms of AI in education, offering a structured framework to assess students' AI usage. This synthesis offers a framework to assess and enhance AI literacy, from foundational understanding to advanced application. The combination of these models encourages a holistic view of AI usage, emphasizing not just technical skills but also critical thinking, creativity and responsibility in the evolving landscape of AI technologies. The framework provides a starting point for assessing whether course activities have successfully fostered higher-order thinking skills among students and effectively mitigated any tendency towards an over-reliance.

Results

Packback is an AI-supported discussion platform used within the course. It operates on an inquiry-based approach, where students initiate discussions by posing questions derived from a prompt. The platform's AI evaluates the quality of these posts and assigns a curiosity score. Aligned with the experiential learning method of the course, the traditional discussions through Packback were designed as GenAI-driven educational labs. The educational labs (Figure 1) consisted of a prompt for experimentation using GenAI in an instructional design context. The goal was to transform present perspective into practical experience, enhancing students' understanding through experimentation and reflection.

Curiosity ratings for the course aligned closely with the average ratings of other courses offered in the same timeframe, suggesting that its design was equally effective in eliciting high-quality responses. The number of posts and post types is also consistent with university averages (Figure 2). Students' reflections highlighted that using Packback significantly enhanced their comprehension and application of GenAI.

Implementing the AI-ICE Framework in GenAI-related activities showed that most students fell into the AI-Directed/Ideas category, with a few exceptions landing in the AI-Supported/Connections tier. This outcome aligns with expectations, given the significant proportion of students with minimal or no GenAI experience. However, the application of the framework highlighted an increasing awareness among students about GenAI's limitations and ethical implications in educational contexts. This deeper understanding was facilitated by the reflective component integrated into each GenAI activity, which was essential for categorizing experiences within the framework. Without these reflections, applying the framework would pose a significant challenge.

Pre-post surveys

Surveys were conducted before and after the course. These surveys aimed to measure the participants' comfort level with GenAI technologies and their understanding of the ethical considerations inherent in using such technologies. Utilizing a one (1) to five (5) scale, where one (1) represents a low level and five (5) represents a high level, the surveys sought to capture the shifts in perception among students.

There is a significant shift towards the higher end of the comfort scale in the post-course survey (Figure 3). Before the course, 37.5% of respondents rated their comfort level at four (4) or five (5). After the course, this figure jumped to 100%, with a striking 60% feeling highly comfortable (level 5). Before the course, 50% of the respondents reported lower comfort levels (1 and 2). Post-course, there are no respondents at these levels, indicating that the course effectively addressed and mitigated initial discomforts or concerns related to GenAI. This data suggests that the course was effective in increasing comfort levels with GenAI among students. The shift from a broad distribution of comfort levels, including significant portions feeling low comfort, to a concentration at the highest levels of comfort demonstrates that the course content and delivery were successful in demystifying GenAI and enhancing students' confidence in usage.

Like comfort level, there is a notable disappearance of respondents who had a low understanding of ethical considerations (levels 1 and 2) in the post-course survey (Figure 4). There is an increase in the highest level of understanding (5), from 12.5% pre-course to 60% post-course. This indicates that the course successfully raised awareness about the ethical considerations of GenAI. The course improved both the comfort level with and understanding of GenAI and its ethical considerations. The transition from lower to higher scores in both surveys indicates a successful integration, with particularly strong outcomes in fostering an understanding of ethical considerations, a critical component of working with GenAI.

Course reflection themes

The process of developing themes from the course reflections entailed a qualitative analysis beginning with a thorough reading for familiarization, followed by initial identification of key concepts. These concepts were grouped into broader categories reflecting major themes through an iterative process of review and refinement (Figure 5). Five major themes emerged: enhanced teaching strategies; personal and professional growth; challenges with GenAI integration; real-world application and future intentions; reflection and continuous learning. Not all themes directly pertain to the integration of GenAI, yet they reflect the attainment of course learning objectives. Making distinct separations between GenAI applications and instructional design objectives sometimes proved challenging. Therefore, all themes are included, encompassing both GenAI and broader achievement of course learning objectives.

Students reported gaining new strategies and skills for applying instructional design and utilizing GenAI to facilitate learning. One student remarked, “I was surprised at just how much help AI could offer to applications within education.” This sentiment was echoed by another who shared their future intentions: “I plan on using AI to help create resources, learning plans and support student review of the material.” Many appreciated the practical application of these strategies in real-world settings, indicating an improvement in their instructional capabilities. Reflecting on their experiences, students noted significant personal and professional growth, including better time management, improved critical thinking and a deeper understanding of instructional design principles. The primary researcher attributed this growth to the hands-on learning approach of the course and the integration of GenAI tools. Students discussed initial challenges in adapting to GenAI tools, including overcoming skepticism and learning to use these tools effectively in instructional contexts. Many students expressed a strong intention to apply the skills and knowledge gained from the course in their current roles and future careers. This includes using instructional design methodologies and performance gap analysis in teaching, integrating GenAI into their workflow and pursuing further education in instructional design and technology. The course reflections highlight a strong emphasis on continuous learning. Students recognized the importance of assessing and adapting their teaching methods based on student needs and performance data. Many expressed a desire to continue exploring and learning about instructional design and GenAI applications in education.

Discussion

The integration of GenAI, such as ChatGPT into educational settings has yielded insightful outcomes, shedding light on the nuanced effects on teaching and learning dynamics. The use of Packback, an AI-supported discussion platform, further aligned with the experiential learning method adopted by the course. By enabling students to engage in GenAI-driven labs through inquiry-based discussions, Packback played a crucial role in improving students' GenAI understanding and application. Packback employs AI detection tools aimed at reducing concerns over academic integrity and cheating. The arms race between GenAI usage and AI detection tools created a challenge in using Packback as an education GenAI lab (Nikolic et al., 2023). The educational labs were tailored to avoid activating AI detection mechanisms, as the activities called for the application of GenAI rather than its restriction. The design of the educational labs may have played an unintentional role in enhancing students' comprehension of how to use GenAI academically. By navigating these constraints, the educational labs may have implicitly taught students the importance of using GenAI responsibly within academic settings. Despite the necessity of using GenAI, merely copying and pasting text would activate the AI detection tool, preventing such actions from improving the curiosity score. Thus, students were compelled to formulate their own questions and reflections to meet the curiosity score criteria, rather than relying on GenAI for a shortcut. This approach likely fostered a deeper understanding among students of the ethical considerations and potential limitations when integrating GenAI, highlighting the balance between leveraging advanced technology and maintaining academic integrity.

Surveys conducted before and after a course revealed a pronounced shift towards higher comfort levels and understanding of ethical considerations. Notably, the post-course data demonstrated a leap from 37.5% to 100% of respondents feeling comfortable with GenAI technologies, with a significant portion achieving the highest comfort level. Similarly, understanding of GenAI's ethical implications saw a considerable rise, underscoring the course's effectiveness in enhancing both comfort with and comprehension of these technologies. Several students expressed reservations about integrating generative GenAI into educational settings. Given that these students were primarily experienced educators, their skepticism was not unexpected. Factors such as plagiarism concerns, limited familiarity with GenAI technology and the prevalent negative media of GenAI in educational discussions may have influenced their cautious position. However, their growing ease with using GenAI can probably be accredited to the scaffolded and authentic application of GenAI tasks that emphasized reflection. This approach allowed students to engage with the tasks and reflection in a fully transparent manner, clearly understanding the role, application and limitations of GenAI. Reflecting on their experiences, one student highlighted, “My experiences were positive due to the assignments showing me how to write prompts effectively and use AI efficiently and ethically.” This testimonial underscores the value of carefully designed tasks in demystifying GenAI's potential and promoting its ethical use in educational contexts.

The AI-ICE Framework application primarily placed students at the AI-directed/Ideas level. The utilization of the framework underscored a growing recognition among students regarding the constraints and ethical considerations of GenAI in educational settings. This enhanced comprehension was enabled by the reflective elements embedded in each GenAI task, crucial for classifying experiences within the framework. Absent these reflective moments, leveraging the framework would be considerably difficult. Thus, the framework designed to better understand student usage of GenAI holds potential beyond its intended purpose. Its value might be realized in how it could be used to empower students' future endeavors, particularly in self-evaluation and analysis of interactions with GenAI. By incorporating the AI-ICE Framework into the learning process, students could gain critical insights into their own decision-making and problem-solving strategies when working with GenAI technologies. This self-evaluation would encourage students to reflect on their approach to utilizing GenAI, assess their reliance on such technologies and understand the balance between machine-generated assistance and their original thinking. It would foster a sense of accountability and promote the development of critical thinking skills, as students would learn to discern when and how GenAI can be most effectively and ethically employed to enhance their work. Moreover, the framework could facilitate a deeper analysis of student interactions with GenAI, offering a methodology to critically examine the outcomes of these interactions. Through this analysis, students could explore the nuances of how GenAI influences their work's content, creativity and integrity. They would become adept at identifying biases, errors and/or limitations within the AI-generated content, thereby gaining a richer understanding of the technology's capabilities and constraints.

The qualitative analysis of course reflections unveiled themes such as enhanced teaching strategies, personal and professional growth, as well as the challenges and opportunities of GenAI integration. These themes not only reflect the attainment of the course's learning objectives but also highlight the broader impact of GenAI on educational practice and instructional design. This study's results resonate with current scholarly work, including that of Mai and Hanh (2024), which highlights how GenAI tools, such as ChatGPT, can markedly boost student engagement and foster in-depth subject exploration. Additionally, Tiwari et al. (2023) support the advantages of incorporating AI into educational settings, noting its effectiveness in improving the quality of learning and making academic activities more efficient. The integration of GenAI into educational contexts emerges as a promising avenue for enriching learning experiences, stimulating ethical discourse and promoting a more engaged and reflective educational approach. While the results are encouraging, they also highlight the importance of balancing GenAI's capabilities with the integrity of educational assessments and the indispensable role of human instruction. This equilibrium is vital for educators and policymakers to consider as they navigate the integration of advanced technologies into educational frameworks, aiming to leverage their potential while preserving the core values and objectives of education.

Recommendations for integration of GenAI

The recommendations proposed for integrating GenAI into educational settings largely reflect well-established pedagogical practices, underscoring the importance of traditional educational strategies even in the face of advancing technology. These strategies include providing support mechanisms to assist students in their learning, implementing scaffolding to gradually increase the complexity of tasks as students develop their skills, ensuring the authenticity of learning experiences to enhance engagement and relevance, promoting experiential learning to enable students to apply knowledge in practical contexts and fostering collaborative learning environments where students can learn from one another. However, what sets these recommendations apart when applied to GenAI is the emphasis on ethical considerations and the responsible use of technology. This entails educating students not only on how to use GenAI tools effectively but also on understanding the broader implications of their use, including issues of bias, privacy and the potential for misuse.

The following recommendations aim to cultivate technologically adept individuals who are equally committed to using GenAI in ways that are beneficial and ethical.

1. Start with clear learning objectives for GenAI tool integration, ensuring that both instructors and students understand the purpose, potential benefits and boundaries.

2. Gradual integration helps in adapting to the new technology and mitigates resistance. Support mechanisms are crucial for overcoming initial hurdles and encouraging experimentation.

3. Appreciate the irreplaceable role of human instruction in enhancing students' critical thinking abilities, fostering creativity and providing essential context and perspective.

4. Provide students with opportunities to use GenAI tools in real-world scenarios. The hands-on experience reinforces learning and illustrates the practical value of GenAI in instructional design.

5. Encouraging students to provide feedback on their experiences with GenAI is a vital aspect of transparency. Open dialogs about these experiences can enhance the learning process, enabling continuous improvement and adaptation of GenAI integration strategies.

6. Facilitate collaborative discussions that allow students to learn from each other's experiences with GenAI tools. Peer learning can enhance understanding and application of GenAI in diverse instructional contexts.

7. Incorporate discussions and/or activities on the ethical use of GenAI in education and encourage critical thinking about the implications of GenAI technology. Students should understand the responsibilities that come with using GenAI in instructional settings.

Limitations and future study

The small sample size poses challenges to the analytical power of the findings, potentially limiting the breadth and applicability of conclusions. This constraint may affect the generalizability of the results, as the participants may not fully represent the broader population of interest. The researchers are mindful of these limitations and suggest caution in interpreting the findings, acknowledging that they may offer more exploratory insights rather than definitive conclusions. Future research endeavors should aim to recruit a larger cohort to validate and expand upon the initial observations, ensuring a more robust understanding.

Figures

Figure 1

Example GenAI educational lab

Figure 2

Number of posts by post type

Figure 3

Comparison of pre-course comfort level with GenAI and post-course comfort level

Figure 4

Comparison of self-reported ethical considerations understanding of GenAI usage

Figure 5

Course reflection themes

Table 1

ICE model

Level	Description
Ideas	Focus on basic concepts, vocabulary and fundamental facts within context. Gathering information from the learning environment
Connections	Analyzing gathered information (details, facts, definitions, concepts) to find patterns and relationships, linking new knowledge with existing knowledge
Extensions	Creative application of new knowledge in innovative ways. Involves internal reflection and extrapolating learning to new contexts

Source(s): Table created by authors adapted from Diao (2021). Copyright 2021 by Elsevier

Table 2

Three paradigms

Level	Description
AI-Directed	This paradigm involves AI directing the learning process, where the learner passively receives knowledge from AI-driven systems
AI-Supported	In this approach, AI supports and enhances the learning experience. The learner collaborates with AI, using it as a tool to aid in their education
AI-Empowered	This paradigm sees the learner taking a leading role, with AI serving as an empowering tool. Here, the focus is on using AI to foster independence and innovation in learners

Source(s): Table created by authors adapted from Ouyang and Jiao (2021). CC BY-NC-ND License, Copyright 2021 by F. Ouyang and P. Jiao

Table 3

Levels of student AI usage in education: A comparison across paradigms

Paradigm	Level	Characteristics
AI-Directed	Ideas	Introduction to basic AI tools and functionalities; Focus on understanding AI in learning processes; Basic competency in using and understanding AI outputs; Emphasis on responsible AI use, awareness of limitations and biases
AI-Supported	Connections	Active use of AI tools in collaborative settings; Integration of AI insights into learning strategies; Advanced competence in manipulating AI tools for educational outcomes; Critical evaluation of AI's ethical implications in education
AI-Empowered	Extensions	Leadership in applying AI creatively and comprehensively; Effective use and adaptation of AI tools, including development of new applications; Deep engagement with ethical considerations of AI, including bias, privacy and equitable access

Source(s): Table by authors