Feedback digitalization preferences in online and hybrid classroom: Experiences from lockdown and implications for post-pandemic education

Abstract

Purpose

This research aims to explore digital feedback needs/preferences in online education during lockdown and the implications for post-pandemic education.

Design/methodology/approach

An empirical study approach was used to explore feedback needs and experiences from educational institutions in the Netherlands and Germany (N = 247) using a survey method.

Findings

The results showed that instruments supporting features for effortless interactivity are among the highly preferred options for giving/receiving feedback in online/hybrid classrooms, which are in addition also opted for post-pandemic education. The analysis also showed that, when communicating feedback digitally, more inclusive formats are preferred, e.g. informing learners about how they perform compared to peers. The increased need for comparative performance-oriented feedback, however, may affect students' goal orientations. In general, the results of this study suggest that while interactivity features of online instruments are key to ensuring social presence when using digital forms of feedback, balancing online with offline approaches should be recommended.

Originality/value

This research contributes to the gap in the scientific literature on feedback digitalization. Most of the existing research are in the domain of automated feedback generated by various learning environments, while literature on digital feedback in online classrooms, e.g. empirical studies on preferences for typology, formats and communication channels for digital feedback, to the best of the authors’ knowledge is largely lacking. The findings and recommendations of this study extend their relevance to post-pandemic education for which hybrid classroom is opted among the highly preferred formats by survey respondents.

Citation

Sedrakyan, G., Borsci, S., Abdi, A., van den Berg, S.M., Veldkamp, B.P. and van Hillegersberg, J. (2023), "Feedback digitalization preferences in online and hybrid classroom: Experiences from lockdown and implications for post-pandemic education", Journal of Research in Innovative Teaching & Learning, Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/JRIT-02-2023-0014

1. Introduction

The pandemic made the largest disruption of education systems in human history (Pokhrel and Chhetri, 2021). Most educational institutions transferred to online education using virtual learning environments and instruments. Due to the situation, digital technologies received attention to provide continuity in education. Both teachers and students were challenged to properly give, receive and seek feedback. Providing timely personalized guidance to students was a major challenge long before COVID-19, but the pivot to online teaching and assessment during lockdown has underlined just how difficult this vital but time-consuming task could become (Carless, 2020). Despite the worldwide importance of digital transitions in educational processes that will likely continue to shape the new norms for online or hybrid classrooms of the future, research on online feedback to guide feedback digitalization processes is scarce (Jensen et al., 2021). To the best of our knowledge, no studies can be found that in addition outline recommended feedback formats and adaptation needs toward post-pandemic education. While it is evident that digitalization will be pivotal for accomplishing a transition to post-pandemic educational environments, where hybrid classroom/campus uniting the physical and digital learning experiences will most likely define the new norms, the field lacks insights to guide informed decisions in the domain of feedback digitalization. Despite transition processes worldwide, still, questions such as “What is the type of digital feedback that worked best during the lockdown?”, “Which new formats used by teachers proved effective among students?”, “Are there preferences in these new formats/elements of feedback to continue even when the lockdown disappears?” remain unanswered.

In this research, we, therefore, explore the following questions:

RQ1.

What were students' needs for digital feedback in online education during lockdown?

RQ1a.

What aspects/types of feedback were most needed/preferred in the context of digital feedback?

RQ1b.

What type of digital feedback elements were likely to promote increased inclusiveness thus decreasing feelings of isolation?

RQ2.

Which digital instruments/formats/features proved effective in communicating feedback in the context of online education?

RQ3.

Do personal characteristics, such as field and level of study, role and gender affect the perceptions and needs related to feedback when feedback is communicated digitally?

RQ4.

Are there potential long-term effects with regard to digital formats/channels of feedback that are likely to continue in post-pandemic education measured by preferred choices?

RQ4a.

What education and feedback formats/elements/instruments used in lockdown education are preferred to stay?

Summing up, while focusing on the teaching-learning process in online learning during lockdown, our study differs from similar studies in terms of research questions that target specifically feedback aspects in the context of digitalization experiences during lockdown, as well as studying elements that are likely to transfer into the so-called new norms after the lockdown experiences. More specifically, our study aimed to explore how different types of feedback were perceived in online/hybrid classrooms by learners during the lockdown, what were the experiences about digital forms of feedback and channels used to communicate them and what the preferences were for specific formats/channels to continue after lockdown. Therefore, in this paper, we use online feedback and digital/digitalized feedback interchangeably. In the context of this research, both terms refer to educational feedback in general that is communicated through digital channels. The automated forms of feedback have not been studied in the scope of this work. The scope of feedback typology is further detailed in Section 3.1.1.

2. Providing feedback during the pandemic: a review

Research on online feedback to guide feedback digitalization is scarce (Jensen et al., 2021). Before COVID-19 pandemic, educational practitioners were actively discussing about the importance and the complexity of providing educational feedback to learners (Kluger and DeNisi, 1996; Pardo, 2018; Shute, 2008). Certainly, feedback can be provided by multiple channels in different modalities, for a variety of purposes, as well as with a variable level of personalization or digitalization (Fidan and Gencel, 2022; Pardo et al., 2019). This variability might affect the benefit of the feedback on learners making certain types and modalities of feedback more effective than others under certain conditions (Jensen et al., 2021; Polat et al., 2022). Nevertheless, none of the existing works in this domain considered the characteristics of online feedback in the context of the massive switch to online education as observed during lockdown learning processes. A few recent studies that focused on online feedback during this timeframe explored dimensions from a very specific point of view that do not contribute significantly to generalizability. Studies on experiences from the use of digital feedback focus on specific types of feedback, such as the effects of peer feedback in online learning (Zong et al., 2021), feedback aiming self-regulation in online learning (Theobald and Bellhäuser, 2022) or online feedback in a specific learning case/context, such as English (Yang et al., 2021), Chinese language education (Zhan et al., 2022) or automated feedback in online learning (Cavalcanti et al., 2021) for various learning tasks (Ruiz et al., 2015; Sedrakyan and Snoeck, 2015). In addition, not many studies can be found that report on empirical findings on feedback digitalization experiences with broader/heterogeneous populations from different study directions/levels in the context of different courses and learning cases.

Related to this research, are also publications that outline the characteristics of online education during lockdown, as these effects also largely contribute to perceptions on feedback digitalization needs. For instance, the overwhelming increase in students' autonomy and decrease in social relatedness, and the lack of interactive learning activities were reported as contributing factors to students' feelings of loneliness during lockdown (Eberle and Hobrecht, 2021), while the lack of constructive feedback often led to feelings of isolation (Hwang et al., 2020), in addition decreasing the feelings of belonging to the community. These feelings, according to studies, both can lead to physiological stress and behavioral changes, which have negative effects on learning processes (Hwang et al., 2020). These findings are also suggestive of rethinking feedback formats that would have more focus on emotional well-being when using digital forms of feedback. The study by Means and Neisler (2020) explored the challenges that students experienced during the lockdown education, among them listing (1) lack of opportunities to collaborate with other students, (2) keeping students interested in the course content, (3) staying motivated to do well in the course after it went online, (4) missing the presence of the instructor and peers to get immediate feedback and (5) poor Internet connectivity, limited access to appropriate devices and finding a quiet place to work. In addition, the lack of standards and guidelines (Dhawan, 2020) was indicated as an important problem that challenged the instructors in the process of reformatting teaching and feedback digitalization content and instrumentation wise.

The study by Dhawan (2020) reported several opportunities for online education, e.g. by allowing teachers to design various flexible course content formats to support better understanding, including audio, video and text which also create a learning environment where students can give or receive their immediate feedback. Interestingly, Iglesias-Pradas et al. (2021) reported that the overall academic performance of students in an online class (remote teaching) was found to be significantly higher than in traditional face-to-face instruction which is in line with the results found by Gonzalez et al. (2020). Eberle and Hobrecht (2021) reported that while students experienced ceased commuting as a positive effect, students' skills to maintain social contacts for interactive learning activities emerged to be a crucial aspect, as many students were not able to cope appropriately. The above-mentioned challenges/opportunities also affected perceptions and choices of design strategies for feedback digitalization.

Close to this research is the study on the effects of feedback channels on perceptions of learners (Espasa et al., 2022), e.g. learners preferred video over audio or written channels because of its ability to support greater interactivity between lecturer and learners. Yet the study uses a population from one particular study direction (psychology students) and does not consider effects from compound variables, e.g. personal characteristics, and the variety of instrument channels, e.g. email, video-conferencing, educational platform.

In addition, to the best of our knowledge, no studies can be found that outline recommended feedback formats and adaptation needs toward post-pandemics education.

3. Method

Several techniques have been used to conduct the different steps in this research. In this research, we made use of qualitative and quantitative methods including interviews and surveys.

3.1 Research model and procedure

In this research, we investigated the existing literature aiming to identify major feedback typologies/formats and elements to design our initial survey. Next, we used a participatory co-design method for refining the survey questionnaire using informal interviews with a relatively smaller target group of students and teachers. These interviews served as a preliminary co-creation step toward a broader survey, by inquiring about students' and teachers' perceptions of the degree of effects of the lockdown and consequent online education specifically on the need for educational feedback to reveal preliminary answers as to what feedback typology found in the literature were preferred most during lockdown for drafting the preliminary questionnaire for a larger study. The participants were asked about their general feelings with regard to the feedback they receive during the lockdown education, e.g. if the amount, format and detail of feedback are sufficient/optimal, if there are particular needs which they think are not addressed or would like to receive a (targeted) feedback for, if there are any types, formats, elements of feedback that they have preferences for based on own or peers' experiences, etc. Several questions from the interviews are included below:

1. Do you think the amount of feedback you receive through online channels during lockdown is sufficient?

2. Do you lack any specific type(s) of feedback during lockdown ?

3. Is there any learning dimension/activity that you need more feedback for in an online/hybrid classroom compared to traditional classroom experiences you had before the lockdown?

Deductive qualitative analysis method has been used to classify the answers into main problem areas for further focus. The findings were also compared with the initial findings from the literature on the topic which was very limited at that time. The major concerns found from our interviews and literature were further generalized into a questionnaire design.

The resulting draft of the questionnaire was used to inquiry about general experiences from lockdown education, e.g. how learning online affected the feedback needs of learners (positive, negative, neutral), types of feedback, aspects of learning the students needed more feedback during lockdown when feedback can be delivered mostly through online environments and channels, what features/formats of the instruments proved useful for online classrooms, etc.

The final survey comprised 27 questions and sub-questions. The first seven questions aimed to collect information on participants' demographics such as age, gender, field, level of study (e.g. high school, bachelor, master, postgraduate, Ph.D.), role (student, teacher, researcher) and country. The remaining questions aimed to reveal perceptions among the respondents:

1. If it was, in general, easier or more difficult to seek/receive feedback through online environments during lockdown?

2. What aspects of learning learners needed more feedback when feedback was communicated through digital channels (e.g. theory, assignments, feedback to task outcome, feedback during a learning process, feedback for group work, etc.)?

3. What were the needs regarding the types of feedback (e.g. teacher/peer feedback, cognitive, behavioral, engagement, motivation, social-emotional, performance/mastery related, etc.) in an online classroom during lockdown?

4. What were the perceptions on the channels for communicating feedback (e.g. mail, messenger, educational platform, video-conference tool, etc.)?

5. What were the most preferred features/elements of digital feedback for learners (e.g. possibility for audio/video recording, chat rooms, ease of use, etc.)?

6. What feedback formats learners preferred (e.g. online, written or offline, combination balance)?

7. What were the most positive/negative experiences concerning feedback during the lockdown?

8. What feedback types/formats/elements or instruments that were used to specifically adapt to the needs of lockdown education are likely (or preferred) to stay after the end of the pandemic?

3.1.1 Summary of feedback dimensions used in the survey

As meaningful, effective and appropriate feedback requires various attributes/characteristics to be considered when formulating interview and survey questions, literature on general feedback types has been consulted as described below.

Previous research distinguishes between immediate feedback (e.g. during the learning task/process) and delayed feedback (e.g. after the learning task). The latter type of feedback is also referred to as outcome feedback (“How did I perform?) given when some kind of learning task is already completed, and process-oriented feedback (“How can I do better?”) guiding throughout the learning process (Sedrakyan, 2016).

Other types of feedback that have been considered in this research include cognitive feedback (constructivist theories) which targets the mental and cognitive aspects of the learning process, e.g. improving understanding of a specific concept, task, learning content/material, explaining why a learner succeeded or failed in a specific task, what needs to be improved, etc. (Sedrakyan et al., 2016).

Behavioral feedback or feedforward (socio-cognitive theories) rather than addressing cognitive issues such as difficulties understanding a task, targets reinforcing successful behavioral actions, e.g. what procedures can lead to the successful completion of a task or better performance.

Feedback targeting social and emotional context focuses on the emotional states of learners (Derick et al., 2017) which are known to play an important role in learning (Trigwell et al., 2012). Emotions drive attention, which, in turn, drives learning and memory (Värlander, 2008). Emotions are often a more powerful determinant of our behavior than our brain's logical and rational processes (Sylwester, 1994). Furthermore, emotions play an essential role in studies on attitudes and motivation (Trigwell et al., 2012).

Research on feedback is closely related to research on regulatory mechanisms of learning, in the context of which peer feedback (Tan and Chen, 2022) is offered to promote self-, co- and socially shared regulation of learning for individual and group learning (Lyons et al., 2021).

Another dimension that is relevant to this research includes learning goals and orientations that can affect feedback needs (Sedrakyan et al., 2020). For instance, students can orient themselves toward mastery- or performance-oriented goals (Salimi et al., 2022). Students with mastery goals are typically interested in learning as an end itself (e.g. “One of my goals in class is to learn and understand as much as I can.”) while students with performance goals are typically interested in learning as means of demonstrating their ability or competence (e.g. “I want to perform better than other students in my class” or “I simply want to avoid failing”). Based on learners' goal orientations, mastery- and performance-oriented feedback can be relevant (Sedrakyan et al., 2020).

Based on digital formats of feedback, a variety of forms such as text, audio, video (e.g. tutor on-screen) (Espasa et al., 2022) simulation and animation have been indicated in the scientific literature (Sedrakyan et al., 2020).

3.2 Research context and study sample

An informal interview technique was deployed for a participatory co-design. The interviews were conducted in the autumn semester of 2020 with a smaller number of teachers and students enrolled in skills courses within the Industrial Engineering and Management bachelor program at the University of Twente in the Netherlands, aiming to reveal first-hand information about areas of concern among students and needs for further focus. The analysis of the initial interviews, in combination with the literature study on lockdown education issues/opportunities, feedback typology and relevant concepts, allowed designing a survey for the new study.

A survey method was chosen to collect responses from students and teachers in the second stage. This survey in addition aimed to include respondents from broader fields of study including graduate and undergraduate students from engineering and social science programs, as well as high schools.

We conducted two surveys in 2021 during the so-called third wave of the pandemic when the educational institutions taking part in the survey already had almost a year of experience in full and partial lockdown education. The first survey was conducted in the winter semester (Sedrakyan et al., 2023), and the second slightly improved version with minor corrections was conducted in the spring semester.

In total, opinions from 247 respondents were collected in this research through interviews and a survey. Data from 45 participants in the interviews have been processed qualitatively. Respondents of the interviews included 33 students from the Industrial Engineering bachelor program at the University of Twente taking part in the skills course, out of which 10 (6 female and 4 male) students chose to give elaborate answers to the additional questions regarding the feedback experiences and needs during the pandemic, while the rest answered the questions selectively with brief responses (e.g. yes/no/maybe/I don't know). In addition, 12 teachers participated in the interviews.

Participants of the survey were reached through e-mail and social media. In total 202 responses to the survey were received (56 and 146 responses from winter and spring semesters respectively), including teachers and students from high school, bachelor and master programs in the Netherlands and Germany. The demographics of the participants are presented in Table 1. The largest age group (55.1%) consisted of participants aged between 17 and 25 years. Males accounted for 55.1% of the participants, while females accounted for 44.9%.

Out of all responses, 32.66% were received from participants with a technical background in fields such as Engineering and IT, Business and Information Technology (BIT), Business Informatics, Information Systems, Automotive Business Management, Industrial Engineering and Management (IEM), while 38.77% were recorded from participants whose field of study was in non-technical sciences such as psychology, agriculture, law, nutrition and operation research.

The majority of participants included students (79.6%), followed by a small cluster of teachers (12.24%), researchers (2.04%), researchers with teaching functions (2.04%) and students with teaching roles, e.g. teaching assistants (2.04%). Out of all responses, the highest number of responses were received from bachelor students (53.06%), followed by high school students (22.45%), Ph.D. researchers (14.29%) and students who enrolled in Staatsexamen (4.08%).

Out of all responses, 48.98% of the responses were received from the Netherlands and 40.82% from Germany.

4. Data analysis

4.1 Interview results (preliminary study)

The initial analysis of the interview data revealed that students encountered certain challenges while adapting to technology-based remote learning, specifically related to reaching for, comprehending and interpreting feedback. Students expressed difficulty in assessing their academic performance, particularly when studying in isolation. These findings were consistent with existing literature on feelings of isolation during lockdown (Eberle and Hobrecht, 2021; Hwang et al., 2020; Theobald and Bellhäuser, 2022).

Furthermore, both students and teachers reported that learners preferred inclusive types of feedback, which made them aware of their peers' performance and how they coped with similar issues. Personalized feedback with audio or video components was the predominant feedback type preferred by learners in the preliminary study. Based on the responses gathered, three main types of issues were identified as generalized challenges faced by students:

1. I would like to know that I am not alone in my struggles, not only when I have difficulties in my learning process, but also in general.

2. I would like to receive more personalized feedback from teachers using video/audio formats, which would help me to “feel what teachers mean and how they mean”.

3. Next to other types of feedback that I receive during the lockdown I would like to have fixed moments for feedback (e.g. once a week).

In addition, the interviews showed an increased interest in feedback on organizational, procedural and behavioral aspects among students (e.g. how the module is organized, what are the procedures for enrolling on certain activities, how to plan and when are they supposed to put effort, which activities should be prioritized, where to find certain resources, etc.).

4.2 Survey results: lockdown education

The first question in the survey aimed to identify the learning aspects that students prioritized for feedback during the lockdown period (Figure 1). The results indicate that learners expressed a high demand for feedback on motivation, to be able to select appropriate goals and plans to sustain engagement and drive learning progress (59.18%). Additionally, social aspects of feedback, including insights into the activities and coping strategies of peers related to stress regulation and emotional well-being (e.g. “what my peers are busy with?,” “how do they deal with stress regulation, emotional well-being?,” etc.) were also considered a high priority (55.1%). Other important feedback preferences included engagement, process-oriented behavioral feedback (e.g. assessing task progress and improvement strategies) and peer feedback (38.78%). Based on the survey results, it is evident that a significant proportion of the participants prioritize outcome-oriented feedback, which entails ascertaining the correctness of their solution to a given learning task and determining any potential shortcomings (e.g. “Is my solution of a given learning task correct? Why not?”). Specifically, this feedback type was indicated as a priority by 36.73% of respondents. In addition, performance-oriented feedback, which is designed to provide information on how well an individual performed in a given learning activity, was ranked second with 34.69% of the preferred choices. Furthermore, comparative feedback, which involves comparing an individual's outcomes with those of their peers and/or their learning goals, was indicated as a priority by 32.65% of the participants.

The majority of the respondents (59.18%) answered positively to the question “If the teacher input is well timed does the channel for communicating feedback make a difference (e.g. mail, messenger, educational platform, video-conferencing tool, etc.)?” (Figure 2).

In response to the question, whether the channels used for online feedback were sufficient, only a small number (4.08%) of participants answered negatively (Figure 3).

The study findings indicate that the majority of participants (61.22%) considered mail as the most suitable channel for receiving text-based feedback, followed by educational platforms (55.10%) and video-conferencing tools, such as Zoom and Microsoft Teams, with response rates of 51.02 and 42.86%, respectively. Instant messengers, such as WhatsApp, were the least preferred channel for feedback communication, with only 18.37% of respondents expressing a preference for them. The respondents identified various features that influenced their channel preference for feedback delivery, including ease of use (62.50%), connectivity performance (45%), audio/video recording (37.50%), interactivity features (37.50%) and speed of delivery (22.50%). Only a minority of respondents (2.50%) considered fun elements as an important factor when selecting a communication channel for feedback.

The respondents were offered the option of specifying additional requirements in the comment field labeled “Other than above, please specify”. Figure 4 presents a synopsis of the requirements expressed by the respondents for effective online feedback channels through this comment field. Based on participant responses, key priorities for features include: “Ease of use” (69.39%), “Capability to create chat rooms where students can collaborate in groups and provide/receive peer feedback” (55.10%), “Facilitation of rapid feedback communication” (53.06%), and “Performance (e.g. the capacity to operate without interruption despite Internet connectivity issues)” (51.02%).

In response to the question “Is it more difficult for a student to ask/receive feedback through online channels (e.g. by mail or in a lecture's chatroom) than on-campus?”, most of the participants (67.35%) gave a positive response and only a smaller group of participants (18.37%) thought that it was not difficult (Figure 5).

Almost half of the respondents (44.90%) considered the method of receiving feedback – whether online or offline – to be important, while 42.86% believed it was not a significant factor for them (Figure 6). Out of all participants, 67.35% preferred receiving the feedback in-person online, while 55.10% found written form through e-mail to be optimal. Additionally, 28.57% of the respondents favored in-person on-campus feedback (Figure 7).

When justifying their answers through comment field, the following main reasons were given:

1. Writing seems good as the student can easily re-read it.

2. It is easier to clarify feedback and to address open questions when it is done in-person.

It is noteworthy that the majority of participants (58.70%) answered positively to the question of whether feedback given online should be more inclusive, meaning that it should not only address individual learning processes but also make students aware of their peers' performance, and how their own performance compares to the class performance (e.g. “am I performing below or above the average of my class”?) (Figure 8).

The principal concepts identified by the participants in relation to making the feedback process more inclusive for them are synthesized as follows:

1. Making room for students to talk or work together.

2. Asking how people are doing, seeing that you are not the only one struggling.

3. For a student to interact with the teacher instead of just watching them through a screen.

4. Including a comparative element, e.g. what peer performance or approaches are in similar situations.

5. Turning on cameras and microphones.

6. Not just lecturing about the content, but also about the learning process.

7. Share experiences (e.g. discuss difficulties, make a competitive element into the assignments e.g. using a game).

8. More group work and communication via peer feedback.

4.3 Survey results: post-pandemic adaptations needs

Participants were asked to provide their preferences for post-pandemic education based on their experiences with digital/hybrid formats of education, platforms/instruments and feedback they received during the lockdown. In response to the question “Which format of classroom instruction do you prefer after the pandemic ends: on-campus, online, combined, or hybrid (e.g. on-campus lecture with online streaming)? ” (Figure 9), the majority of respondents favored the hybrid format (59%), followed by a combination of online/on-campus (46%), on-campus only (39%) and a minority preferred fully online formats (8%). These findings suggest that a hybrid format, combining online and on-campus instruction and feedback, may be the most popular option for post-pandemic education.

Regarding the inquiry “Which learning aspects did you feel lacked feedback during the lockdown period?” (Figure 10), the participants identified insufficiencies in socio-cognitive elements such as lack of social presence and motivation (65%), as well as group work (57%). This was followed by more conventional approaches, such as teacher feedback (51%) and process-oriented guidance (58%). The participants were then questioned on whether they would maintain the same preferences once physical on-campus opportunities were fully restored. Most of the participants (60.4%) reported that they would uphold their initial choices, while the remaining 39.6% indicated that they would alter their preferences.

Figure 11 displays the responses from this subgroup regarding their choices for the post-pandemic period, suggesting a preference for more traditional feedback methods, such as process-oriented feedback for intermediate outcomes and individual learning.

In response to the inquiry on preferred feedback formats for post-pandemic education, the majority of respondents expressed a preference for video formats, such as video-feedback conveyed through a conferencing tool or recording, and online or hybrid activities (54%), followed by hybrid formats with recordings (16%).

Table 2 presents a summary of the participants’ detailed feedback preferences regarding the types/elements of feedback, as well as the instruments/channels they would prefer to utilize in the post-pandemic educational landscape.

When asked “Do you think you need more inclusive types of feedback (e.g. how do I perform compared with peers?) with digital feedback (1) during or (2) after the pandemic?” the participants answered positively for both, with significantly higher preferences for post-pandemic education (52% and 60%, respectively).

Comments about online and on-campus presence in post-pandemic education were somewhat conflicting. While the majority of respondents favored online opportunities to continue facilitating swift and flexible forms of feedback after the pandemic ends (e.g. through video-conferencing tools) which could save commuting time, there was also a strong inclination toward on-campus interactions and feedback. Among the participants, many expressed a preference for “pre-recorded (micro-)lectures” and “Lectures online but tutorials on-campus.” It is notable that when online technology was involved, the respondents' preferences were oriented toward more learner-centric approaches, such as “more freedom of choices” and the “opportunity for learners to influence the choices” in post-pandemic education, e.g. “I would prefer if I could choose what/when to attend in-person or online”.

4.4 Effects from controlled variables: gender, age, role, field, level of study, country

In general, no significant effects from age, role, field, level of study, gender and country were revealed using Pearson correlation analysis. Nevertheless, there were observations of moderate effects (0.5 ≤ r < 0.7). The percentage of distributions suggests that higher degree program students, in general, reported higher satisfaction with the amount and frequency of feedback they received through online channels in lockdown education, e.g. Ph.D. researchers, master program students (75%), whereas bachelor and high school level students reported relatively lower satisfaction and higher needs for feedback (52.9% and 48.5%, respectively). Furthermore, the technical direction of the study might have potentially affected the level of satisfaction positively (68.4% of BIT vs. 46.1% of educational psychology students), suggesting that there could also be computer self-efficacy effects that made the use of digital instruments easier to use for this cluster.

Male students reported relatively higher satisfaction with the amount of online feedback they received during lockdown (33.3%) than female respondents (17.2%). There were also gender differences in preferences for feedback channels and formats. In particular, male respondents reported higher needs for in-person online feedback (52.2%) and written feedback (45.9%), e.g. via mail, whereas female respondents, in addition to similar preferences for written (54.1%) and in-person online feedback (47.8%), also reported significantly higher preferences for video (71.4%) and audio recorded feedback (66.7%) compared to their male peers (33.3% and 28.6%, respectively). Female respondents also had higher needs in feedback given on the level of a learning task (62.5%) than their male peers (37.5%), whereas male respondents had higher preferences in feedback for organizational aspects (71.4%) vs. female respondents (28.6%). Both genders reported almost similar interest in behavioral feedback and feedback comparing their own performance with peers (55–57%). No significant effects from the country variable were found. No differences in post-pandemic feedback preferences based on extraneous variables were detected.

5. Discussion

The findings of this research showed that intermediate, outcome- and process-oriented feedback for learning tasks as well as feedback for behavioral and organizational aspects were equally demanded both during and after lockdown. Motivation and engagement were found to be among the key elements learners prefer to see integrated into digital forms of feedback when communicating feedback through online instruments (RQ1a), followed by feedback considering social aspects such as enabling online group learning and peer feedback, that, in addition, can contribute to the feelings of inclusiveness among learners (RQ1b). Face-to-face feedback that would let learners engage in an immediate follow-up discussion with the teacher and is also key to ensuring social presence, was found to be among the highest preferred digital forms, followed by preferences for audio/video feedback that can be replayed (RQ1a).

The analysis also showed that, when communicating feedback digitally, more inclusive formats are preferred, e.g. informing learners about how they perform compared to peers. Performance-oriented comparative feedback that makes the learners aware not only of their own but also of peers' performance and experiences was found to contribute to participants' feelings of higher inclusiveness and lowered feelings of isolation (RQ1b). The possibility to review and provide/receive feedback from peers was among highly prioritized activities in the context of frequent group work, also for its potential to compensate for the lack of social interaction (RQ1a).

The results showed that instruments and digital channels supporting effortless interactivity and ease of access are among the highly preferred options for giving/receiving feedback in online/hybrid classrooms (RQ2), which are in addition opted for post-pandemic education (RQ4).

The needs for digital feedback were found to vary based on the level of study (RQ3), participants with lower levels (e.g. high school, bachelor programs) reporting relatively higher needs, and participants with higher levels (e.g. master, Ph.D.) reporting moderate needs in feedback. Moreover, respondents from technical study programs/backgrounds (e.g. Business and IT, Industrial Engineering Management) showed higher satisfaction with the amount, frequency and formats of digital feedback (RQ3), which could potentially be attributed to higher self-efficacy with digital environments and instruments. Gender of learner was found to potentially affect the choices of feedback and formats (RQ3), with male respondents showing preferences for face-to-face online and written (e.g. via mail) formats with prevailing interests in feedback on organizational aspects, whereas female respondents in addition also showed preferences for a larger choice of feedback channels and formats, e.g. video/audio recorded feedback, with more focus on feedback given on the level of a learning task. Both genders shared similar interests in behavioral feedback and feedback comparing their own performance to peers.

The recent experiences from the massive switch to online education also influenced the expectations/preferences for post-pandemic education for which combined formats, e.g. hybrid classroom or combinations of on-campus/online activities were opted (RQ4), with learner preferences to co-participate in making choices regarding what/when to attend on-campus/online. Combinations of on-campus and in-person feedback was prevailing in the respondents' answers for post-pandemic education. The majority of respondents also indicated preferences for keeping more digital formats of educational activities, instruments and feedback (e.g. online lecture/feedback to benefit from saving on commuting) and re-useable forms of feedback (e.g. replaying a recording).

In terms of main implications, the results of our study suggest that:

1. The need for digital feedback might vary based on the level of study (high school, bachelor, master, Ph.D.) and direction (Engineering/IT, non-technical sciences). Our data suggest that respondents at lower study levels (e.g. high school, bachelor programs) reported a somewhat higher need for feedback. Respondents with more technical direction (e.g. Engineering/IT) reported relatively higher satisfaction with the amount, frequency, and formats of digital feedback. This could be attributed to easier learning curves with digital environments/instruments due to higher technical skills and thus potentially higher self-efficacy. This result is in line with the design implications reported earlier that not only the quantity or the quality of feedback but also the personal characteristics (such as background and skills) of the learner make a difference in the variation of feedback type and format when using digital formats (Jensen et al., 2021; Maier and Klotz, 2022; Theobald and Bellhäuser, 2022). The findings from our study in addition suggests that a higher variety of feedback and communication channels/formats are opted by female learners in contrast to male ones.

2. Students have higher preferences for inclusive formats when feedback is communicated through online instruments (e.g. comparing own and peers' performance). This could potentially be attributed to lockdown effects such as learning in isolation. However, this can be also suggestive of the use of inclusive formats for digital feedback due to its potential to contribute to the participants' feelings of higher inclusiveness during online education in general. The findings also suggest that improved inclusiveness can be in addition achieved by the use of frequent group tasks that offer possibilities for reviewing each other's work and giving/receiving feedback to/from peers. This confirms that especially for distance learning it is necessary to anticipate design elements that can support such inclusive formats (McLoughlin and Oliver, 2000; Sousa, 2021).

3. While precise effects on learning goals have not been measured in this study, nevertheless the findings seem to suggest that the use of digital learning and feedback alone can affect learners' goal orientations, potentially making them more inclined to performance (e.g. “I want to perform (better than/not worse than) the average of class, …”), rather than mastery-oriented learning goals (e.g. “I want to obtain competency/skill, …”).

4. Face-to-face interaction that ensures social presence and opens up an immediate follow-up dialogue with the teacher is among highly rated needs regardless online or on-campus setup. Our data seem to suggest that whenever flexible and/or combined forms of education can be afforded (e.g. saving time on commuting) feedback in audio/video formats/channels can be an alternative for on-campus feedback. This indication to have both online and face-to-face moments of feedback, is also in line with previous findings (King et al., 2019) that online feedback is perceived as more challenging than face-to-face session, both by teachers and students, despite the online session is considered more efficient.

5. The choice of channels through which feedback is communicated can have an impact on learners' perceptions: interactive features and ease of use were found to be among the highest priorities of the participants. This indication is also in line with earlier findings that channels do not affect learner’s performance but experience (Espasa et al., 2022; Fidan and Gencel, 2022).

6. Our analysis also showed that fixed (digital) feedback moments are among highly preferred options in digital forms of education that can partially compensate for the lack of in-person feedback.

7. Digital/hybrid forms of classroom/feedback are likely to continue to be in use in post-pandemic education due to the fact that both learners and teachers experienced beneficial effects (e.g. saving on commuting, reusability by replaying recordings, etc.). In general, the results of this study suggest that while digital formats of education/feedback are the preferred options for post-pandemic education, careful balancing of online/offline approaches should be recommended to ensure the right amount of social presence, inclusiveness, as well as learner-centric co-participatory design of digital formats (e.g. allowing more learner choices with regard online/on-campus activities).

6. Conclusion

Overall, the massive switch to digital forms of education during lockdown also shaped some expectations and preferences for post-pandemic education. While the majority of respondents preferred in-person on-campus feedback to resume, they, in addition, indicated that it would be nice to continue using online feedback possibilities such as written feedback provided by mail, audio and video feedback even after the lockdown. Altering between on-campus and online activities and hybrid classrooms with the right balance (e.g. theory online, in-depth discussions and hands-on on-campus) and continuous use of digital feedback for both showed the highest preference rates. Among the advantages of keeping digital formats, saving time on commuting, the possibility to watch back recorded lectures/feedback and flexibility to access when necessary were prevailing in survey responses. In addition, respondents showed high interest in learner-centric approaches, e.g. flexibility to choose what to attend online/offline in post-pandemic education.

The results seem to suggest that the experiences from the massive switch to digital instruments and feedback in lockdown education may have effects on learners' perceptions and needs for post-pandemic education. The findings showed that learners prefer the feedback types, instruments and features that they experienced positively to stay even after the pandemic ends. Therefore, when designing digital forms of feedback, teachers should carefully consider the following aspects: feedback types, formats and communication channels. While the study has been partially conducted during the lockdown, the results are largely relevant to feedback digitalization theories, in addition to informing on post-pandemic feedback for which hybrid classroom is opted among the highly preferred formats by respondents. Additionally, constructs, such as gender, level and direction of study, can also influence learners' choices and perceptions of digital feedback. The findings need to be interpreted with caution as this study was partially conducted in full lockdown, therefore, effects, e.g. learning in isolation, could not be measured directly. Online/digital formats may increase the need for inclusive/comparative feedback also affecting students' performance-/mastery-orientations, the effects of which have not been measured directly by this study. For privacy reasons, we were also unable to measure directly if the participants that took part in our interviews also took part in the surveys. However, we could estimate the overlap range based on the participants' study program, which suggests that even though there could potentially be an overlap, it would not impact the study design and outcomes.

Further research with larger populations is recommended considering the limitations of this research as potential future research areas in the first place, aiming for a theoretical framework for feedback digitalization in online/hybrid classrooms as an ultimate goal.

Figures

Figure 1

Which aspects of learning do you think students lack or need more attention/feedback in online classrooms in lockdown education?

Figure 2

If the teacher input is well timed does the channel for communicating feedback make a difference (e.g. mail, messenger, educational platform, video-conferencing tool, etc.)?

Figure 3

Are there any specific features that make these channels more attractive to you in terms of communicating feedback?

Figure 4

What are the requirements (important features) that are needed for the feedback communication channels in online education during lockdown?

Figure 5

Is it more difficult for a student to ask/receive feedback through online channels (e.g. by mail or in a lecture's chatroom) than on-campus?

Figure 6

If the teacher input is well timed does it matter whether feedback is given online, written or offline?

Figure 7

Which way of receiving feedback seems most optimal to you during lockdown?

Figure 8

When given online, should feedback be more “inclusive” allowing comparing own vs. peers' performance?

Figure 9

What classroom/feedback format do you prefer in post-pandemic education?

Figure 10

What learning aspects did you lack feedback for most during lockdown?

Figure 11

Which learning aspects do you prefer to receive feedback for in post-pandemic education?

Table 1

Demographics of the participants

Gender
Male	55.1%
Female	44.9%
Preferred not to answer	0%
Age
Min age	17 year
Max age	58 year
<= 25	55.1%
>25 ≤ 35	8.16%
> 35	12.24%
Preferred not to answer	24.5%
Role
Student	79.6%
Teacher	12.24%
Researcher	2.04%
Student and researcher	2.04%
Teacher and researcher	2.04%
Student and teacher	2.04%
Study direction
Technical (Engineering & IT)	32.66%
Non-technical sciences	38.77%
Preferred not to answer	28.57%
Study level
High school	14.29%
Bachelor	53.06%
Master	12.24%
PhD	14.29%
Staatsexamen	6.12%
Country
Netherlands	48.98%
Germany	40.82%
Preferred not to answer	10.2%

Note(s): Staatsexamen (“state examination” or “exam by state”) is a German government licensing examination that future physicians, dentists, teachers, pharmacists, food chemists, psychotherapists and jurists (i.e. lawyers, judges, public prosecutors, civil-law notaries) as well as surveyors have to pass to be allowed to work in their profession (https://en.wikipedia.org/wiki/Staatsexamen)

Source(s): Table by authors

Table 2

Preferences for feedback adaptations after the pandemic

Feedback type	• Virtual questions and answers groups/sessions with follow-up immediate feedback • Online peer feedback • Online meetings with students once in a while • More personalized feedback made possible through online channels • Breakout rooms per topic easy to find answers to questions • More exam practice material with intermediate online feedback
Elements	• Online lectures for theory with follow-up online feedback, on-campus for in-depth discussions, hands-on tutorials/workshops, projects • Video-recordings of lectures/tutorials that can be learned at own pace
Channels	• Online feedback via mail • Audio/video feedback as they can be replayed
Instruments	• Messengers for quick interactive feedback, e.g. WhatsApp • Conferencing tool for face-to-face feedback sessions, e.g. Teams, Zoom, Skype • Peer-feedback tools (e.g. Padlet/Eduflow) • Topic-based interaction, e.g. Slack • Any providing flexible access from various mobile devices

Source(s): Table by authors