Enhancing cognitive motivation: an evaluation model for emergency preparedness exercises

Abstract

Purpose

Evaluating emergency preparedness exercises is crucial for assessing enhanced knowledge, facilitating learning and implementing knowledge in organizations. The cognitive process of motivation for action is a precursor for action, coping behavior and individual learning. This study aims to focus on how guided evaluation of emergency preparedness exercises can enhance cognitive motivation and influence the mental readiness of exercise participants.

Design/methodology/approach

This is a conceptual paper with a model approach design. The main conceptual contribution is suggesting a model for guided evaluation in emergency preparedness exercises. We present a theoretical background for understanding the increase in motivation based on social cognitive learning theory. In particular, this study discusses how different evaluation steps contribute to enhanced motivation and learning for exercise participants.

Findings

Increased motivation and enhanced personal performance standards could be achieved through using processes that lead to self-efficacy in guided exercise evaluation. Specifically, sources of enhanced motivation, such as repeated coping experiences, self-regulation processes, mastery motivation and performance motivation, would proliferate the readiness of individual crisis managers and teams.

Practical implications

This article suggests an evaluation model for use in emergency preparedness exercises. This approach combines bottom-up and top-down processes for debriefing, reflection and feedback, both individually and in teams. This approach aims to enhance exercise participants’ motivation and utilize exercise evaluation for organizational learning.

Originality/value

The conceptual discussion leads to developing implications for evaluation practice, suggesting how to structure evaluation and why. This study is novel for its explanation of how to use evaluation in the learning process.

Citation

Andreassen, N., Elvegård, R., Villanger, R. and Johnsen, B.H. (2024), "Enhancing cognitive motivation: an evaluation model for emergency preparedness exercises", The Learning Organization, Vol. ahead-of-print No. ahead-of-print. https://doi.org/10.1108/TLO-06-2023-0100

Introduction

Emergency preparedness exercises are vital to ensure that personnel and responders are adequately trained to perform the tasks required of them during emergencies. These individuals often have little applicable experience to rely on in the case of complex emergencies (Haddow, Bullock, & Coppola, 2014). As Sinclair, Doyle, Johnston, and Paton (2012) claimed, training exercises are intended to develop people’s capacity to respond to a crisis, as well as develop personal standards for applying a specific skill. Emergency preparedness exercises form an integral component of emergency management training, traditionally serving to stimulate testing procedures, ensure proficient handling of technology and offer experience in managing high-pressure situations within a safe and supportive environment (Paton, Flin, & Violanti, 1999; Sinclair et al., 2012). These exercises play a crucial role in developing effective crisis management plans and fostering a deeper understanding of the complexities involved in crisis response. All emergency preparations aim to enhance the readiness of well-functioning, proactive and robust responders and emergency preparedness personnel. The overarching question is how to enhance emergency responders’ readiness through exercises to cope with high-risk operations. Although preparation exercises are an essential means of enhancing the readiness of emergency response personnel, there is no guarantee that they will be able to perform their tasks as anticipated.

Motivation is central to gaining individual learning in a training program or exercise (Marcus & Shoham, 2014; Tobias & Fletcher, 2000). However, there is a lack of understanding regarding how motivation enhancement can be integrated into exercise structure (Badura, Grijalva, Galvin, Owens, & Joseph, 2020). Prior studies investigated the effectiveness and indirect influences of motivation on different management aspects, such as relational coordination (Bolton, Logan, & Gittell, 2021), coaching competence (Boyatzis et al., 2022), creativity behavior in organizations (Auger & Woodman, 2016), daily self-leadership and proactive behaviors (Bakker, Breevaart, Scharp, & de Vries, 2023), gamification of training programs (Mohanty & Christopher, 2023) and others. Afsar and Masood (2018) explored motivation variables by analyzing employees’ willingness to experiment and take risks in work conditions with tight resources and time constraints.

Exercise evaluation is recognized as a vital step in finishing the learning cycle and assessing the enhanced knowledge, skills and competencies of the participants. Evaluation may facilitate the effectiveness of learning and processes for implementing and transferring knowledge to organizations (Kozlowski, Brown, Weissbein, Cannon-Bowers, & Salas, 2000). However, exercise evaluation is often an unsystematic top-down process that focuses on increased performance rather than increased learning (Berge, 2008). In many cases, exercise evaluation is limited to a symbolic value (Beerens & Tehler, 2016), which may result in nonoptimal learning and, in the worst case, decrease motivation for coping with disasters (Seggelen-Damen & Van Dam, 2016).

This study echoes the call of Beerens and Tehler (2016) for studies to develop scientific foundations for emergency preparedness exercise evaluations. They remind researchers that exercises are resource- and time-consuming activities, so it is important to understand how the evaluation method achieves and fulfills the exercise purpose. Specifically, this study considers how emergency preparedness exercise evaluation enhances participants’ cognitive motivation by incorporating various approaches and tools. By integrating the literature on social cognitive theory and emergency preparedness exercise evaluation, it is possible to discuss the influence of various evaluation tools on participants’ cognitive motivation during exercise evaluation.

The structure of the rest of the paper is as follows. First, a critical literature review on exercise evaluation approaches and an overview of evaluation tools are presented. Then, we proceed by reviewing the concept of cognitive motivation as a vital part of the mental readiness of responders. By elaborating on factors influencing cognitive motivation during an exercise, we develop recommendations for an emergency management exercise evaluation model. We discuss and conclude on how different steps in the evaluation model may capture important motivational factors to enhance mental readiness to respond during crises.

Literature review

Emergency preparedness exercise evaluation approaches

Evaluation is recognized as an important component of exercise design. Evaluation validates what works well for the participants and what should be improved. It also assesses exercise performance, promotes the development of the cognitive competence essential for communication and decision-making in emergencies and facilitates operational integration (Paton & Jackson, 2002). Tobias and Fletcher (2000) claim that theoretical models of training evaluation are scarce. Most evaluations rely on trainee reaction data and performance, not on learning data; on their account, the claim that training or exercising can lead to learning is misleading (Tobias & Fletcher, 2000, p. 484). In a review study of scoping disaster exercise evaluation, Beerens and Tehler (2016) highlight that literature describes how lessons are mainly collected from observations or debriefings, primarily making use of the participants’ impressions and judgments of observers. In a subsequent study, Beerens, Tehler, and Pelzer (2020) delve into the perceived usefulness of evaluation descriptions, with a specific focus on the clarity of how the object of the evaluation, the analysis and the conclusions are presented. While this research underscores the significance of documenting emergency exercise evaluations, it also highlights a pressing need for clear and comprehensive guidelines in this regard.

Kirkpatrick’s (1976) evaluation approach is considered the most widely used and includes four levels of evaluation: trainee reactions, learning, behavior and organizational results. This methodology is often used for training programs and large-scale exercises and focuses on whether the learner has acquired the knowledge and skills. Conventionally, these evaluations have typically centered on providing a detailed account of the activities undertaken, often overlooking the crucial inquiry into the underlying reasons for participants’ performance. It is challenging to capture the organizational changes that have occurred after an exercise ends. Trainees’ reactions and impressions of coping with training fulfillment may influence the development of commitment and motivation to perform similar tasks in the future (Tannenbaum, Mathieu, Salas, & Cannon-Bowers, 1991). Kraiger, Ford, and Salas (1993) expanded Kirkpatrick’s (1976) evaluation typology by incorporating notions from cognitive psychology. They suggested that new conceptualizations of evaluation approaches should measure organizational learning after implementing training and exercises. In particular, they suggested that learning may be evident from changes in three constructs: cognitive, skill and attitudinal capacities. Cognitive outcomes include verbal knowledge, knowledge organization and cognitive strategies. Skill-based outcomes include skill compilation and automaticity. Finally, attitudinal and motivational outcomes, such as disposition, self-efficacy and goal-setting, are proposed as key constructs affecting learning outcomes (Tobias & Fletcher, 2000).

Another well-known evaluation methodology is that of Fagel (2010, p. 280), who suggests the following criteria: whether the exercise has achieved its objectives; whether there is a need for improvements in plans, procedures or guidelines; management system as a whole; training and staffing deficiencies; equipment needs; and need for additional exercising. It is fundamental to determine, during evaluation, whether the intended outcomes of training have been achieved. An exercise evaluation methodology must seize opportunities for future follow-up and modifications. Evaluation should enhance organizational learning in relation to expected improvements in capacity and capability and support overall emergency management effectiveness and informed decision-making (Borodzicz & Van Haperen, 2002). However, due to the uniqueness and rarity of emergencies, there might be no clear criteria for the right solution, which might lead to the avoidance of coping behavior or the use of nonfunctional solutions in some situations (Skjærvik, Villanger, Aandal, & Johnsen, 2024). Although the evaluation approach suggested by Fagel (2010) includes important practical elements, it lacks an explanation of what fuels the development of expertise and motivation. This may call for an increased understanding of more specific methods to enhance motivation and learning in situations where established procedures are not optimal or even contra-indicated.

Exercises are used to improve emergency preparedness plans; however, the exercise itself needs to be evaluated for the continuance of the exercise cycle (Sinclair et al., 2012). According to Paton and Jackson (2002), the didactical exercise planning approach is fundamental in facilitating the pedagogical use of learning for the disaster response team. Although this is not a specific evaluation approach, didactical planning includes evaluation and assessment as a component in relation to any educational activity. Exercise evaluation is an important component of exercise design, and with regard to the relational didactical model, it should relate to reaching learning outcomes within the setting framework and given exercise scenario and context. To achieve a credible exercise design, the evaluation approach must be planned thoroughly to mirror the participant prerequisites for learning and the learning process (Elvegård & Andreassen, 2023). Evaluation is an important step in organizational learning toward uncovering required improvements in capacity and capability, overall emergency management effectiveness and informed decision-making (Borodzicz & Van Haperen, 2002). At the organizational level, learning is proportional to the interpretation of learning outcomes of the organizations (Daft & Weick, 1984). Interpretation is needed to evaluate a result or refine an action (Nakanishi, 2023). Exercise evaluation could be viewed as a method of organizing knowledge structures. According to Marks, Sabella, Burke, and Zaccaro (2002), one should focus on the knowledge structures that lie behind the behavior. These structures are prerequisites for team processes and team performance (Marks, Zaccaro, & Mathieu, 2000, Marks et al., 2002). Critically, Torgersen and Saeverot (2016) discuss a didactic planning approach to training for unforeseen events, stating that learning goals during unpredicted or complex events were not known or apparent under such conditions and, therefore, had to be replaced by other, more generic competence objectives. In this case, it might be possible to influence participants’ motivation rather than contribute to uncertainty in training outcomes.

Evaluation tools

After determining the broad objectives of evaluation, the next step is to identify how to conduct it. In most cases, evaluations are based on a pre- and postexercise survey that assesses participants’ impressions and viewpoints (Beerens & Tehler, 2016). Beerens and Tehler (2016) claim that the existing literature on exercise evaluation emphasizes the purpose of the evaluation as an important element of or step in an exercise rather than providing details and explanations of how evaluation should be conducted.

The most widely used methodological tools during exercise evaluation are debriefing, feedback and reflection. Debriefing can be used to provide comments to participants that increase their self‐awareness and create an opportunity for them to learn from experience (Borodzicz & Van Haperen, 2002). Debriefing is a guided process during which students and faculty engage in a session to examine actions and learning outcomes (Coomes, 2019). Debriefing has a top-down approach; it is usually run by an experienced facilitator who aims to determine what went right or wrong and why without criticizing anyone. Debriefing includes facilitating students’ integration of theoretical and practical knowledge (Coomes, 2019).

Feedback is used to provide information to a learner in response to some action to help them improve their knowledge and skill acquisition (Shute, 2008). Feedback involves evaluating observed behavior and performance; it also raises awareness among participants regarding the learning goals. In emergency response exercises, feedback models are an important evaluation tool that enhances the learning outcome of the exercise. High-quality feedback is characterized by being specific, informative, objective and nonjudgmental in assessing behavior (Chowdhury & Kalu, 2004). Learning-oriented feedback can be understood as a reaction to an emergency management exercise that increases learning. Participants can reveal their learning points against their practice and experience and evaluate the exercise. Previous studies, where trainees do not encounter any cases with prior experience, utilized a five-step model for feedback (Alrø, Nørgaard Dahl, & Schuman, 2017). These five steps include evaluation of concrete observed behavior, subjective experiences of the event, subjective interpretations of the event, suggestions for improvement and dialogue. The feedback should be collected immediately after the case has been completed by establishing a dialogical space during this phase (Decuyper, Dochy, & Van den Bossche, 2010).

A significant method of generating a more adequate understanding is through the process of reflection. Reflection is traditionally viewed as a cognitive process of critical questioning that focuses on the “self” and self-related experiences (Matthew & Sternberg, 2009) and provides meaning and valence to the cognitive process at stake (Bandura, 1999). Reflection is an evaluation tool used in emergency preparedness exercises that stimulates interpretation, alternative understanding of an event and explanation of the need for knowledge. Individuals, teams and organizations can learn by reflecting on emergency preparedness exercises based on their performance and experience.

Low-quality feedback often results in defensiveness, which hampers the process of reflection (Chowdhury & Kalu, 2004). If reflection fails to offer guidance, it may result in the maladaptive strategy of rumination and prevent organizational learning (Seggelen-Damen & Van Dam, 2016). Rumination is the process of reoccurring negative thoughts, creating negative emotions and feelings of incompetence. If an individual has a motive, that is, a high expectation that a specific behavior would lead to an attractive goal, the probability of performance would be high. If the expectations were low, the probability of a response would be low. Rumination can reduce the sense of self-efficacy by learning a lack of coping in a critical situation. However, according to Seggelen-Damen and Van Dam (2016), reflection may adapt to the effect of rumination. Addressing an individual trainee’s subjective expectation of coping as a key factor is important in a crisis response that requires effort and learning. Maintenance of individual standards is crucial for team performance skills.

Reflection and feedback techniques involve both team and individual information exchange in order to guide the organization of knowledge structures. Team reflection and feedback have the purpose of creating new or altering previous mental models.

Cognitive motivation

Vakola (2014) acknowledges cognitive motivation as an aspect that is associated with confidence in one’s abilities (standards). Together, these two factors support the mental readiness of responders. Mental readiness refers to the ability of a person to create a balanced psychological state in which they can respond to a situation (Thompson & McCreary, 2006). Individual mental readiness refers to a variety of different cognitive and physiological processes. It comprises a set of interrelated, interdependent functions that collectively produce the desired response outcomes (Marquardt, Hannig, & Hannig, 2018). In unforeseen circumstances, where mental preparedness for specific behaviors is lacking, the effectiveness of crisis management may be compromised. Thus, to determine what enhances motivation, we need to look more closely at cognitive motivation and personal standards.

The social cognitive learning perspective is based on the core assumption that people act in a goal-directed way. In order to organize behavior toward a specific target, individuals use perceived ability to perform actions in specific situations as the key internal motivational process (Bandura, 1999; Schunk & Dibenedetto, 2020). This perceived ability refers to self-efficacy (Bandura, 1999). Self-efficacy is not the intention itself to perform a specific behavior but the evaluation of the individual’s ability and impetus of goal-directed behavior. Humans have to be motivated to execute the responses needed. According to Schunk and Dibenedetto (2020, p. 1), “motivational processes are personal (internal) influences that lead to outcomes such as choice, effort, persistence, achievement, and environmental regulation.” Motivation refers to processes that instigate and sustain goal-directed activities. The cognitive process of motivation for action could be viewed as a precursor for action and a predictor of coping behavior. Another important element in goal-directed behavior is personal standards. Personal standards are defined as subjective frames of reference, or thresholds, against which occurring events are compared. In order to transform intention into action, a threshold has to be exceeded (Dettaff, Hollinshead, Graham, Baumann, & Fluke, 2020). In operational settings, personal standards are important to understand goal-seeking behavior (Bandura, 2001). Emergency managers and responders compare tactics, behavior, and operational procedures against their individual standards to ensure alignment and effectiveness.

This does not automatically result in the performance of these tactics and procedures in a crisis situation. Behavior is guided by a combination of self-reflection on previously learned history and forethought, previous causes of success and failures and the attraction of reaching the goal according to standards and cognitive motivation. Motivation for action is the core element of individual mental preparedness, which is based on the self-regulatory process of self-judgment. In all forms of crisis management, crisis response participants are required to regulate and maintain emotional, cognitive, and behavioral control to ensure important safety aspects and maximize operational effectiveness (Thompson & McCreary, 2006). Social cognitive learning theory separates task efficiency (self-evaluation of ability to act) from self-regulatory or barrier efficacy. The latter includes the ability to perform in situations where obstacles are present, but the target is attractive and attainable.

Unpredictable operations demand that emergency response coordinators fulfill a wide range of management functions and roles related to information sharing, decision-making, coordination and command. Role clarity is a prerequisite for behavioral readiness. These role expectations may be complicated in the case of unpredictable emergency management operations because it is a multi-staged process with different behavioral requirements for responsible individuals and groups at different stages of the process (Smits & Ezzat Ally, 2003). Kraiger et al. (1993) explained that at high levels of knowledge and complexity, two types of motivation predominate one after another: performance motivation (getting it right) and mastery motivation (willingness to learn).

Personal influences include processes that help instigate and sustain performance motivation (Schunk & Usher, 2019). For instance, personal factors or standards guide behavior, and behavior can create expectations to fulfill a role. The forethought of what the individual wants to obtain influences the direction and intensity of behavior. Increased specificity of behavior and increased proximity to the attractive goal enhances mastery motivation (Bandura, 1986; Locke & Latham, 2002, 2015; Locke, 2018).

It is then possible to summarize the factors that lead to self-efficacy and enhanced personal standards. It is important to establish sources of enhanced motivation, that is, repeated coping experiences, self-regulation processes, role performance motivation and mastery motivation.

Design

This article is designed as a conceptual paper. As Jaakkola (2020) argues, a conceptual paper can bridge existing theories in interesting ways or link work across disciplines. She reviews four main types of conceptual design approaches – theory synthesis, theory adaptation, typology and model. Of these, the theory synthesis approach to disaster exercise evaluation encompasses various aspects, as described in several review articles (e.g. Beerens & Tehler, 2016; Borodzicz & Van Haperen, 2002). The domain theory of exercise evaluation is not well understood, and basic evaluation tools are still widely used in practice; therefore, theory adaptation or typology construction does not seem to be required. However, a conceptualization model to explain and predict how each of these methods affects participants’ motivation needs to be designed. A model is a methodological argumentation explaining why a sequence of evaluation steps leads to an outcome. We have therefore chosen a design approach instead of synthesis, adaptation, or typology conceptualization methods, which are useful for further basic research and theoretical studies, and suggest a model that can be used and tested in practice.

The main conceptual contribution of this article is to suggest a model for guided evaluation of emergency management exercises by explaining how different steps in this process contribute to enhanced motivation and learning among exercise participants. We summarize arguments for relationships between exercise evaluation tools and sources of motivation and propose a model that should be tested later in empirical studies. This study discusses the theoretical framework for the evaluation of emergency preparedness exercises and addresses key elements or tools of evaluation. These tools are sourced from a literature review, although their sequence is based on the practice of the coauthors, who are professionally engaged in crisis management training at the Center for Crisis Management and Collaboration – NORDLAB, Nord University, Norway, and the Norwegian Police National Emergency Center at the Oslo Police District. Additionally, the suggested steps are based on previous emergency preparedness exercises. These steps are further explained from the perspectives of social cognitive theory, which serves as the theoretical foundation of this conceptualization. We suggest a methodology to explain how and why different evaluation steps enhance the cognitive motivation of participants, thereby opening up new vistas of knowledge.

Factors influencing cognitive motivation during exercising

We examine how evaluation tools that might potentially influence cognitive motivation are incorporated into an exercise evaluation approach. From the perspective of social cognitive learning theory, we explain how different evaluation tools may target various sources of motivation – repeated coping experiences, self-regulation processes, role performance motivation and mastery motivation – for those individuals involved in crisis management exercise evaluations.

Coping experience

Motivation can be achieved through processes developed from experience. Malmberg, Hagger, and Webster (2014) identified repeated coping experiences as important sources of efficacy. They add that a history of successful coping during similar conditions would increase the generalizability of self-efficacy. Self-efficacy varies according to the dimensions of intensity and generalizability. The intensity could be affected by evaluations of the goal-directed behavior (i.e. progress toward success) as well as assessment of environmental factors (type of barriers, comparison with others, etc.). The generalizability dimension could be influenced by previous coping experiences (Blanchard et al., 2007; Oettingen, Sevincer, & Gollwitzer, 2018).

Crises and emergencies are characterized by ambiguity caused by confusion, lack of information, time constraints, threats, uncertainty and possible serious adverse consequences. Sometimes, available or previously learned procedures are shown to be insufficient. Two interesting observations are often seen during exercises when operators meet situations where previously learned tactics and procedures are not suitable (Skjærvik et al., 2024). First, a “more of the same” type of reaction has been observed. Although it is obvious that a procedure does not work, the same procedure is repeatedly applied. This could, at best, have no effect and, at worst, be fatal. The second observation is an avoidance reaction, where operators do not initiate any coping behavior based on the argument that they need new tactical assets. As a consequence, no initiative is shown while they wait for the resources that are lacking (e.g. fire trucks).

Various exercises in emergency preparedness may be aimed at gaining coping experiences or challenging standard tactics and procedures. For exercise evaluation, behavioral markers or areas of attention can be designed to be the focal points of feedback and reflection, especially to facilitate the sharing of understanding and suggestions from team members. Decisions on the areas of attention are made by generating expectations regarding the anticipated outcomes of their coping behavior. The shared mental model approach fosters shared coping experiences and comments from the team (Saus, Espevik, & Eid, 2010). The reflections of all members help to monitor individual coping experiences or a willingness to change strategy. During the reflection activity, team members develop shared knowledge structures, including capabilities and capacities within the team. During the feedback activity, team members express a collective attitude toward the solution of the problem, and participants actively seek information to solve any potential problem.

Self-regulation

Self-regulation is driven primarily by the comparison of personal standards (Bandura, 2001). Personal standards are developed as a consequence of previous experience, and self-regulation influences people’s behavior based on comparison to the standards of conducting the action. Self-regulation is characterized by one’s motivation and ability to regulate and influence their behavior. Reflection and feedback after exercises are essential in developing and adjusting standards, thereby influencing the motivation for action.

Self-regulation includes the processes of self-monitoring, self-judgment, and self-reaction. Self-monitoring includes cognitive processes like attention and understanding one’s own behavior, motivation, goals and perception of performance reflection. An individual reflection on learning goals could be a source for self-monitoring processes. For instance, Phan (2014) showed that reflection mediated the relationship between self-efficacy and performance, whereas Seggelen-Damen and Van Dam (2016) demonstrated that reflection mediated between self-efficacy and well-being. In addition, feedback from team members (i.e. dialogue) is a central element that fuels self-monitoring through reflection on reaching learning outcomes. Thus, reflection is essential to the development of personal standards.

Self-judgment is evaluating one's own behavior, including development, and comparing it against personal standards. Standards can be related to work effort and cooperation (Bandura, 1986). According to Zimmerman (2000), self-regulation can be learned and altered through the phases of forethought, performance, and self-judgment or self-reflection. Forethought is represented by goal-setting, planning, and deciding an appropriate response. Individual reflection on behavior may be the main causal factor for self-reflection, motivating future actions and goals (Bandura, 1979). Reflection critically questions the “self” standards and self-related experiences during exercise (Matthew & Sternberg, 2009). People who possess standards that spur hard work expend more resources when confronting difficulties and are subsequently more motivated than less determined individuals. Likewise, standards encourage collaboration and teamwork.

Self-reactions include affective processes often driven by self-reinforcement and punishment. It is important to debrief personal perceptions, emotions, and reactions. The process of debriefing is related to cognitive motivation for the execution of targeted performance. Debriefing can lead to an emergency management exercise designed to increase learning and motivation. For example, in Alrø et al. (2017) model, participants discuss concrete observed behavior, subjective experiences of the event, subjective interpretations of the event, suggestions for improvement and dialogue. According to Schunk and Usher (2019), the relationship between self-efficacy and the execution of behavior is strengthened as a function of feedback from team members on the operators’ improved self-evaluations.

Standards enhance problem-solving through self-reflective processes. Self-efficacy is the result of reflection upon the operators’ previous successes and failures combined with forethought related to desired goals and ways to achieve the goal. Thus, by increasing the quality of reflection, a positive effect on motivation and, consequently, performance could be expected. We suggest the use of a guided process of reflection in order to increase both learning and motivation.

Role performance motivation

Performance motivation can refer to the type of motivation that Kraiger et al. (1993) labeled as “getting it right” in situations with high levels of knowledge and complexity. In general, there is a motivational tendency that is adaptable to situations (Marcus & Shoham, 2014). In an emergency response, participants can adapt their developed role or planned function most appropriately to a situation and its complexity. Therefore, they can adapt to a motivational disposition for role performance according to a situation.

Exercises usually involve the coordination of emergency response activities, which are predeveloped through procedures or emerging in response to unplanned situations. The coordination processes may be preprogrammed in the form of standard operating procedures, mechanisms, or rules and may increase the effectiveness of the response in stable environments (Okhuysen & Bechky, 2009). Therefore, during ab exercise, it is important to start with a role performance disposition, determining the preplanned performance. After the exercise, it would be beneficial to reveal how the role or planned function was adapted to the situation and its complexity. Feedback from an experienced facilitator helps to determine what went well and what went wrong. Another example is a crisis management plan, which is often the starting point in an emergency preparedness exercise. (Smits & Ezzat Ally, 2003). The exercise evaluation process serves to promote apt role performance in crisis management or prompts the revision and updating of the crisis management plan. Developing an understanding of one’s own role is crucial to developing performance motivation.

Mastery motivation

Mastery motivation refers to the “willingness to learn” (Kraiger et al., 1993). Expectations for success in an emergency situation may refer to the amount and quality of effort, the choices related to specific tasks and the level of performance (Marcus & Shoham, 2014). In quickly changing situations and complex environments, coordination is less dependent on preplanned design than on ongoing work activities that emerge in response to imminent coordination challenges (Isabelle et al., 2012). In such situations, organizations and people engaged in response operations will need to adapt to their roles and choose which tasks, people, and expertise are needed to coordinate at different times. Mastery motivation should be important, especially when a crisis is escalating. Exercising one’s own and others’ roles (cross-training) provides a base for reflection on how other individuals perceive the established roles and demonstrates the deeper implications of mastery motivation. After participants have talked through a situation, the trainees revert to their own roles and reflect on the outcome (Wilson, 2000). Learning-oriented feedback from facilitators can enhance role performance understanding. For mastery motivation, developing an understanding of not only one’s own but also others’ management roles is crucial.

Toward a guided exercise evaluation model

We propose the following as a general evaluation model for emergency preparedness exercises to increase cognitive motivation for action during a crisis (Figure 1). The left side of the figure shows recommendations for guided exercise evaluation. The right side shows the sources of motivation discussed in the previous chapter, which are coping experience, self-regulation, role performance motivation and mastery motivation. These sources each include processes or factors that evaluation sessions aim to develop. These recommendations may be applied to exercise evaluation by emergency management professionals and leaders of crisis response teams.

In the exercise evaluation model, reflection is coupled with feedback procedures. The dual process of “bottom-up” and “top-down” evaluation increases the sharing and (re)constructing of knowledge structures (Molin, Haelermans, Cabus, & Groot, 2021). Subject matter experts, facilitators, or evaluators play a crucial role by offering constructive feedback on the participants’ performance. This is the “top-down” process in evaluation. The evaluators should facilitate a reflection process to increase the participants’ understanding of their performance. This part of the evaluation process is the “bottom-up” evaluation. “Bottom-up” reflections or debriefs provide an opportunity for shared understanding with the participants, which increases the effectiveness of the evaluation (Faber, Luyten, & Visscher, 2017). The leader of the evaluation should emphasize retrospective reasoning, forethought, and regulation of emotions, as well as instilling trust in the participants. In line with that, Kim (2013) claims that only with two-way debriefings can individual and organizational learning through simulations finally take place.

The evaluation procedure should be flexible. If a strictly structured procedure is employed, the main themes and learning points could be missed due to the specificity of the exercise scenario. We suggest that the guided reflection should be based on predefined themes and provide an opening for significant issues experienced by the exercise participants.

A prerequisite for the model is a common theoretical base for the trainees and the facilitators. A shared understanding would increase the specificity of the reflection and feedback (Decuyper et al., 2010). A suggested model for an emergency management exercise evaluation that could enhance cognitive motivation is presented in Figure 1.

Discussion

We discuss step by step how the evaluation model may capture important motivational factors to enhance mental readiness to respond during crises. We suggest that the facilitators begin guiding participants’ reflections even before the exercise begins, often during the brief initiation process. By having the trainees define their areas of attention for the upcoming exercise, the main themes for learning and reflection are set. Trainees may, therefore, determine areas of attention that could be focus points during evaluation (Skjærvik et al., 2024). A focal point could center around the participants’ performance related to these predefined topics. The areas of attention should be related to significant technical and/or nontechnical skills associated with the successful completion of the exercise. The facilitator could actively organize the process and encourage trainees to focus on relevant aspects, ensuring that the topics defined are reflected upon and targeted during feedback.

The next step would be team reflection and feedback from team members. As Decuyper et al. (2010) claimed, effective team learning requires establishing a possibility for dialogue among team members and facilitators. The integration of knowledge structures commences during the reflection process, occurring within teams and across team interactions. Participants share their understanding of their capacities and capabilities in a given situation.

Guided exercise reflection involves feedback from team members as a next step to facilitate suggestions from the team that can confirm their coping experience during the exercise. The quality of feedback from team members and subject matter experts would influence the effects of reflection (Choi, Oh, Lee, & Hyun-Sun, 2020). During feedback, team members discuss, explain, and reflect upon tactical and nontechnical aspects of the exercise. These abilities may be taken further to other situations. Nontechnical skills as precursors of decision-making and team processes (e.g. shared mental models, role expectations and role acceptance) could also be focused on. Acceptance of making progress increases belief in coping, enhancing motivational outcomes.

Exercise evaluation proceeds with facilitators’ and subject matter experts’ feedback on what went well and what can be improved, as well as suggestions for alternatives. This feedback will complete the development of role performance motivation, comparing feedback from facilitators to the preplanned role performance the participants went through before the exercise. If the training consists of several scenarios or the participants’ expertise is low, expert feedback will have a bigger role in influencing role performance motivation. Time should be dedicated for individual participants to reflect on the causes of their and their team’s successes or failures. To increase the sharedness of the mental models, information related to different roles should be shared and discussed. Information and performance needs related to different roles could also be reflected upon since this could be a part of vicarious experience and social persuasion aimed at building shared mental models.

As a next step, a debrief of trainees’ reactions and affection contributes to self-reaction processes and, consequently, to self-regulation as a source of motivation. The affective processes are important to facilitate new or alternative suggestions within teams, as well as to enhance encouragement from facilitators and improve self-regulation.

The main aim of reflection is to facilitate the configuration of mental models, allowing participants to make comparisons with their own abilities or develop their own personal standards of behavior in a given situation. Individual reflection on behavior begins a process of self-reflection, leading to self-regulation and increased cognitive motivation. To increase cognitive motivation during individual reflection, the reflection phase could involve building mental models. Both the present exercise’s relation to previous experience and forethought could be touched upon. Sharedness and accuracy of mental models, communication routines, trust, supporting behavior, monitoring, adaptive behavior, and team orientation and coordination are examples of nontechnical skills that could be involved in the post-training reflection. To achieve this, reflection may involve describing and explaining the situation and predicting possible near-future conditions. Evaluation leaders should consider written forms of individual reflection on behavior as the next step, as it may facilitate increased autonomy in reflection and systematized feedback over time.

Individual reflection on reaching the learning outcomes may contribute to self-monitoring and mastery motivation. Self-monitoring is another factor that enables self-regulation as an important pathway toward increased motivation (Schunk & Dibenedetto, 2020). Participants should reflect on reaching their learning goals and, eventually, extending their personal standards on a subject. Trainees can also focus on predefined areas of attention (Skjærvik et al., 2024). Personal factors of self-monitoring, including coping expectations and personal standards, are suggested as topics. Environmental aspects may facilitate or mitigate the motivation for executing coping behaviors. This could include novel situations where previously learned procedures are inadequate. After participants reflect on their behavior, they revert to their own and others’ roles in the situation. A reflection on understanding a broader system and the way forward creates a willingness to learn and increases mastery motivation.

Conclusion

In this conceptual paper, we review some of the evaluation approaches and tools most frequently used during emergency preparedness exercises. In response to the demand for studies on developing scientific foundations for emergency preparedness exercise evaluations, we have suggested a model for guided evaluation. The theoretical implication of this paper is that, by using the social cognitive perspective as a method theory, we have helped close a gap in the literature that explains how to structure evaluation with a central assumption that different steps in an evaluation model enhance cognitive motivation and personal standards. The emergency manager’s or responder’s motivation to perform goal-directed actions in order to mitigate a crisis is the most crucial element in emergency preparedness. Introducing an evaluation process grounded on social cognitive theory may enhance mental readiness for crisis situations. Furthermore, such an approach also results in an increased and shared understanding of what is learned.

The suggested guided evaluation model includes the main steps to help plan and guide the evaluation of an exercise in emergency management. The plan for exercise evaluation will depend on the learning outcomes, complexity of the exercise, type and timeframe. The steps are different configurations of debriefing, feedback, and reflection, depending on whether it is completed individually or with a team, and guiding both approaches in a bottom-up or top-down direction. The evaluation guide discusses the following steps: reflection with the team (before the exercise), feedback from team members, feedback from facilitators, debriefing of reactions and individual reflections on behavior and reaching learning outcomes.

The practical implications of this paper are that the evaluation of exercises based on the principles of guided reflection could strengthen emergency service management practices through the development of proactive and motivated operators with high personal standards or confidence in their abilities to manage emergency preparedness exercises and actual crises. In particular, the sources of enhanced motivation, including repeated coping experiences, self-regulation processes, performance motivation and mastery motivation, would proliferate the readiness of individual crisis managers and teams.

The suggested model should be tested in further research studies. An empirical study is needed to test the model’s and concepts’ applicability in exercises with different emergency preparedness organizations. Exploring whether distinct steps warrant emphasis in various types of exercises could offer valuable insights. Future empirical studies should specifically investigate potential biases associated with the selection and sequence of different steps. These studies can draw on a review of existing literature and leverage the professional experience of practitioners, forming a robust foundation for this conceptual study. Additionally, a theoretical inquiry could establish connections between the evaluation steps and discussions on sustaining collective situational awareness, with a focus on team and organizational behavioral levels.

Figures

Figure 1.

An evaluation model for emergency management exercises to enhance cognitive motivation