When employees understate their stress: defensive biasing in work stress surveys

Abstract

Purpose

The purpose of this study is to investigate the phenomenon of defensive biasing in work stress surveys, which occurs when employees trivialize potential stressors and strains due to fear of negative consequences from their supervisors or management. This study aims to better understand the factors that influence this behavior and to develop a scale to measure it.

Design/methodology/approach

The study used an online survey of 200 employees to investigate the factors influencing defensive biasing behavior. The researchers developed a scale for defensive biasing with the help of subject matter experts and derived possible factors from the literature. Participants were presented with a hypothetical scenario in which they imagined a work stress survey in their organization and were asked to answer related items. The data were analyzed using regression analysis.

Findings

The study found that defensive biasing behavior was significantly predicted by perceived anonymity and neuroticism. Participants who felt less anonymous and had higher levels of neuroticism were more likely to engage in defensive biasing. Job insecurity and trust in supervisors were not found to be significant predictors of defensive biasing.

Originality/value

This study contributes to the literature on work stress surveys by developing a scale for defensive biasing and investigating the factors that influence this behavior. The study highlights the importance of making the survey process more transparent to reduce defensive biasing and obtain trustworthy results.

Citation

Greulich, B., König, C.J. and Mohr, R. (2024), "When employees understate their stress: defensive biasing in work stress surveys", International Journal of Workplace Health Management, Vol. 17 No. 2, pp. 102-122. https://doi.org/10.1108/IJWHM-06-2023-0072

Response bias, a tendency where individuals respond in particular ways that may not reflect their genuine attitudes, beliefs or behaviors, can significantly affect the validity of survey results, potentially leading to inaccurate conclusions (Navarro-González et al., 2016; Paulhus, 1991; Rammstedt et al., 2010). This bias is particularly critical in work stress assessment, typically measured by self-report. According to a study on response behavior in work stress surveys (Greulich et al., 2021), some employees may downplay their stressful work conditions to avoid potential negative repercussions from superiors. This self-protection strategy, termed “defensive biasing,” may lead to the underreporting of work stress. To gain a deeper understanding of defensive biasing, which has so far only been qualitatively examined in one study (Greulich et al., 2021), we have developed a scale for quantitative examination. Moreover, we propose theoretical arguments regarding factors influencing defensive biasing, incorporating the social identity model of deindividuation effects (Reicher et al., 1995). This study thereby aims to enrich the understanding of motivational processes underlying response behavior in work stress surveys.

Theoretical background

Changes in working life regarding the use of complex technology, globalization and high competitive pressure lead to a high workload for many employees, and these stressful working conditions result in psychological strain at the workplace that affects both psychological well-being and physical health (Niedhammer et al., 2021). Outcomes of a high workload include job dissatisfaction, turnover intentions and counterproductive work behavior (e.g. Fida et al., 2015). In the long run, psychological strain at work might result in physical and mental illnesses such as migraine, cardiovascular diseases, sleeping disorders, depression and burnout (e.g. Taouk et al., 2020).

Sickness caused by work stressors is associated with considerable suffering for the person concerned, but also with economic problems for the organization due to sick leave and costs for society (Han et al., 2019). To mitigate these negative consequences, organizations need to know the (potential) sources of work stress. Suitable interventions could then be taken to minimize sources of stress and thus maintain the well-being and health of the employee. This makes it important for organizations to assess working conditions that result in health problems for their employees. In addition to concerns about organizational and employee well-being, legislation also drives the implementation of work stress surveys. The German Occupational Safety and Health Act specifically requires an assessment of work-related stress, underscoring the legal impetus for such survey.

Measuring work stress

Various models play a crucial role in understanding the mechanisms of work-related stress in the field of occupational health, with regards to stressors, resources and strain. Karasek’s (1979) job demand control (JDC) model emphasizes the correlation between job demands and control in defining employee well-being. Its enriched version also integrates social support. Building on the previous discussion, Siegrist’s (1996) effort-reward imbalance (ERI) model presents a crucial perspective by highlighting the disparity between employees' efforts and the corresponding rewards. This imbalance causes significant stress related to work and can lead to negative health consequences. Additionally, Demerouti et al.'s (2001) job demand resource (JDR) model classifies job attributes into resources and demands, offering a comprehensive framework for understanding the nature of work. It underscores that an imbalance between these elements can lead to negative outcomes such as job dissatisfaction and health issues.

The primary method for evaluating work stressors, resources and strain is self-report surveys, where participants rate the intensity of specific work stressors, resources and strain (e.g. Kristensen et al., 2005; Schulte et al., 2021; Semmer et al., 1999). This method is efficient, allowing for substantial data collection with minimal time and resource investment. Furthermore, employees, being intimately familiar with their job demands, serve as expert informants. Moreover, subjective perceptions of work situations are often more suited for strain reactions compared to objective indicators (Perrewé and Zellars, 1999; Spector and Jex, 1998).

Nevertheless, self-report surveys suffer from some disadvantages because they might be biased due to factors such as negative affectivity, acquiescence and situational contexts. Evidence for these biases can be found in various research fields that use self-reports (e.g. Debus et al., 2015; Morgeson and Campion, 1997; Schmit et al., 1995). Among these various biases, social desirability proves to be particularly noteworthy since it compels respondents to manipulate their answers to gain more favor or appear more acceptable (van de Mortel, 2008). Results of Greulich et al. (2021) regarding the response behavior in work stress surveys show that participants in work stress surveys engage in manipulative answering behaviors, which can be considered a novel form of response bias and might be classified as a subtype of social desirability bias. Participants reported that they deliberately understate or trivialize their ratings of stressors or strain when they fear negative consequences from their supervisors to protect themselves against material and immaterial loss (e.g. ostracism or job loss; Greulich et al., 2021).

Numerous studies have demonstrated that fear-based motives can result in employee silence, where individuals intentionally withhold work-related information or criticism (e.g. Pinder and Harlos, 2001; Van Dyne et al., 2003). The literature has established the concept of “silence” (e.g. Hao et al., 2022; Morrison and Milliken, 2000), defined as the withholding of work-related information due to underlying motives (Brinsfield et al., 2009). Brinsfield et al. (2009) investigated the specific fears that may cause employee silence and found that fear of negative consequences was a common motivation. This form of silence, referred to as “defensive silence,” is for self-protection and may arise from concerns of job loss, appearing incompetent, being labeled a complainer, or causing conflict in the workplace (Jahanzeb et al., 2018; Milliken et al., 2003). De Clercq et al. (2020) demonstrated that belief in verbally abusive leaders can prompt employees to engage in defensive silence, which can ultimately reduce the risk of negative performance evaluations. This highlights the significance of defensive silence as a mechanism used by employees to avoid negative performance evaluations by abusive leaders.

Although the fear of potential negative consequences has been found to make employees remain silent (Kish-Gephart et al., 2009; Morrison and Milliken, 2000) and could be considered a special case of impression management, this fear hat not been explored in depth in the context of work stress surveys. When people are forced to make a statement by participating in a questionnaire, they have to resort to another kind of self-protection: trivializing stressors and strains and presenting resources as more pronounced. The concealment of stressful working conditions to protect oneself from negative consequences can thus bias stress surveys. Based on the construct of “defensive silence” (see, e.g. Jahanzeb et al., 2018; Milliken et al., 2003; Van Dyne et al., 2003), this intentional and proactive response behavior in work stress surveys can be called “defensive biasing.” This defensive biasing likely distorts the quality of the obtained data and leads organizations to make ineffective management and human resources decisions. To improve the validity of work stress surveys, it is crucial to better understand the phenomenon of defensive biasing.

The qualitative results of the study by Greulich et al. (2021) and the literature on survey response behavior and defensive silence offer arguments which factors likely influence defensive biasing. Factors that lead to voice and silence are often divided into motivators and inhibitors, respectively (see Morrison, 2014). These factors can in turn be divided into situational circumstances, such as characteristics of the relationship with the supervisor and also into individual dispositions, such as certain personality traits. Furthermore, beliefs about personal consequences and sensitivity to anonymity seem to have a significant influence on the response behavior (Mueller et al., 2014).

Anonymity as a predictor for defensive biasing

The employees interviewed for the qualitative study of Greulich et al. (2021) emphasized that the anonymity of the survey plays a major role in defensive biasing. They mentioned a fear of being recognized despite assured anonymity, for example, based on demographic data. They kept this possibility in mind when answering questions for employee work stress surveys. This suggests that the perceived anonymity of the survey influences the fear-related response behavior of survey participants, which is also consistent with the literature on organizational surveys in general that also assigns great importance to the perceptions of anonymity (Rogelberg et al., 2006). This sensitivity to anonymity is particularly relevant in the German context, where privacy concerns are deeply rooted (Oghazi et al., 2020). The heightened emphasis on privacy in Germany, backed by strict regulations (Custers et al., 2018), is likely to exacerbate German employees' fears of being identified, thereby influencing their response behavior in surveys. Understanding this heightened privacy concern is critical to interpreting the results from such a workforce.

There are two forms of anonymity: literal anonymity and perceived anonymity (Dunnette and Heneman, 1956). It is quite possible that a survey objectively preserves anonymity (i.e. has literal anonymity), but respondents still do not perceive it as such. Perceived anonymity has already been established as a relevant factor for self-reports (e.g. Saari and Scherbaum, 2011). A meta-analysis summarized that different objective implementation strategies of anonymity yielded only small effects on the response behavior, which speaks for a rather subjective impression of anonymity (Singer et al., 1995).

The assumption of the influence of perceived anonymity on specific fear-based response behavior is plausible against the background of the so-called social identity model of deindividuation effects (Reicher et al., 1995), which originally stems from the research about computer-mediated communication, a field of social psychology. Social identity theory (Tajfel and Turner, 1986) and self-categorization theory (Turner et al., 1987) form the basis of this model, postulating that a low level of identifiability leads individuals to orient themselves to their values and norms since they do not have to fear sanctions. In contrast, as identifiability increases, individuals become more oriented towards the norms and values of the outside group. The social identity model of deindividuation can also be applied to anonymous employee surveys (Chudziak and Maus, 2008) and may better explain the relationship between subjectively low perceived anonymity and defensive biasing in an organizational context than the classic social desirability models. In the context of work stress surveys, high perceived identifiability towards the outside group (i.e. the supervisors or management) leads to increased adoption of the norms and values of this group. To avoid sanctions by more powerful supervisors or management, employees refrain from response behavior that can lead to punishment. Therefore, answers may be intentionally biased due to fear as doubts about anonymity grow. These theoretical considerations and findings lead to the assumption that the subjectively low perceived anonymity of the survey reinforces the defensive biasing of answers. In the context of work stress surveys, being sure that employees are not identifiable seems crucial for honest evaluations (so employees are not engaging in defensive biasing). In the case of stressful working conditions, employees may perceive that a true rating could cause negative actions from their supervisors (e.g. ostracism). By trivializing possible stressors or strains and presenting resources more positively, employees can try to protect themselves from negative consequences. This leads to the following hypothesis:

H1.

Perceived anonymity is negatively related to defensive biasing.

Neuroticism

In the realm of silence research, personality traits are frequently examined as potential motivators or inhibitors of employee voice and silence (e.g. Hao et al., 2022). Among the Big Five personality traits (Costa and McCrae, 1992), neuroticism is one of the most widely studied and frequently examined factors in personality psychology (Tackett and Lahey, 2017). Furthermore, this trait is also commonly examined in the context of stress research (e.g. Roloff et al., 2022). Neurotic individuals tend to experience negative emotions such as anxiety or anger more frequently and are more sensitive to stressors than their emotionally stable counterparts (Costa and McCrae, 1992). They are also more prone to focusing on negative stimuli in the environment, which contributes to their perception of situations as more threatening (Nettle, 2006). Consequently, their elevated fear and insecurity can lead to reduced expression of opinions, concerns and problems in the workplace (LePine and Van Dyne, 2001). This finding is supported by Li and Xu (2020), who demonstrated that employees with higher levels of neuroticism exhibit lower levels of expressive behavior at work, which is mediated by emotional exhaustion. Brinsfield’s (2013) research also indicated that high levels of employee neuroticism are associated with increased defensive silence. Additionally, De Clercq et al. (2020) suggest that the effectiveness of defensive silence may be particularly pronounced among individuals with high levels of neuroticism. This argument also extends to defensive biasing in work stress surveys, as those who are more neurotic may fear negative consequences more than their emotionally stable counterparts when asked to report work stressors and strain. Therefore, we expect that there is a relationship between neuroticism and defensive biasing:

H2.

Neuroticism is positively related to defensive biasing.

Job insecurity

Job insecurity reflects employees’ worry about losing their present job and has strong psychological impact on employees (Lee et al., 2018). Furthermore, job insecurity decreases employee voice and increases employee silence (Breevaart et al., 2020). There is also evidence that job insecurity also affects respondents' answering behavior. In interviews, employees expressed job loss as a feared negative consequence of answering work stress surveys truthfully (Greulich et al., 2021). These interviewees also mentioned that staff reductions led to a tense atmosphere and fear in the company which was reflected in a previous employee survey in the form of trivialization of answers (Greulich et al., 2021). One way of coping with job insecurity is to ensure that others see the value they bring to their organization by engaging in impression management. Empirical support for this argument comes from a study by Huang et al. (2013), who found job insecurity and impression management to be positively related. Furthermore, job-insecure people might also increase their work efforts and their performance to protect their jobs (Staufenbiel and König, 2010). Defensive biasing in surveys could be seen as a further protection strategy against potential job loss because it helps make the impression that the work situation is still good and that the employee is still able to maintain high performance and not be impaired by stress or emotional exhaustion. In the context of work stress surveys, job loss fear could increase the motivation for defensive biasing. For example, if the company’s economic situation is poor and the probability of losing one’s job is thus greater, it is conceivable that employees in work stress surveys are more likely to present stressors and strains in a mitigated form because they want to avoid behavior that could jeopardize their job. The mentioned empirical findings and theoretical considerations lead to the arguments that employees will increasingly engage in defensive biasing when having doubts regarding the stability of their jobs. More formally, we can state the following hypothesis:

H3.

Job insecurity is positively related to defensive biasing.

Trust in supervisor

A fourth main predictor of defensive biasing should be trust in the supervisor. Trust can be defined as “the willingness of a party to be vulnerable to the actions of another party based on the expectation that the other will perform a particular action important to the trustor, irrespective of the ability to monitor or control that other party” (Mayer et al., 1995, p. 712). The amount of trust employees have in their supervisors appears to be particularly important for how they respond to fear (Detert and Burris, 2007). The influence of trust on employees’ voice and silence behavior has already been supported by empirical findings: A significant negative correlation was found between trust and employees' fear-based silence (Brinsfield et al., 2009) and the less trust there was in the supervisor, the more likely employees were to remain silent for fear of negative consequences (Dedahanov et al., 2015). Thus, a good relationship of trust between supervisor and employee can lead to a reduction in the fear of negative consequences when expressing opinions or criticism because the supervisor takes the employees’ problems seriously and is prepared to listen to their suggestions. The goodwill expressed by the supervisor towards their employees and the resulting trust can have an encouraging effect not only on voicing criticism, for example of work processes, but also on communicating personal burdens, as is also the case in a work stress survey. Therefore, if employees trust that their supervisors will take appropriate actions when getting negative feedback, this might also generalize to employees’ strategies while filling out work stress surveys. We can thus propose the following hypothesis:

H4.

Trust in supervisor is negatively related to defensive basing.

Method

Data collection

The data for this study was gathered through an online survey created and administered via the Unipark survey platform. Participants were directed to a unique link for access to the survey. Utilizing self-reporting in this initial study allows us to capture employees' perceptions and experiences directly, thus building a fundamental comprehension of defensive biasing. This approach serves as a pivotal starting point in our endeavor to unravel the intricacies of this phenomenon. The survey was designed to be user-friendly and minimize participant burden, with completion achievable in approximately fifteen minutes. The online survey’s format provided participants with the flexibility necessary to pause and resume the survey as required. The recruitment of participants utilized the snowball sampling method. This approach was favored for its efficiency in reaching a wide-ranging group of respondents through existing social networks. Participation in the study was completely voluntary, without any compensation being provided, to ensure that responses were driven by genuine interest or willingness to contribute to the research. Strict anonymity was also guaranteed to uphold participant confidentiality and encourage honest responses. Furthermore, participants were informed that the survey was conducted exclusively for scientific purposes, and it was not obligatory for them to disclose their employer’s name.

Sample

The sample consisted of 200 German employees, with a majority of 61% being female. These participants were distributed across a range of industries. Specifically, 31% of the participants were engaged in service sectors. About 25% were from various manufacturing industries. The health sector comprised 17% of the sample. The education sector accounted for 7%, whereas 6% were involved in the social sector. The remaining 14% of participants were categorized as “other”, which included a range of different industry sections. Participants' ages were categorized: 14% under 24, 31% between 25 and 34, 14% between 35 and 44, 22% between 45 and 54 and 19% over 55. All were non-self-employed, with 24% holding management positions and 83% having permanent contracts. The average professional experience was 15.8 years (standard deviation (SD) = 12.8), with 10.7 years in their current job (SD = 10.5). About 12% had prior experience with work stress surveys.

Questionnaire

Scenario

First, participants were asked to imagine themselves involved in a work stress survey being distributed in their organization. They were presented with the following text (translated from German): “Imagine being invited to participate in a survey within your organization. Carefully review the instructions provided for the Mental Stress Risk Assessment. Imagine this survey being administered in your organization in exactly the same way, including the questions provided. We encourage you to fully immerse yourself in this scenario. Approach the questionnaire section as if you were responding to actual conditions in your organization.” Next, the scenario started and the participants were presented with the following text: “Dear Employee, thank you for participating in this survey! The purpose of this survey is to assess the psychosocial work situation in your company to identify possible problem areas and to derive measures to improve your working conditions. Your data will be collected anonymously so that no conclusions can be drawn about individual responses. The data will be analyzed by an external person and will not remain within the company. The results of the survey will be shared with your company only in a statistically aggregated form. Participation is voluntary. Below are several statements. Please rate the extent to which these statements apply to you. Please put a cross next to a statement, even if you are not sure. Then select the alternative answer that best describes your situation. Note about privacy: Only groups with at least 5 participants will be analyzed. Smaller groups will be combined. This ensures that your data remains anonymous” (translated from German). In the scenario, we utilized items from the Copenhagen Psychosocial Questionnaire (COPSOQ), which was originally developed by Kristensen et al. (2005). Specifically, we employed the German adaptation by Nübling et al. (2005), which adds additional topics to the initial instrument while also maintaining its reliability and validity in German-speaking contexts. Participants first filled in demographic data before answering selected items from 25 scales of the German version of the COPSOQ, which assesses stressors, resources and strain. The COPSOQ items were chosen in consultation with one of its authors (M. Nübling, personal communication, 07.05.2018), focusing on those most likely to cause fear of consequences if answered honestly with low perceived anonymity. Table 1 displays the ten selected items. After completing the work stress survey section, respondents qualitatively described their response behavior and reported their experience with work stress questionnaires.

Defensive biasing

No existing instrument was found in the literature search that specifically measures defensive biasing, necessitating the development of a new instrument. To capture defensive biasing, a direct measure was employed, wherein participants were asked whether they would respond to work stress surveys in a deliberately understated manner given the described situation. This approach utilized behavioral intention as a proxy for actual or future behavior. To enable this inference, it is crucial to provide a detailed description of the survey context, including its implementation (Ajzen and Fishbein, 1980).

Drawing upon qualitative interviews conducted by Greulich et al. (2021), interviews with eight subject matter experts and relevant empirical studies on voice and silence research, four items were developed to describe potential reasons for defensive biasing (see Table 2). Items 1 and 4 address the fear of material losses. By expressing job-related grievances in a mitigated manner and providing more positive responses regarding their supervisor, individuals may seek to avoid negative consequences such as job loss, conflicts with their supervisor and a deterioration of the work climate. On the other hand, items 2 and 3 pertain to the fear of intangible losses if participants were to answer the questions truthfully. To avoid appearing weak or incompetent, individuals may downplay personal feelings, worries and stressors. Thus, the scale encompasses behaviors aimed at mitigating both material and immaterial losses. Participants rated the items on a five-point scale ranging from 1 (strongly disagree) to 5 (strongly agree). The wording of the items was designed such that agreement indicates a stronger tendency towards defensive biasing.

Perceived anonymity

To assess perceived anonymity, the Perceived Anonymity Scale (PANON) (Whelan and Thompson, 2009) was translated into German by a professional translator and then back-translated into English by another translator. During the translation process, both translators deemed the item “My responses will blend in with the responses of other people” as untranslatable into German, leading to its exclusion. Consequently, the scale consisted of five remaining items, see Table 3. Participants were instructed to consider the preceding section of the work stress survey (i.e. the scenario part) when responding to these items. A five-point scale ranging from 1 (strongly disagree) to 5 (strongly agree) was used for participants to rate their agreement, with higher scores indicating greater perceived anonymity.

Neuroticism

Neuroticism was measured using the 8-item scale from the German version of the Big Five Inventory (BFI) (John et al., 1991; Rammstedt and John, 2005), see Table 4. The items were answered on a five-point scale from 1 (strongly disagree) to 5 (strongly agree).

Job insecurity

Job insecurity was measured using the 4-item scale of Staufenbiel and König (2010), coded so that high values indicated high levels of job insecurity (see Table 5).

Trust in supervisor

Trust in supervisor was assessed with the 9-item scale from the German version of the Workplace Trust Survey (Ferres and Travaglione, 2003; Lehmann-Willenbrock and Kauffeld, 2010), with agreement indicating high trust. A sample item is “My supervisor treats personal conversations confidentially.”

Results

Factor analysis

A principal axis factor analysis was performed on the four defensive bias items, with oblique rotation. The Kaiser–Meyer–Olkin (KMO) measure confirmed sampling adequacy, KMO = 0.8 (Hutcheson and Sofroniou, 1999), and all individual item KMO values exceeded 0.74, surpassing the acceptable limit of 0.5 (Field, 2013). Initial analysis for the defensive bias scale produced eigenvalues for each factor. One factor had an eigenvalue above Kaiser’s criterion of 1, explaining 72% of the variance supporting the retention of one factor. Table 2 presents rotated factor loadings, with items clustering on one factor, indicating defensive biasing. The defensive biasing scale demonstrated high reliability, with Cronbach’s Alpha at 0.84 and McDonald’s Omega at 0.87.

Experience with work stress questionnaires

A minority of participants (12%) had prior experience with work stress surveys. Among them, 26% reported positive experiences and changes in their company, while 30% reported negative experiences and no changes, particularly at the management level. The other participants did not provide any information.

Qualitative data

In the qualitative part of the study, participants were asked to reflect on their response approach to the work stress survey with the question: “Considering the context of this survey, how did you approach the answering of these questions?” The responses were subjected to a thematic content analysis, allowing for a systematic examination and interpretation of the data. The findings from this qualitative analysis are quantitatively represented by the frequency of recurring themes identified in participants' responses. 58% of respondents provided qualitative descriptions of their response behavior. Among them, 21% could realistically imagine themselves in the situation and 12% did not trust the survey’s anonymity and feared negative consequences, with one participant stating he/she answered “cautiously” due to mistrust in the word “anonymously.” Seven percent followed their feelings when answering, while four participants mentioned feeling stressed and worrying about it. A fifth (21%) said they answered the questions truthfully, with one person writing that they felt the questionnaire was anonymous. Another person added that although they felt inferences could be made about the individual, this had not mattered to them because of their good relationship with their supervisor.

Test of hypotheses

A multiple linear regression analysis was conducted to examine the prediction of defensive biasing from perceived anonymity, neuroticism, job insecurity and trust in supervisor (see Table 6 for mean values, SDs and Cronbach’s alpha). The results (Table 7) showed a significant association between perceived anonymity and neuroticism. Perceived anonymity negatively predicted defensive biasing (β = −0.18, p < 0.01), confirming Hypothesis 1. Neuroticism positively predicted defensive biasing (β = 0.17, p < 0.05), supporting Hypothesis 2. However, job insecurity (β = 0.03, p = 0.73) and trust in supervisor (β = 0.05, p = 0.49) were not significant predictors, failing to confirm Hypotheses 3 and 4.

The analyses revealed noteworthy findings about defensive biasing beyond our initial hypotheses, see Table 8. Younger employees were found to have a greater tendency towards defensive biasing, as indicated by the significant negative correlation observed between age and defensive biasing (r = −0.18, p < 0.05). Additionally, work experience (r = −0.20, p < 0.01) and employment in current job (r = −0.19, p < 0.01) were both found to have negative associations with defensive biasing. The results suggest that individuals with less work experience and shorter job tenures tend to underreport or manipulate their responses in work stress surveys. Furthermore, a negative correlation was observed between management responsibility and defensive biasing (r = −0.18, p < 0.05), indicating that individuals with such responsibilities are more inclined to underreport. There were no significant correlations between the COPSOQ items used and defensive biasing (see Table 9).

Discussion

This study aimed to quantitatively investigate a new form of biasing specifically in work stress self-reports: defensive biasing. This phenomenon occurs when respondents bias their answers in employee surveys for self-protection. To investigate this phenomenon, we first developed a scale to measure the construct of defensive biasing. Second, we made a first attempt to link other concepts to defensive biasing. We empirically demonstrated that perceived anonymity and neuroticism significantly predicted defensive biasing, whereas job insecurity and trust in superiors could not be confirmed as predictors.

Initially, the process of developing a psychometric instrument for measuring defensive biasing resulted in a four-item scale that can be used to assess deliberate understatement or trivialization of employee ratings of stressors or strain. The use of this scale will help to understand employees’ motives to bias their responses in work stress surveys. A factor analysis confirmed that the new scale works well as a univariate measure of defensive biasing. Furthermore, the scale demonstrated good reliability.

The results of this study show that low perceived anonymity leads to an increased response behavior of defensive biasing. This result also confirms the arguments of Mueller et al. (2014) that the perception of anonymity plays a decisive role in answering items in a survey. Furthermore, the social identity model of deindividuation (Reicher et al., 1995) provides a plausible explanation for this connection: In cases of low perceived anonymity of the work stress survey (i.e. high identifiability of the employees by supervisors or management), employees’ response behavior is adjusted to the norms and values of supervisors and management for fear of sanctions. In the context of a work stress survey, the behavior that is avoided is truthfully rating of bad working conditions, because this would mean negative feedback for the foreign group of supervisors or management. The qualitative comments of the employees collected in this study also support the relevance of perceived anonymity. As an answer to the question about their response behavior, participants already noted concerns about assured anonymity (e.g. “I don’t trust the phrase anonymously very much, so I responded carefully.”). Furthermore, it was stated that “[…] anonymity cannot be guaranteed at all in smaller organizations.” As we investigated the significant impact of perceived anonymity on survey responses, it is important to situate our results within the unique framework of data protection in Germany. The strong focus on privacy, which is deeply ingrained in the consciousness of the German public and strengthened by comprehensive regulations (Custers et al., 2018), could impact the response behavior that we observed during our study. With their heightened awareness of data privacy concerns, German employees may approach participation in surveys with particular caution, which could have an impact on the accuracy and validity of their responses. This distinctive aspect of the German context must be carefully considered when interpreting our results. It could significantly affect participants' views on anonymity and, as a result, their willingness to provide honest feedback.

The significant main effect of neuroticism on defensive biasing indicates the important role of personality traits in response behavior. Neurotic individuals appear to do more defensive biasing in surveys. It could be that these employees are in principle less likely to express opinions, concerns, or criticism, regardless of whether an inference can be personally drawn about them or not. A conceivable explanation for this lies in the characteristics of neurotic individuals: A high level of neuroticism is often associated with insecurity and less self-confidence (e.g. Abdellaoui et al., 2019), which may make these individuals more cautious with their statements, triggering them to formulate statements in a weakened manner. Moreover, they tend to perceive situations more threateningly than they actually are (Schneider, 2004). Possible negative consequences could be interpreted by them as much more likely. To protect themselves from this, they are left with only defensive biasing.

Although the interviewees in the qualitative study of Greulich et al. (2021) had explicitly spoken of the relevance of job insecurity to the biasing response behavior, this relationship could not be shown quantitatively in this study. In fact, the type of employment contract also had no significant influence. This may be due to the strong protection for employees in Germany (where the data was collected) due to the German employment protection legislation, which makes firing employees rather complicated. This also fits with the rather low mean value of job insecurity (i.e. 2.19). Furthermore, unemployment rates in Germany are not very high (5.7%, Bundesagentur für Arbeit, 2021). In countries with different legal regulations and higher unemployment rates, it might be that job insecurity affects the occurrence of defensive biasing.

The assumption that trust in supervisor influences the likelihood of defensive biasing could not be confirmed by our data. We had argued that a trustful relationship between employees and supervisors should increase the likelihood of employees admitting to being overtaxed and exhausted, but this does not seem to be the case. A possible explanation is that employees often do not know who exactly is meant by the term supervisor in surveys. Especially in organizations with many hierarchical levels, several persons could be considered (cf., Hackman and Wageman, 2004; Therkelsen and Fiebich, 2004). On the one hand, the participants could think of their team leader, who would allocate tasks to them. On the other hand, they could also think of a higher level of superiors who are familiar with personnel decisions. This ambiguity becomes significant when considering that an employee may have a strong trust relationship with his or her immediate team leader, but not necessarily the same level of trust in the organization’s chief executive officer (CEO). For example, employees’ positive experiences with their direct team leader might lead them to respond positively on the scale measuring trust in supervisor. However, this does not necessarily reflect their trust in senior management or the CEO, who may be perceived as more distant and less familiar. The lack of trust in senior management, such as the CEO, despite a good relationship with a direct supervisor, could be a critical factor influencing defensive biasing. Employees may feel comfortable sharing concerns with their direct supervisor, but might remain apprehensive about how their responses might be perceived by senior management. This discrepancy in trust levels may therefore contribute to the observed non-correlation between trust in supervisors as a general concept and defensive bias. It highlights the need for a more nuanced understanding of the term “supervisor” in survey contexts and its impact on response behavior. Future research should explore these differential trust relationships within organizational hierarchies to better understand their influence on employees' willingness to disclose genuine experiences of work-related stress.

Scores of defensive biasing were also significantly correlated with age and work experience, employment in the current job and management responsibility. These results suggest that defensive biasing might be a phenomenon that tends to affect younger employees with less work experience and less management responsibility.

In the course of our investigation, we have identified defensive biasing as a critical element in work stress research. Our findings demonstrate the substantial impact this bias has on interpreting survey data, particularly in the context of meta-analytical studies that draw on primary data from work stress surveys. The survey responses may contain defensive biasing, particularly in studies that examine the dynamic between job insecurity, trust in supervisors and employee health outcomes (e.g. Inceoglu et al., 2018; Rönnblad et al., 2019). This requires a reassessment of commonly accepted correlations. If left unaddressed, defensive biasing can distort interpretations, conceivably obscuring the genuine nature of the relationships. This raises important considerations for future research methodologies. The identification and mitigation of defensive biasing’s influence is pivotal in ensuring the integrity of data interpretation.

Limitations

As with all studies, this study has its limitations. First, the method relies on participants' capacity to imagine a hypothetical scenario of a work stress survey in their organization, which may impact its effectiveness. Participants were asked to envision themselves in a hypothetical situation, which may have led to a lack of authenticity in their responses. As individuals may behave differently when faced with real-life scenarios, the validity and generalizability of the findings could be called into question. Additionally, in a real-world setting, participants have the option to decline participation. However, they may also feel compelled to participate due to perceived pressure from management, as low participation rates could be negatively viewed. Particularly neurotic individuals may feel obligated to participate, thus potentially biasing their responses and presenting their working conditions in a more positive light than reality. In such cases, participants may feel they have no alternative but to bias their responses, further complicating the interpretation of the data. Despite these limitations, qualitative feedback suggested that a substantial proportion of participants, specifically 40.9%, were able to effectively simulate this scenario, with comments such as “I treated the question as if my employer were asking it.” This lends some validity to the employed method. Second, all data were gathered exclusively through self-report in a single questionnaire, which could introduce potential common method variance (CMV), thus limiting the interpretability of the results. Nonetheless, the nature of the study made self-reporting indispensable as it was the only way for participants to express their perceptions of anonymity and their response behavior. Moreover, it is important to consider the impact of method effects, such as CMV, on the results of our study. According to Lance et al. (2010), the potential bias introduced by CMV may be offset by the effect of unreliability of measurement instruments. This implies that while CMV may be a concern in survey-based research, its influence is likely to be offset by the inherent imprecision of the measurement instruments used. Nonetheless, future research could consider supplementing self-reports with data collected from other sources. Furthermore, the sample size of 200 participants is relatively small, which may limit the generalizability of our findings. A larger, more representative sample would help capture the diversity and variability in responses and characteristics. Additionally, although snowball sampling can be beneficial for quickly accessing a specific respondent group, it does not produce representative samples. The method often results in participants recruiting individuals from their social networks, potentially leading to homogeneity within the sample than in the population. It is important to acknowledge these limitations when interpreting the results of this study, and future research should strive for a larger and more representative sample to improve the generalizability of the findings. Nevertheless, this study provides valuable insights into the defensive bias tendencies and establishes the groundwork for future research in this field.

Future research and implications

This study highlights the use of defensive biasing by employees in work stress surveys as a means of self-protection. Future research should investigate the extent to which defensive biasing skews relationships between constructs assessed in organizational surveys. Additionally, further exploration is needed to understand the subjective perception of anonymity and identify potential influencing factors. The newly developed scale for measuring defensive biasing offers significant advantages for future research in this area. Its practical length and high reliability make it a valuable tool. Unlike indirect deductions of defensive biasing from questionnaire responses, which may involve interpretation and yield less reliable findings, this scale directly measures behavior. Thus, future research should focus on validating the scale and employing it in further investigations of defensive biasing.

Furthermore, this study used self-reports to explore defensive biasing. Although self-reporting is direct and practical and thus provided a reasonable starting point to explore this new phenomenon, it is crucial to note that this method only represents one approach of a wider methodological spectrum, and future research should expand its methodological scope beyond self-reports and adopt a more comprehensive approach, such as experimental designs. This expansion is crucial, as it enables the exploration of various factors influencing defensive biasing, including anonymity. For instance, experimental designs can vary the degree of anonymity and facilitate group comparisons, shedding light on nuanced aspects of response dynamics. This necessity for methodological diversification is exemplified in research on applicant faking in employment processes, where the combination of self-reports and experimental methods provides a holistic view (e.g. Melchers et al., 2020). Building upon our initial findings with a multifaceted approach will contribute to a more thorough comprehension of response dynamics in workplace stress surveys.

This study reveals the intricacy of response behavior in work stress surveys. Although our research concentrates on defensive biasing, it is crucial to acknowledge that the opposite inclination - the exaggeration of workload - could also contribute to survey responses. This phenomenon may arise when respondents see a potential benefit in exaggerating their work-related stress, such as affecting the allocation of more resources or personnel because of reported high quantitative demands. Indeed, Greulich et al. (2021) found that respondents may also overstate stress levels for perceived personal or organizational advantages. This suggests that in understanding response behavior in stress surveys, it is important to consider both defensive biasing due to social desirability and the possibility of exaggeration as another form of response bias. Such behavior could be seen as a strategic misuse of surveys, reflecting a complex interplay of motivations that affect response patterns in workplace stress evaluations. These dual tendencies of underreporting and overreporting stress emphasize the need for nuanced interpretation of survey data. They highlight the importance of considering the possibility of both underreporting and overreporting of stress levels in organizational decision-making. Future research should further investigate these contrasting biases to develop more dependable methods for precisely assessing workplace stress. This study also provides practical implications for conducting employee surveys. Transparent communication regarding the purpose and process of surveys can alleviate employees' concerns and fears (see also Schweiger and DeNisi, 1991). Explaining to employees why specific questions are being asked can reduce the fear of negative consequences associated with describing work stress. Additionally, when surveys are conducted by external companies, employees may perceive them as fair and unbiased, benefiting both supervisors and management. Another practical implication emerges from qualitative responses regarding past experiences with work stress surveys, where participants often criticized the lack of resulting changes. This underscores the importance of meeting employees' expectations and implementing changes based on survey findings. Failure to satisfy these expectations may negatively impact future surveys of a similar nature. For instance, employees might resort to defensive biasing or, conversely, exaggerate the severity of their work conditions to instigate change or express frustration. Therefore, special attention should be given to implementing meaningful changes following survey administration.

Conclusion

The purpose of the current study was to take the first step towards quantitatively investigating the occurrence of defensive biasing and its cause-effect relationships. The findings suggest that perceived anonymity and high neuroticism were associated with an increased occurrence of defensive biasing. Although the study offers preliminary insights into this important phenomenon of employees potentially protecting themselves by not honestly describing their work stress experiences, further research is warranted to validate and expand upon these findings.

Table 1

The items used in this study (from the Copenhagen Psychosocial Questionnaire, Nübling et al., 2005) and the scales they belong to

Item	Scale
Do you feel that the work you do is important?	Meaning of work
Are you proud of being part of this company?	Commitment to workplace
To what extent would you say that your immediate superior gives high priority to job satisfaction?	Quality of leadership 1
To what extent would you say that your immediate superior is good at work planning?	Quality of leadership 2
How often is your immediate superior willing to listen to your problems at work, if needed?	Support at work
How often do you feel unjustly criticized, bullied or shown up in front of others by your colleagues and your superior?	Unfair treatment
Can the employees trust the information that comes from the management?	Trust and justice
Is your work recognized and appreciated by the management?	Recognition
In the past 12 months, how often have you thought about changing your job?	Intention to leave profession/job
How often do you feel emotionally exhausted?	Burnout symptoms

Source(s): Authors’ own work

Table 2

Defensive biasing scale

	Items	Factor loadings
If a risk assessment of psychological stress was conducted in my company in the manner and with the given questions mentioned above, I would …
1	… present work-related grievances in an attenuated way	0.89
2	… slightly downplay circumstances that burden me	0.89
3	… trivialize statements regarding my personal condition or worries	0.75
4	… answer questions regarding my superiors or colleagues a bit more positive than reality might reflect them	0.64

Note(s): The original German items are available from the authors

Source(s): Authors’ own work

Table 3

Perceived anonymity scale (PANON) scale

	Items
1	I feel my responses are indistinguishable from the responses of others who have taken this survey
2	It would be impossible to trace my responses to this survey back to me
3	I feel that my responses are unidentifiable from the responses of others
4	I feel certain that this survey is anonymous
5	If someone saw my responses, they would never know who it was who filled out the survey

Source(s): Authors’ own work; adapted from Whelan and Thompson (2009)

Table 4

Neuroticism scale

	Items
1	Werde leicht deprimiert, niedergeschlagen
2	Bin entspannt, lasse mich durch Stress nicht aus der Ruhe bringen
3	Reagiere leicht angespannt
4	Mache mir viele Sorgen
5	Bin emotional ausgeglichen, nicht leicht aus der Fassung zu bringen
6	Kann launisch sein, habe schwankende Stimmungen
7	Bleibe ruhig, selbst in Stresssituationen
8	Werde leicht nervös und unsicher

Source(s): Authors’ own work; German version from Rammstedt and John (2005)

Table 5

Job insecurity scale taken from Staufenbiel and König (2010)

	Items
1	My job is secure
2	In my opinion, I will keep my job in the near future
3	In my opinion, I will be employed for a long time in my present job
4	My workplace is secure in every respect

Source(s): Authors’ own work

Table 6

Means, standard deviations and Cronbach’s alpha

	Variables	M	SD	95% CI	Cronbach’s α
1	Defensive Biasing	2.42	0.92	[2.30, 2.56]	0.87
2	Perceived Anonymity	3.35	0.80	[3.23, 3,47]	0.79
3	Neuroticism	2.64	0.69	[2.55, 2.74]	0.90
4	Job Insecurity	2.19	1.10	[2.03, 2.34]	0.88
5	Trust in Supervisor	3.97	0.80	[3.85, 4.08]	0.94
6	Gender	1.61	0.49	[1.53, 1.67]	–
7	Age	3.02	1.36	[2.84, 3.22]	–
8	Work Experience	15.80	12.83	[14.00, 17.61]	–
9	Employment in Current Job	10.70	10.52	[9.24, 12.19]	–
10	Employment Contract	0.18	0.38	[0.12, 0.22]	–
11	Management Responsibility	0.24	0.43	[0.18, 0.30]	–

Note(s): CI = Confidence interval around the mean; N = 200; gender: 1 = male, 2 = female; age 1 ≤ 24 years, 2 = 25–24 years, 3 = 35–44 years, 4 = 45–54 years, 5 ≥ 55; employment contract: 1 = permanent, 0 = temporary; management responsibility: 1 = yes, 0 = no; work experience and employment at current job in years

Source(s): Authors’ own work

Table 7

Regressing defensive biasing on perceived anonymity, neuroticism, job insecurity and trust in supervisor

Variables	B	SE	t	p	95% CI
Constant	2.25	0.55	4.10	0.000	[1.17, 3.30]
Perceived anonymity	−0.21	0.08	−2.63	0.009	[−0.37, −0.05]
Neuroticism	0.23	0.10	2.25	0.026	[0.03, 0.43]
Job insecurity	0.02	0.06	0.35	0.729	[−0.10, 0.15]
Trust in supervisor	0.06	0.08	0.70	0.490	[−0.11, 0.22]

Note(s): CI = Confidence interval. N = 200

Source(s): Authors’ own work

Table 8

Correlations

Variables	1	2	3	4	5	6	7	8	9	10
1 Defensive biasing
2 Perceived anonymity	−0.19**
	[−0.32, −0.06]
3 Neuroticism	0.18**	−0.09
	[0.04, 0.31]	[−0.23, 0.05]
4 Job insecurity	0.08	−0.03	0.36**
	[−0.06, 0.22]	[−0.17, 0.11]	[0.23, 0.47]
5 Trust in supervisor	−0.02	0.10	−0.28**	−0.24**
	[−0.16, 0.12]	[−0.04, 0.23]	[−0.40, −0.15]	[−0.37, −0.11]
6 Gender	0.02	−0.07	0.23**	0.01	−0.14
	[−0.12, 0.16]	[−0.21, 0.07]	[−0.09, 0.35]	[−0.13, 0.15]	[−0.27, 0.00]
7 Age	−0.18*	0.01	−0.12	−0.15*	−0.14	0.01
	[−0.31, −0.04]	[−0.13, 0.14]	[−0.25, 0.02]	[−0.28, −0.01]	[−0.27, 0.00]	[−0.13, 0.15]
8 Work experience	−0.20**	0.04	−0.03	−0.14*	−0.18**	−0.07	0.82**
	[−0.33, −0.06]	[−0.10, 0.18]	[−0.17, 0.11]	[−0.28, −0.00]	[−0.32, −0.5]	[−0.21, 0.07]	[0.77, 86]
9 Employment in current job	−0.19**	−0.04	0.01	−0.17*	−0.15*	−0.07	0.66**	0.76**
	[−0.32, −0.05]	[−0.18, 0.10]	[−0.13, 0.15]	[−0.30, −0.03]	[−0.28, −0.00]	[−0.21, 0.07]	[0.58, 0.73]	[0.69, 0.81]
10 Employment contract	0.10	0.05	0.04	0.35**	0.15*	0.04	−0.32**	−0.39**	−0.38**
	[−0.04, 0.232]	[−0.09, 0.19]	[−0.10, 0.18]	[0.22, 0.46]	[0.01, 0.28]	[−0.10, 0.18]	[−0.44, −0.19]	[−0.50, −0.27]	[−0.49, −0.25]
11 Management responsibility	−0.18*	0.05	−0.08	−0.06	0.02	−0.23**	0.15*	0.21**	0.23**	−0.13
	[−0.31, −0.04]	[−0.09, 0.19]	[−0.22, 0.6]	[−0.20, 0.08]	[−0.12, 0.15]	[−0.36, −0.10]	[0.01, 0.28]	[0.08, 0.34]	[0.10, 0.36]	[−0.27, 0.01]

Note(s): N = 200; gender: 1 = male, 2 = female; age 1 ≤ 24 years, 2 = 25–24 years, 3 = 35–44 years, 4 = 45–54 years, 5 ≥ 55; employment contract: 1 = permanent, 0 = temporary; management responsibility: 1 = yes, 0 = no; work experience and employment at current job in years

Source(s): Authors’ own work

Table 9

Correlations of defensive biasing with Copenhagen psychosocial questionnaire (COPSOQ) items

	1	2	3	4	5	6	7	8	9	10
1 Defensive biasing
2 Meaning of work	−0.05
	[−0.18, 0.09]
3 Commitment to workplace	−0.06	0.39**
	[−0.19, 0.08]	[0.27, 0.50]
4 Quality of leadership item 1	0.09	0.14	0.39**
	[−0.05, 0.23]	[−0.00, 0.27]	[0.27, 0.50]
5 Quality of leadership item 2	−0.12	0.27**	0.50**	0.44**
	[−0.25, 0.02]	[0.14, 0.40]	[0.38, 0.59]	[0.32, 0.55]
6 Support at work	−0.03	0.26**	0.48**	0.38**	0.61**
	[−0.17, 0.11]	[0.12, 0.38]	[0.36, 0.58]	[0.25, 0.49]	[0.52, 0.70]
7 Unfair treatment	0.11	−0.11	−0.28**	−0.09	−0.29**	−0.41**
	[−0.03, 0.25]	[−0.25, 0.03]	[−0.40, −0.15]	[−0.23, 0.05]	[−0.41, −0.15]	[−0.52, −0.29]
8 Trust and justice	0.05	0.18*	0.38**	0.30**	0.38**	0.36**	−0.21**
	[−0.09, 0.19]	[0.04, 0.31]	[0.25, 0.49]	[0.17, 0.42]	[0.26, 0.49]	[0.23, 0.47]	[−0.34, −0.07]
9 Recognition	−0.02	0.26**	0.53**	0.36**	0.54**	0.51**	−0.28**	0.53**
	[−0.16, 0.12]	[0.13, 0.39]	[0.42, 0.62]	[0.24, 0.48]	[0.43, 0.63]	[0.40, 0.61]	[−0.40, −0.15]	[0.42, 0.62]
10 Intention to leave job	0.05	−0.29**	−0.52**	−0.29**	−0.38**	−0.41**	0.31**	−0.31**	−0.35**
	[−0.09, 0.19]	[−0.41, −0.16]	[−0.61, −0.41]	[−0.41, −0.16]	[−0.50, −0.26]	[−0.52, −0.29]	[0.17, 0.43]	[−0.43, −0.18]	[−0.47, −0.22]
11 Burnout symptoms	0.12	−0.19**	−0.40**	−0.24**	−0.32**	−0.24**	0.24**	−0.27**	−0.36**	0.40**
	[−0.02, 0.25]	−0.32, −0.05]	[−0.51, −0.28	[−0.37, −0.10]	[−0.44, −0.19]	[−0.37, −0.10]	[0.11, 0.37]	[−0.39, −0.13]	[−0.47, −0.23]	[0.27, 0.51]

Note(s): N = 200; Values in square brackets indicate the 95% confidence interval for each correlation; * indicates p < 0.05; ** indicates p < 0.01

Source(s): Authors’ own work