An experimental investigation of BPMN-based corporate communications modeling

Abstract

Purpose

Despite corporate communications having an immense impact on corporate success, there is a lack of dedicated techniques for their management and visualization. A potential strategy is to apply business process management (BPM) approach with business process model and notation (BPMN) modeling techniques.

Design/methodology/approach

The goal of this study was to gain empirical insights into the cognitive effectiveness of BPMN-based corporate communications modeling. To this end, experimental research was performed in which subjects tested two modeling notations – standardized BPMN conversation diagrams and a BPMN extension with corporate communications-specific concepts.

Findings

Standard conversation diagrams were demonstrated to be more time-efficient for designing and interpreting diagrams. However, the subjects made significantly fewer mistakes when interpreting the diagrams modeled in the BPMN extension. Subjects also evolved positive perceptions toward the proposed extension.

Practical implications

BPMN-based corporate communications modeling may be applied to organizations to depict how formal communications are or should be performed consistently, effectively and transparently by following and integrating with BPM approaches and modeling techniques.

Originality/value

The paper provides empirical insights into the cognitive effectiveness of corporate communications modeling based on BPMN and positions the corresponding models into typical process architecture.

Citation

Polančič, G. and Orban, B. (2023), "An experimental investigation of BPMN-based corporate communications modeling", Business Process Management Journal, Vol. 29 No. 8, pp. 1-24. https://doi.org/10.1108/BPMJ-08-2022-0362

1. Introduction

The main goal of communicating is to comprehend and convey information from one person or group to another person or group. Accordingly, communication is recognized as an essential human activity and a critical organizational capability, which needs to be managed and evolved within a corporate environment (Cornelissen, 2020, p. 5). Due to its role in spreading human knowledge, Benczúr (2003) positions human communications at the center of information systems. Accordingly, the management and orchestration of all internal and external communications in an organization is defined as corporate communications, whose aim is to create a positive and coherent view among stakeholders upon whom the company depends (Riel and Fombrun, 2007).

Communication has been intensively investigated in recent decades, resulting in numerous theories and models (Littlejohn and Foss, 2011). The models based on the Shannon-Weaver data transmission model (Shannon and Weaver, 1948) are the most commonly used communication models at the theoretical level, representing communications as a process (Al-Fedaghi, 2012). Considering the “process view” of communicating, internal and external (formal) organizational communications share similarities to business processes (as depicted in Figure 1). Both have a significant impact on corporate success (i.e. business value) (Richmond and McCroskey, 2008; Mohamad et al., 2018), and so represent essential organizational capabilities that must be managed and continuously evolve (Chang, 2005). They are intangible assets (Orand, 2011, p. 13) and are so observable only indirectly via conceptual representations (e.g. diagrams). Moreover, while any collaborative work is supported by communication activities, both types of processes are highly interrelated (Figure 1) – communicating processes can either be seen as a subclass of business processes (Fuks et al., 2005) or as an abstraction of collaborating activities (OMG, 2011).

However, while internal workflows and collaborative processes can be effectively managed with business process management techniques and tools, communication processes (especially in terms of communication tools, Figure 1) commonly lack systematic management (Richmond and McCroskey, 2008). Besides, researchers report (Thøger Christensen, 2002) that formal corporate communication paths are commonly unclear where the communication tools are selected and used inconsistently, redundantly and nonsystematically throughout an organization. The same authors (Thøger Christensen, 2002) also state that “the question of which information to provide and in which form is an important strategic issue”. Accordingly, it is increasingly becoming a challenge to coordinate formal corporate communications in a consistent and effective manner (Cornelissen, 2020, p. 7); where these findings also coincide with those found by Schütze and Baum (2013), stating that existing modeling languages cannot cope with the complexity of human communication processes. Accordingly, this study aimed to gain empirical insights into the cognitive effectiveness of business process model and notation (BPMN)-based corporate communications modeling, which relates to the speed, ease and accuracy with which a model representation can be processed by the human mind (Moody, 2007). To this end, we performed experimental research in which we answered two research questions (RQs).

RQ1.

Do corporate communications modeling language (CCML)-based diagrams gain any measurable benefits in light of cognitive effectiveness compared to BPMN 2.0 conversation diagrams?

RQ2.

Do CCML-based diagrams gain any measurable benefits in light of user perceptions compared to BPMN 2.0 conversation diagrams?

2. Research background

2.1 Corporate communications

Due to its role in spreading human knowledge, Benczúr (2003) positions human communications at the center of information systems. Accordingly, the management and orchestration of all internal and external communications in an organization are defined as corporate communications, whose aim is to create a positive and coherent view among stakeholders upon which a company depends (Riel and Fombrun, 2007). Steyn (2004) defines corporate communications as a strategic management process that provides a strategic framework that helps to connect communication plans to the corporate mission. Yavuz Görkem (2014) summarizes other roles assigned to corporate communications as corporate identity, image, reputation, brand management, media relations, investor relations, government relations, employee relations, community relations, marketing communication, corporate advertising, corporate advocacy, financial communication, corporate social responsibility, organizational communication, management communication, stakeholder analysis, crisis communication and corporate philanthropy. So, a part of corporate communications is organizational communication, which serves six essential functions within organizations: informing, regulating, integrating, managing, persuading and socializing (Turkalj and Fosic, 2009). Organizations apply a rich set of digital media for their corporate communications, which impacts the effectiveness and efficiency of communications, corresponding business processes and corporate success (Richmond and McCroskey, 2008). However, if communications lack proper management techniques, the applied tools tend to be used inconsistently, redundantly and nonsystematically throughout an organization, resulting in nontransparent communication paths. Accordingly, there is an increasing need to coordinate all corporate communications consistently and effectively (Cornelissen and Cornelissen, 2017, p. 7). Recently, Park and Nyhuis (2021) also stated that the lack of a specific understanding of communication in the context of Industry 4.0 leads to the inability to design effective communication concepts in factory systems. These also imply the need for a visual representation of how communications are performed, achieved with corporate communications modeling.

2.2 Corporate communications modeling

Literature reveals the modeling of corporate communications with process-oriented visual languages, e.g. architecture of integrated information systems (ARIS) (Davis and Brabander, 2007), KODA (communications diagnostics) (Kühnle, 1998), unified modeling language (UML)'s communication diagrams (Rumbaugh et al., 2004) and BPMN Conversation diagrams (OMG, 2011). However, these generic process modeling languages have been demonstrated as having limited capabilities in terms of modeling communication-related concepts (e.g. lack of dedicated concepts for modeling of type of communication, communication media and rules applied to the communications) (Schütze and Baum, 2013). So, alternatives were proposed in the form of dedicated modeling languages, such as the one proposed by Schütze (2009), in which the essential elements (e.g. actors and communication relationships) may be additionally specialized with communication-related concepts (e.g. communication interest, communication establishment, communications duration, communication success and communication channel). With respect to the communication channels, the following options are predefined in the proposed language: personal, telephone, e-mail, fax, videoconference and paper letter (Schütze, 2009, p. 113). What lacks the language proposed by Schütze (2009) is a solid, state-of-the-art and standard-based foundation (e.g. UML or BPMN), lack of a formally specified internal structure (i.e. meta-model), and a threat of being outdated since some of the proposed concepts are technology-related (e.g. fax element).

Another visual language, which includes communication-related concepts (e.g. e-mail, short message service (SMS), letter and telephone conversation), is customer journey modeling language (CJML) (Halvorsrud et al., 2014). As the name implies, CJML is a visual language for modeling customer journeys and can support service providers in developing new services and for the maintenance of existing services (Halvorsrud et al., 2016). Halvorsrud et al. (2016) also emphasize that, in contrast to BPMN and UML activity diagrams, which are complex and generic languages, CJML is a dedicated language for modeling and visualizing the service delivery process from the customer's perspective. With respect to modeling corporate communications, CJML is not suitable for depicting them since CJML connectors depict the order of elements (i.e. a journey of “touchpoints”). In contrast, BPMN conversation diagrams' connectors (i.e. conversation links) may represent communication channels. CJML also lacks a formal meta-model.

Besides diagrams, corporate communication processes are partially addressed with “communication plans,” which represent the initial critical step in establishing a project environment (Daojin, 2010). Communication plans usually specify similar information in a tabular form, namely the sender, the receiver, the message and the media used for specific communication. The design structure matrix (DSM – also called dependency structure matrix) is a similar technique. In DSM, the diagonal cells typically represent the system elements (e.g. components in a product, people in an organization or activities in a process), and the off-diagonal cells represent relationships (e.g. dependencies, interfaces or interactions) (Browning, 2016).

The communication between two or more subjects is also conceptualized in subject-oriented process modeling (S-BPM). This approach uses subject interaction diagrams (SID) to capture subjects and the messages exchanged between them. While SIDs do not show any sequential or logical relationship between messages, they are conceptually similar to BPMN choreography diagrams (Moattar et al., 2022).

While BPMN is primarily aimed at business process modeling, it also provides an extension mechanism that allows BPMN to be extended beyond its primary scope. Thus, several extensions have already been proposed (Braun and Esswein, 2014; Zarour et al., 2019); however, we did not identify any BPMN extensions dedicated to the modeling of communication processes. The most similar domain, which was addressed with a BPMN extension, is the social business process management (BPM) (Brambilla et al., 2012). In this extension, new types of “social” BPMN tasks (e.g. broadcast, posting and commenting), “social” events (e.g. community-generated events) and “social” containers (i.e. internal performer, internal observer and external observer), were presented by extending the BPMN process and collaboration diagrams. What hinders the extension is the absence of requirements analysis, a discussion of the semantic fit of domain concepts with BPMN elements and a methodological approach applied in the extension development (Braun and Esswein, 2014). Concerning Industry 4.0. a BPMN extension to model inter-organizational processes was introduced by Ribeiro et al. (2021). According to the authors, the extension is useful for companies certified by the ISO 9001 quality standard that must disclose their processes and third-party collaborations. Recently a dedicated extension for modeling corporate communications was introduced (CCML) and is presented in chapter 3.

2.3 BPMN conversation diagrams

Conversation diagrams (Figure 2) represent the top-level view of BPMN collaboration diagrams. They help represent process landscapes and high-level interactions between involved parties, i.e. representing an overview of a network of partners and how they communicate with each other. Thus, conversation diagrams focus on the message exchange of two or more parties, abstracting from the precise order in which messages occur (OMG, 2011).

Concerning the theoretical communication model (Shannon and Weaver, 1948), the following concepts are supported with BPMN conversation diagrams. The sender and the receiver are modeled with a BPMN pool. The communication content (i.e. message) may be modeled by labeling conversation nodes, whereas the channel is conceptually represented with communication links. The transmitter and receiver represent communication technology, which has no specific concept and representation in conversation diagrams. Like other generic process modeling languages, no special concepts dedicated to communications modeling exists in BPMN (e.g. type of participant, type of communication and rules applied in a communication). This specific communication-related information could be (partially) addressed with BPMN textual annotations; however, in this case, the information would practically reside outside the scope of the notation, whereas the text is also less cognitively effective for encoding information and should only be used as a “tool of last resort” (Oberlander, 1995).

3. A BPMN extension for modeling corporate communications

In this chapter, we will provide an overview of a BPMN extension (hereinafter referred to as CCML, which stands for corporate communications modeling language), which was initially introduced by Polančič and Orban (2019). We will focus on the visual part of the proposed extension (i.e. notation) since the subjects of the experimental research investigated this part.

3.1 The extended notation

To enable modeling of corporate communications, BPMN conversation diagrams' notation was extended with new concepts and corresponding graphical symbols, as well as with new grammatical rules where necessary (e.g. in the case of new connections). Table 7Table 1 presents the CCML elements, their descriptions, use-cases and an association of the notation, which shares a similar concept to its depiction.

3.2 A demonstration CCML diagram

Based on the proposed CCML notation, we have modeled a diagram (Figure 3) by considering the scenario of how software companies (e.g. mobile solutions providers) commonly interact in a corporate environment (i.e. human relations and public relations). Starting from the customer's point of view, a customer (a person type of participant) may purchase the software via several retailers (e.g. these could also be online markets). In case of problems, customers may contact the helpdesk, which involves retailers as well as the corresponding software company (if necessary), where it is evident that the help desk process is performed according to the Information Technology Infrastructure Library (ITIL) best practices framework. Customers may also apply for periodical software updates, which are announced by the software company via a push-type of communication (e.g. Twitter). The data for the applications and updates are downloaded from the production servers automatically (one-way communication only). On the other hand, production servers report statuses (e.g. incidents) directly to the software company. If interested, the potential retailers may apply to negotiations, and in the case of being successful, the business collaboration can proceed (extend relationship used). This collaboration can be further upgraded in case of good business results. The software company also collaborates with several subcontractors, which need to have appropriate certifications for providing daily reports. A standardized way of communication is also required by the financial administration, which offers the possibility to apply for legislation updates (e.g. announced as an really simple syndication (RSS) feed). Conformance with software company processes is periodically evaluated by external auditors, which may also perform corrective action proposals if necessary. As is evident from the diagram, audits are performed in a standardized way, whereas the external auditors simultaneously perform communications in the background, which may be modeled on a separate diagram (e.g. communications with subcontractors).

Figure 3 demonstrates that we could represent the stated scenario in a valid CCML diagram. Despite including new elements, the diagram tends to be comprehensible for BPMN modelers since the meaning of basic shapes (e.g. rectangles and hexagons) remains the same as in the standardized BPMN specification. It is important to stress that, similar to standardized BPMN conversation diagrams, the conversation nodes of the proposed extension may be precisely specialized in the underlying BPMN collaborations diagrams (e.g. with individual message flows). In this manner, a CCML-based diagram help to visualize the critical internal and external communication paths to answer the most important questions, e.g. “Who says what in which channel to whom?” The notation is also functional where there is a need for the standardization of processes, but the workflows are weakly defined, unstable or weakly structured. CCML complements standard BPMN collaboration diagrams in a way that allows for a more detailed description of fundamental interactions between participants and the precise specifications of individual conversation nodes.

4. Experimental investigation

According to the goal of the research, namely to gain empirical insights into the cognitive effectiveness of BPMN-based corporate communications modeling, an experimental investigation was performed to answer two RQs, which have been specified in chapter 1. In order to provide answers to the RQ, an experimental investigation was performed. We chose an experimental approach since we were able to compare the diagrams modeled with CCML against the standardized BPMN conversation diagrams, an approach that was applied in similar studies (Radloff et al., 2015; Schultz and Radloff, 2014). The following subsections describe the research model, followed by a description of the experimental design. The latter includes a definition of the subjects and sampling, experimental process and the corresponding instruments. Finally, the actual operation of the experiment is provided, addressing the preliminary tests and the experiment itself.

4.1 Research model

By considering the RQs and research literature, we anticipated that the diagrams of the extended notation would positively affect the concepts related to cognitive effectiveness, including technology acceptance model (TAM)-related concepts. The resulting research model is presented in Figure 4.

As is evident from Figure 4, we identified one independent variable, “type of notation”, with two levels: standardized notation and extended notation (CCML). In line with the review of process model understandability indicators (Dikici et al., 2018), we defined the following dependent variables. Subjective impacts (i.e. perceived understandability) were conceptualized with TAM variables: perceived ease of use (PEOU), perceived usefulness (PU) and intention to use (ITU), all measured with standardized Likert items as initially defined by Davis (1989), also representing a typical operationalization in the process understandability domain (Dikici et al., 2018).

The objectively measured impacts (i.e. objective understandability) were conceptualized by effectiveness (i.e. accuracy) and efficiency (i.e. speed), where both concepts were additionally divided into impacts performed by modeling activities (i.e. activities for creating diagrams) and impacts of interpretation activities (i.e. activities related to reading and interpreting of diagrams) (Gemino and Wand, 2004). As a result, four dependent concepts were defined: “modeling efficiency,” “diagram interpretation efficiency,” “modeling effectiveness” and “diagram interpretation effectiveness.”

Both efficiency-related concepts were measured with “task time” (t_m and t_i), which is a common measure of efficiency (Garcia et al., 2005; Glezer et al., 2005; Reijers et al., 2011; Mendling et al., 2012), with lower values preferred. The modeling and interpretation effectiveness concepts were measured by the number of syntactical or semantical errors generated (E_m) or semantically wrong answers provided (E_i), which is in line with a subset of effectiveness measures as proposed by Gemino and Wand (2004). Additionally, testing how well users comprehend the content of a diagram is also known as “interpretation fidelity” (Recker, 2013). In the case of “diagram interpretation effectiveness”, answers marked as “undecided” were added to the E_i measure (E_{i =} number of wrong answers + number of “undecided” answers). Again, lower E_i and E_m values were preferred. Based on the research model, we defined the following hypotheses (Table 2).

As evident from Table 2, we specified seven research hypotheses in which we stated that the extended notation (CCML) would provide measurable benefits compared to the standard notation. Thus we expected greater “extended notation” values in the case of TAM-related measures, whereas in the effectiveness (i.e. number of errors) and efficiency-related measures, lower values were expected for the “extended notation.” Two experimental treatments were required to make effects visible and to test the stated hypotheses (Table 2), namely the use of standardized notation and the use of extended notation (CCML).

4.2 Subjects and sampling

Theoretically, the ideal candidates as experimental subjects would be experts in process modeling (i.e. process owner, process analyst, quality manager), with no specific preferences or attitudes with respect to both notations. Since such candidates could not be practically attracted (the extended notation was our new proposal), we specified the sample frame criteria as subjects with no prior knowledge of any notation and no domain knowledge, which may also affect the results (Parsons and Cole, 2005). Thus, we involved first-year information technology (IT) students who represent the novice population. This is in accordance with the theoretical and empirical evidence that suggests that changes in visual appearance impact understanding significantly, especially for novices (Moody, 2009; Reijers et al., 2011; Burton-Jones and Meso, 2008). Additionally, when selecting participants, Parsons & Cole (Parsons and Cole, 2005) stated that experts should not be used. While this sacrifices the external validity of the experiment, the authors argue that participants should be able to extract the information only from the diagram and not from their background knowledge.

Fifty-one students participated in the experiment, being motivated with bonus points for their study program course. The subjects were completely unaware that the extended notation was our proposal, which could otherwise have impacted the experimental outcomes.

4.3 Experiment process

The experiment process was defined as follows (Table 3). First, to minimize the effects of a within-subjects experiment, we randomly (R) assigned the subjects into two experimental groups – G₁ (25 students) and G₂ (26 students). Each received equal treatment, yet in different orders. Before both experimental treatments, the subjects answered some demographic questions (i.e. gender and education) and questions related to prior knowledge with modeling notations (O₀).

Within each experimental treatment (X), the subjects performed two main types of activities. In the first part of each experimental procedure, we tested the comprehension of a diagramming notation (i.e. the effectiveness of interpreting diagrams). This was done by subjects who indicated (by answering “yes,” “no,” or “undecided”) whether the provided statements were consistent with the provided syntactically verified diagram. This verification was necessary since syntactically, wrong diagrams could also imply incorrect semantics. The resulting sum of “wrong” answers and “undecided” answers represented the measure “number of interpretation errors” (E_i) of the concept “diagram comprehension effectiveness”.

In the second part of the experimental procedure, the subjects received a structured scenario of corporate communications where their task was to create a valid diagram (Appendix). This part of the experimental procedure tested “modeling effectiveness” and was measured by the number of syntactic and semantic errors in the resulting diagram (E_m). Simultaneously, the durations of both parts of the experimental procedure (i.e. interpretation activities and modeling activities) were observed to acquire interpretation (t_i) and modeling times (t_m), which indicated the corresponding type of efficiency. The aggregated efficiency and effectiveness values were calculated as follows.

1. Total time working with standardized notation: t(s) =ti(s) +tm(s)

2. Total number of errors when working with standardized notation: E(s) =Ei(s) +Em(s)

3. Total time working with extended notation: t(e) =ti(e) +tm(e)

4. Total number of errors when working with extended notation: E(e) =Ei(e) +Em(e)

5. Total experimental time: texp=t(s) +t(e)

6. Total number of errors in the experiment: Eexp=E(s) +E(e)

After each performed treatment, the subjects expressed their perceptions concerning the used notation by indicating the level of agreement with the statements relating to PEOU, PU and ITU. As previously stated, we adapted the standardized Likert-based statements for measuring the TAM concepts.

4.4 Experiment instruments and operation

In order to perform the experiment process, paper-based instructions were provided to the subjects (Appendix). They included basic information about the experiment, instructions for performing experimental treatments and the corresponding questionnaire. In line with Parsons and Cole's recommendations (2005), the subjects also received a concise description of the standardized and extended notations used for quick reference (i.e. a cheat sheet). Table 4 summarizes the structure of the subjects' instructions, which were provided to the experimental group G₁. In contrast, the experimental group G₂ received instructions in the opposite order of the provided experimental treatments (i.e. standardized notation, following extended notation). These instructions were pretested and afterward improved (e.g. by omitting complex terms, the language of the task descriptions was suited to the subjects).

As is evident from Table 4, the experimental process consisted of three main parts. Initially, the subjects received the first part of the questionnaire, which included the introduction, demographic questions and questions related to the subjects' expertise. When all subjects completed the first part, they received the second part of the instructions, i.e. the first experimental treatment. When the subjects finished the second part of the experiment, the third part of the instructions was delivered to them – CCML diagram-related tasks in the case of G₁ instructions. The experiment was defined as finished when all subjects finished all of the experimental tasks.

The second and the third part of the instructions were additionally divided into the comprehension and modeling parts, as stated below. The comprehension part of the instructions consisted of a diagram modeled in one of the investigated notations. Both diagrams were modeled and validated against the same scenario and were so information equivalent. Below the diagram, joint diagram-related statements were provided, where the subjects were asked to provide answers with respect to their validity (“yes,” “no” or “undecided”). In line with the second Parsons and Cole (2005) criteria, the statements focused on measuring semantics conveyed by modeling constructs in a diagram without reasoning beyond the information presented.

The modeling part of the instructions was defined in a similar manner. Again, in both treatments (standardized and extended notation), a consistent scenario was defined in a structural text form, where the subjects were instructed to create a diagram in the specified notation with the aid of an A4-size description of the corresponding notation, as recommended by Parsons and Cole (2005).

5. Data analysis

The data analysis was performed using SPSS statistics. For the purpose of testing the differences between the standardized and extended notation-related treatments, several paired t-tests were performed, which measured whether the means from a within-subjects test group varied over the two testing conditions – in our case, standardized and extended notations. While performing a within-subjects design, another set of t-tests was performed to investigate the potential differences in the means of observed variables due to the moderating variable' order of experimental treatments'. To reduce the chances of obtaining false-positive results when multiple tests are performed on a single set of data, the family wise error rate (FWER) was considered by using a single-step “Bonferroni correction” method (Bland and Altman, 1995). We considered normal distributions in cases of moderate or larger sample sizes, “paired samples t-test” to be reasonably accurate even when violating the normality assumption. A commonly accepted value for moderate sample size is 30.

Data analysis results are graphically represented within the box-plot diagrams. The boxes represent 25 to 75% of the responses. The entire range of values chosen by the participants is indicated by horizontal markers placed outside the box plot. The whiskers present minimum and maximum values, whereas the midline indicates the median value.

5.1 Subjects and experimental groups statistics

As previously mentioned, 51 subjects were involved in the experiment, 41 of them male (80.4%) and ten female (19.6%). The subjects displayed similar expertise with respect to the processing of modeling notations (Table 5); the majority of them reported “not knowing” any of the stated process modeling notations, which was in line with the preferred experimental sample, as stated in section 4.2.

The mean value of the total time the subjects spent performing the experimental procedures (t_exp), was 1523.8 s or approximately 25 min (t_exp in Figure 5).

Since the subjects were classified into two experimental groups (G1, G2), with the opposite sequence of experimental procedures (Table 3), we performed several independent samples t-tests to investigate the experimental results' differences between these two groups. The results show significant differences in the cases of measuring the PEOU of the extended notation and all efficiency measures (t_i(s), t_m(s), t_i(e), t_m(e), t(s), t(e)). Except for the PEOU, the differences between the mean values of the investigated variables also remained significant after applying the “FWER” (“Bonferroni correction” method) (Bland and Altman, 1995). Contrarily, the differences between the means of experimental groups were found to be insignificant in the cases of all effectiveness measures (E_i(s), E_m(s), E_i(e), E_m(e)) and remaining TAM measures (PU, ITU). Additionally, the means of the total experimental time (t_exp) did not differ significantly between experimental groups, meaning that both groups finished their experimental tasks within a similar time frame.

5.2 Hypotheses testing

The results of the hypotheses testing were structured in accordance with the scales of the latent dependent variables of the experiment into efficiency measures (t_i and t_m), effectiveness measures (E_i and E_m) and TAM measures (PEOU, PU, and ITU).

The box plots in Figure 5 show the results of the efficiency measures (i.e. modeling and diagram interpretation times as well as the aggregated values) concerning standardized and extended notations. All efficiency measures were defined in seconds with lower preferred values. As is evident from the boxplots (Figure 5), the means were lower in the case of the standardized notation: t_i(s) = 177.53s and t_m(s) = 498.78s, where it took more time to model and interpret diagrams in the extended notation – t_i(e) = 247.88s and t_m(e) = 595.65s.

Paired t-tests were performed to determine whether the means of efficiency measures differed statistically for standardized and extended notations. The results show a statistically significant difference between diagram interpretation efficiency means (t_i(s) and t_i(e)) at p = 0.001, where no significant difference between t_m(s) and t_m(e) means was determined (p = 0.078). In the case of aggregated values, a significant difference between the means of t(s) and t(e) was found at p = 0.014.

The next set of measures was used to investigate modeling (E_m) and interpretation (E_i) effectiveness by counting the number of mistakes when interpreting diagrams or creating diagrams in standardized and extended notations. Again, lower values were preferred.

As is evident from the box plots (Figure 6), the interpretation of diagrams in the extended notation was more effective (mean = 2.67) since the subjects interpreted the diagrams with fewer errors when compared to the standardized notation (mean = 3.80). In contrast, more errors were produced when modeling diagrams with the extended notation (effectiveness means: E_m(s) = 2.75, E_m(e) = 5.20). The paired t-tests show statistically significant differences between the means of both effectiveness measures at p < 0.001. Since the number of errors in the interpreting diagrams measure (E_i) was a sum of the “wrong” answers and “uncertain” answers (as defined in section 4.1), we additionally performed an analysis of individual measures with the following results: means E_i-wrong(s) = 2.22, E_i-wrong(e) = 1.49, E_i-uncertain(s) = 1.59, E_i-uncertain(e) = 1.18. The differences between the means were significant in the case of wrong answers (E_i-wrong) at p = 0.001, while in the case of uncertain answers, the p-value equals 0.066. In the case of aggregated values, a significant difference between the means of E(s) and E(e) was found at p = 0.008.

The last set of measures was used for measuring standardized TAM variables: PEOU, PU and ITU on a seven-point Likert scale, with higher values preferred.

As evident from the box plots in Figure 7, the means of all variables were similar, ranging from the values' undecided' (value 4) to “somewhat agree” (value 5). The means of the measures of the extended notation were in all cases higher than the means of measures of the standardized notation (PEOU(s) = 4.25, PEOU (e) = 4.63, PU(s) = 4.16, PU (e) = 4.40, ITU(s) = 4.12, ITU (e) = 4.16), meaning that the subjects reported more positive perceptions with respect to the extended notation.

The performed paired t-tests indicate significant differences between the means of PEOU concerning standardized and extended notations at p = 0.003, where no statistical differences were found between the means of the remaining two TAM variables (PU and ITU).

After considering the above results, it was possible to test the stated hypotheses (Table 2). We either failed to reject or reject the null hypotheses in favor of alternative ones (i.e. research hypotheses) (Table 6).

As evident from Table 6, we rejected four out of seven null hypotheses, which were stated in Table 2. Out of them, two hypotheses were rejected in favor of the extended notation – CCML (PEOU, number of interpretation errors – E_i), and two hypotheses were rejected in favor of the standardized notation (interpretation time – t_i, number of modeling errors – E_m). On the opposite, no significant differences were found between the means of the standardized and extended notation with respect to the following observed variables: PU, ITU and modeling times (t_m).

In order to reduce the chances of obtaining false-positive results when multiple tests are performed on a single set of data, the “FWER” was additionally considered (Bland and Altman, 1995), and the experiment-wide type 1 error rate of 0.05 was divided by the number of investigated hypotheses (0.05/7 = 0.007). This intervention was also logical due to significant Pearson's Correlations, which were found between the measurable variables. However, while the highest significance level within the rejected null hypotheses was 0.003 (H01) the corrected significance level still supports the conclusions presented in Table 6.

6. Discussion

With respect to the tested hypotheses, the performed research resulted in the following new insights. We found that dealing with the CCML was more time-consuming in both cases when interpreting the stated diagrams (t_i) as well as in the case of modeling diagrams based on the textual descriptions (t_m). However, significant differences were only found in the case of time spent interpreting the stated diagrams (i.e. comprehension) in favor of the standardized notation (H5). These results may be explained as follows. Both notations were previously unknown to the subjects, where the “graphic complexity” of the extended notation (eleven additional elements) was higher than the standardized one (eight elements). Thus, the subjects required more time to learn the extended notation, which is in line with the cognitive limits on the number of visual elements (i.e. categories) that may be effectively recognized (Lemon and Lemon, 2000).

In light of the modeling and interpretation effectiveness measures, the differences between the means of the corresponding measures were significant. In the case of modeling activities (E_m), Conversation diagrams have been demonstrated to be more effective since the subjects produced significantly fewer errors when using it compared to the CCML. Again, since the extended notation is more complex than the standardized one, those subjects who were previously unaware of both notations made more errors when selecting appropriate elements for modeling in the case of the extended notation, as was also found by Nordbotten and Crosby (1999), and this also cost some additional time, as is evident from measuring the modeling times (t_m). In contrast, the subjects responded significantly more accurately to the statements (E_i), when interpreting the diagrams in CCML. We assume that this measure (E_i) was positively impacted by the semantic transparency of the investigated symbols – since, as the subjects were unaware of both notations, they recognized the symbols and corresponding icons of the extended notation (i.e. device icon, human icon, push conversation icon) more easily when compared with the standardized elements (i.e. hexagon representing a conversation node) or textual annotations. The extended notation uses “semantically immediate” icons so their meaning can be perceived directly or easily learned. In contrast, the existing notation mainly specifies “semantically opaque” (i.e. conventional) symbols. Therefore, subjects quickly recognized the meaning of the extended notation elements and responded to the statements more accurately. These conclusions are in line with existing empirical studies showing that visual syntax significantly affects a diagram's understandability, especially by novices (Irani et al., 2001; Purchase et al., 2004), whereas Recker (2013) found that implicit (i.e. nonintuitive) representations led to lowered comprehension, which in turn can lead to low consistencies, ambiguities and thus poor decisions made based on the models. In general, semantic transparency appears to be a common problem of design notations, where most effort is spent on semantics with graphical conventions mainly as an afterthought (Moody, 2009).

Experiences gained with modeling and interpretation activities impacted subjects' perceptions in the following ways. Concerning all observed TAM variables (PEOU, PU, ITU), the subjects reported more positively regarding the extended notation, where significant differences between the investigated mean values were found in the case of PEOU in favor of the extended notation. According to these, we may conclude that the behavioral intentions to use the extended notation are higher when compared to the standardized one.

Based on the above, the answers to the stated RQs may be provided as follows. With respect to RQ1, we can conclude that diagrams based on CCML convey information more precisely when compared to BPMN conversation diagrams yet are more time-consuming to construct and interpret. With respect to RQ2, we can conclude that subjects developed more positive perceptions when dealing with CCML-based diagrams, with higher variable means, when compared to BPMN conversation diagrams.

6.1 Validity threats

The concerns regarding the internal construct and the external validity were investigated and addressed according to Trochim and Donnelly (2006) and Neuman (2005, pp. 259–266). Concerning internal validity, the following threats were considered and controlled. The “Selection bias” threat was minimized by randomly assigning experimental subjects into two experimental groups. Second, the questions that were answered before any treatments took place asked subjects about their basic information and expertise (Table 4) and thus did not impact the dependent variables (i.e. “testing effect”). “Diffusion of treatment” was managed by instructing subjects not to communicate with each other during the experimental process, where the “experimenter expectancy” threat was controlled by the “double-blind” experimental approach. While the subjects' results might also be impacted by unclear experimental instructions, they were pretested for comprehension. “Statistical regression” threats were minimized by using objective measures for effectiveness and efficiency, whereas subjective measures for TAM were based on standard items, as initially proposed by Davis (1989). “Demand characteristics” was addressed using the “double-blind” experimental approach.

The next set of validity threats relates to constructs and related indicators (“measurement validity” and “face validity”). From the standpoint that a quality diagram has to be syntactically and semantically correct, we used a number of corresponding errors as indicators of modeling effectiveness. Accordingly, we defined diagram interpretation effectiveness. Second, we presumed that a modeler is more efficient if they perform their work more quickly, so we used the duration of the corresponding experimental activities as a measure of efficiency. These variables and the related indicators also corresponded to “cognitive effectiveness” as defined by Larkin and Simon (1987).

Threats to external validity were also identified. First, experimental subjects corresponded to novice modelers and were selected since they were unfamiliar with the notations used in both experimental treatments. However, we were aware that experienced modelers who would be trained in both notations could react differently. We assume that new insights could be obtained if repeating the experimental process after equally training subjects with both modeling notations. “Mundane realism” was considered by simulating real-world scenarios in the stated diagrams and structured textual descriptions of corporate communications scenarios.

6.2 Implications and future work

By considering the results of the diagram interpretation effectiveness (i.e. fewer errors were generated when interpreting CCML diagrams), we can conclude that CCML enables a practical use and interpretation of diagrams that depict corporate communications. CCML may be helpful for employees, who could apply the proposed diagrams to obtain the answers to the most common communication-related questions visually, namely “Who says what in which media/tool to whom?” Additionally, the proposed notation is also applicable where there is a need for the standardization of processes, but the workflows are still mainly unstructured and thus challenging to specify. From an IT perspective, CCML diagrams may simplify the specification, implementation, or selection of corporate communication technologies (i.e. TO-BE communications) and provide clear insight and consolidation of used communication technologies (i.e. AS-IS communications). Moreover, while the communication technologies may convey sensitive data, CCML may also simplify the assurance of compliance with data protection regulations (e.g. by clearly defining the information paths of formal corporate communications).

The proposed language can be applied to existing organizational business architecture as follows. A common way to organize the system of an organization's business processes is a process architecture, where the Camunda BPMN framework (Freund and Rücker, 2016) defines two architectural levels – strategic and operational (Figure 8). The landscape level, which explains how processes are interrelated, is positioned above the strategic process level and usually depicts how the core processes are represented as a value chain, which implies sequential execution (Malinova et al., 2014).

In contrast to value chain-based diagrams, interaction diagrams (e.g. CCML diagrams) do not depict the order of activities but focus on how participants interact (i.e. communicate). While both types of landscape-level diagrams represent the top-level views of an organization's processes, they act complementary since they expose different kinds of relationships: workflow-related and communications-related. Accordingly, value chain-based landscape diagrams refer to (or derive abstract information from) underlying BPMN process diagrams and their sequential relationships, whereas, as previously stated, CCML diagrams refer to (or gain abstract knowledge from) underlying BPMN collaboration diagrams (between-participants interactions). As already mentioned, CCML diagrams have been perceived by experimental subjects as easy to use and for conveying information effectively (both significantly), presumable due to a higher ratio of visual/textual information provided in the diagrams. In this manner and by considering potential diagram users (e.g. chief information officers, quality managers and other decision-makers), CCML diagrams share similarities with business intelligence dashboards, which also heavily rely on visual information. The drawbacks of CCML-based diagrams, associated with the correctness of the corresponding modeling activities, do not directly impact these decision-makers as the diagrams are commonly modeled by operational staff. Besides, these drawbacks of CCML could be addressed by implementing CCML into modeling tools that typically assist modelers with advanced functionalities such as element recommendations and syntactical validations.

Future work is planned in the following directions. First, since standardized BPMN conversation diagrams outperformed CCML in light of the speed of interpreting diagrams and modeling errors, incremental improvements to the language are planned (e.g. new concepts, refinements to concepts' depictions, complexity management mechanisms built into the language, etc.). Second, with respect to testing CCML, and related modeling languages, we plan to perform experimental and case-study-based investigations, also considering the needs and characteristics of landscape-level modeling. We also plan to implement CCML into modeling tools for more realistic results, enabling subjects to create and test more complex diagrams. Additionally, case studies with BPMN experts are planned to obtain quantitative and qualitative insights about the cognitive effectiveness and applicability of CCML and its constituent elements.

Figures

Figure 1

Relationships between business processes and communication processes

Figure 2

BPMN Conversation diagram

Figure 3

Example diagram – corporate communication with external stakeholders

Figure 4

Research model

Figure 5

Box plots for efficiency measures

Figure 6

Box plots for effectiveness measures

Figure 7

Box plots for TAM measures

Figure 8

Positioning CCML diagrams in a process architecture

Figure A1

Example diagrams in standardized notation (left) and extended notation (right)

Table 1

Extended notation

Table 1

Extended notation

Table 2

Research and corresponding null hypotheses

Research hypothesis	Null hypothesis
H1: PEOU (standardized) < PEOU (extended)	H01: PEOU (standardized) = PEOU (extended)
H2: PU (standardized) < PU (extended)	H02: PU (standardized) = PU (extended)
H3: ITU (standardized) < PU (extended)	H03: ITU (standardized) = PU (extended)
H4: tm (standardized) > tm (extended)	H04: tm (standardized) = tm (extended)
H5: ti (standardized) > ti (extended)	H05: ti (standardized) = ti (extended)
H6: Em (standardized) > Em (extended)	H06: Em (standardized) = Em (extended)
H7: Ei (standardized) > Ei (extended)	H07: Ei (standardized) = Ei (extended)

Table 3

Experimental process

		Observations	First treatment	Observations	Second treatment	Observations
R	Group 1 (G1)	O0	X(s)	Os	X(e)	Oe
	Group 2 (G2)	O0	X(e)	Oe	X(s)	Os

Note(s): R … random assignment; G_x … sequence of treatments according to the “type of notation”; O … observations; X … treatments; X = {s … standardized notation, e … extended notation}

Table 4

Subjects' instructions (group “G₁”)

#		Task name	Task description	Observations
1		Introduction	Basic information about the experiment and corresponding questionnaire	/
		Basic information	Demographic questions and questions relating to the subject's expertise (e.g. “Are you familiar with the following process modeling notations?”)	Gender
				Expertise
2	First experimental treatment	Standardized diagram/comprehension	Subjects received an example diagram in standardized notation and were instructed to give answers (i.e. “yes”, “no” or “undecided”) to the corresponding statements. Additionally, subjects received an A4 size description of the standardized notation	Ei(s)
				ti(s)
		Standardized diagram/modeling	Subjects received textual descriptions of corporate conversations and were instructed to create a consistent diagram by using the standardized notation	Em(s)
				tm(s)
		Standardized diagram/TAM statements	Subjects were instructed to provide levels of agreement to Likert-based TAM statements relating to PEOU, PU and ITU with respect to the standardized notation	PEOU(s)
				PU(s)
				ITU(s)
3	Second experimental treatment	CCML diagram/comprehension	Subjects received an example diagram in the extended notation and were instructed to provide answers (i.e. “yes,” “no” or “undecided”) for the corresponding statements. Additionally, subjects received an A4 size description of the extended notation	Ei(e)
				ti(e)
		CCML diagram/modeling	Subjects received textual descriptions of corporate conversations and were instructed to create a consistent diagram in the extended notation	Em(e)
				tm(e)
		CCML diagram/TAM statements	Subjects were instructed to provide levels of agreement to Likert-based statements relating to PEOU, PU and ITU with respect to the extended notation	PEOU(e)
				PU(e)
				ITU(e)

Table 5

Subjects' expertise with process modeling notations

Level of expertise	Not knowing	Knowing did not tested	Already tested	Using
BPMN	48	3	0	0
UML	50	1	0	0
EPC	49	2	0	0

Table 6

Hypotheses testing

Null hypothesis	Measure	Findings
H01	PEOU	H01 was rejected in favor of H1. The reported PEOU of the extended notation was significantly higher (p = 0.003) when compared to the standardized one
H02	PU	H02 failed to reject. No significant differences were indicated between the reported PU means of standardized and extended notation (p = 0.110)
H03	ITU	H03 failed to reject. No significant differences were indicated between the reported ITU means of standardized and extended notation (p = 0.689)
H04	tm	H04 failed to reject. No significant differences were found between the times spent creating diagrams (tm) in standardized and extended notation (p = 0.078)
H05	ti	H05 rejected. Time spent interpreting the diagram, modeled in standardized notation (ti) was significantly lower when compared to the extended one (p = 0.001)
H06	Em	H06 rejected. Significantly more mistakes (Em) were generated when creating diagrams in extended notation in relation to the standardized one. (p < 0.001)
H07	Ei	H07 was rejected in favor of H7. Significantly more mistakes (Ei) were generated when answering statements relating to diagrams created in standardized notation in relation to the extended one (p < 0.001)

Appendix

Comprehension part

The following figure represents example diagrams (a translated version) in the standardized and extended notation, which were provided to experimental subjects in the comprehension part of the experimental treatments.

According to one of the above diagrams, the subjects were asked to provide answers to the following statements (a translated version) {yes, no, undecided}

1. A Student represents an organization.

2. An Official shall be only one person.

3. Several conversations may be performed between an Official and a Student.

4. There is only one conversation between the Mayor and the Officials.

5. Communication between the Student and SCT Ltd. is one-way.

6. The Mayor has, in addition to the official, additional rights.

7. A student can only apply for the application when a call for funding for a scholarship is announced.

8. Communication of SCT Ltd. can be viewed on a separate model.

9. There is a three-party agreement when signing a scholarship contract.

10. A student can sign a scholarship contract when a call is issued.

Modeling part

The following scenario (a translated version) was provided for the experimental subject in the modeling part of experimental treatments. The subjects were asked to model diagrams in one of the investigated notations (standardized notation, extended notation).

1. John reads the news via the ‘XY’ application on his mobile phone.

2. The application ‘XY’ on the phone retrieves news from the news server xy.com, via RSS technology.

3. In the ‘XY’ application, Google is advertising by sending message ads at the application's request. John receives an advertisement for a nonalcoholic drink.

4. John wants to buy this nonalcoholic drink in large quantities, so he communicates with the ABC store through consecutive messages via e-mail.

5. The ABC store has a complex way of ordering the nonalcoholic drink between Company A and Company B, which is irrelevant to our case and can be placed in a separate model.