Reliable introduction of critical assets: an explorative case study in railways

Abstract

Purpose

Asset-intensive organizations rely heavily on physical assets that are often expensive, complex and have a significant impact on organizational performance. Past introductions of critical assets in various industries showed that despite many preparations in maintenance and operations, shortcomings were identified after deployment resulting in unreliable performance. The main purpose of this qualitative study is to explore the factors that determine how asset-intensive organizations can achieve reliable outcomes in critical asset introductions despite random failures as a result of increasing complexity and infant mortalities.

Design/methodology/approach

To gain a detailed understanding of the issues and challenges of critical asset introductions, a case study in railways (rolling stock introductions) was conducted and analyzed using qualitative analysis.

Findings

The case showed that organizational factors were perceived as decisive factors for a reliable performance of the introduction, while the main focus of the introduction was on the asset and its technical systems. This suggests that more consideration toward organizational factors is needed. Therefore, a critical asset introduction framework was proposed based on 15 identified factors.

Originality/value

Reliable performance is often associated with technical systems only. This empirical study emphasizes the need for a more holistic perspective and the inclusion of organizational factors when introducing critical assets seeking reliable performance. This study demonstrated the application of the affinity diagramming technique in collectively analyzing the data adopting a multidisciplinary orientation.

Citation

Moerman, J.-j., Braaksma, J. and van Dongen, L. (2021), "Reliable introduction of critical assets: an explorative case study in railways", Journal of Quality in Maintenance Engineering, Vol. 27 No. 3, pp. 537-549. https://doi.org/10.1108/JQME-05-2020-0037

1. Introduction

Asset-intensive organizations rely on physical assets that are often expensive, complex and have a significant impact on their performance. The introduction of new critical assets should therefore be carefully prepared to avoid unwanted variance in performance. However, past introductions of critical assets in various industries showed that despite many preparations, shortcomings were identified after commissioning, resulting in unreliable performance. These failures can have a detrimental impact on organizations, not only from a safety and financial perspective, but also from a social and environmental point of view. A typical example from aviation is the accidents with the introduced Boeing 737 MAX. While the first aircraft entered into service in May 2017, the crash of two 737 MAX passenger aircrafts in late 2018 and early 2019, and the grounding of the entire fleet of 737 MAX aircrafts, turned a global spotlight on Boeing's practices and culture. Explanations for the crashes, as summarized by Herkert et al. (2020), include: design flaws within the 737 MAX's new flight control software system; internal pressure to keep pace with Boeing's chief competitor; Boeing's lack of transparency about the new software; and the lack of adequate monitoring of Boeing by the FAA during the certification of the 737 MAX and following the first crash. This already illustrates the many challenges associated with critical asset introductions. However, which factors need to be taken into consideration to cope with unwanted variance in performance is still unanswered.

The main purpose of this study is to explore the factors involved that determine how asset-intensive organizations (e.g., airliners and railway operators) can achieve reliable outcomes when introducing critical assets, despite random failures [e.g., as a result of early infant mortality failures (Klutke et al., 2003)], which cannot always be predicted in advance. This study is a first exploration on the factors involved when introducing new critical assets using an in-depth case study in railways. Past introductions of rolling stock showed that despite many preparations, shortcomings were identified after operational deployment resulting in unreliable train services for passengers. For example, the Dutch V250 train services were cancelled in 2013 after two months in operation, after the introduction had already been delayed for several years. To gain a detailed understanding of the challenges and issues of rolling stock introductions, the authors adopted case study design (Yin, 2003) using affinity diagramming techniques for a collective analysis of the data. The affinity diagramming technique was described by Brassard (1989), and has been widely used in design methods and systems thinking. The affinity diagram organizes a large quantity of information by natural relationships.

The remaining part of this paper proceeds as follows: Section 2 presents the key theoretical concepts of critical asset introductions. Section 3 introduces the inductive research design, focusing on the collective data analysis technique. Section 4 introduces the context of the case and presents the findings. Section 5 elaborates on the theoretical and practical implications. Section 6 concludes this paper with the limitations of this study and future research directions.

2. Critical asset introductions

Asset-intensive organizations rely on physical assets that have a significant impact on their performance. The management of these physical (critical) assets, such as buildings, infrastructures, utilities, plants and transportation, is addressed by the discipline of asset management. Nowadays, asset management has gained momentum and interest in both academia and industry as a business process and as a discipline (El-Akruti et al., 2013). Asset management is defined as: “a coordinated activity of an organization to realize value from assets” (International Standard Organization, 2014). It involves the balancing of costs, opportunities and risks against the desired performance of assets to achieve organizational objectives. Following Ouertani et al. (2008) assuming a “cradle-to-grave” approach, the lifecycle of a typical asset consists of five main phases: acquire, deploy, operate, maintain and retire. In railways, one distinguishes five stages in the lifecycle of trains for each type of rolling stock series (Van Dongen, 2015): investing, phasing-in, guaranteeing and optimizing, extending operational life and phasing-out. The primary scope of this research is the phasing-in phase (deploy in Figure 1), although linkages between other stages are considered.

The reliability of critical assets is determined by the number of failures when operated. Following the standard bathtub curve (Figure 2), as often mentioned in reliability theory, random failures often occur in the infancy and wear out stages. The bathtub curve occupies a place of considerable importance in reliability practice, particularly in justifying phasing-in strategies and techniques for improving system reliability (Klutke et al., 2003). Moreover, Van Dongen (2011) states that safety, availability and reliability can not only guaranteed by technically perfect designs and maintenance concepts. The attitude of managers, operators and mechanics determine, to a great extent, the final result. This is also reflected in a Delphi study, which states that cultural and behavioral change is listed in the top three priorities of asset management organizations and is considered especially important as it removes some of the key roadblocks which have been limiting progress in all other areas of asset management (Akkermans et al., 2016).

3. Research method

To gain an initial understanding of the issues and challenges of critical asset introductions, the authors adopted case study design (Yin, 2003) using affinity diagramming techniques for a collective analysis of the qualitative data incorporating three recent rolling stock introductions. Due to the inherent nature of rolling stock introductions (and critical asset introductions in general), which can be considered complex functions, a qualitatively approach was adopted. The affinity diagramming technique was described by Brassard (1989), and has been widely used in design methods (Simonsen and Friberg, 2014) and systems thinking (Maani and Cavana, 2000). The affinity diagram organizes a large quantity of information by natural relationships. It was invented in the 1960s by the Japanese anthropologist Kawakita Jiro (KJ) and is sometimes referred to as the KJ method.

The rationale for adopting affinity diagramming was the point to facilitate a collective process of interpretation based on the collected data (interviews and additional documentation) from three former rolling stock introductions [Sprinter Light Train (SLT), the V250 and the FLIRT]. Different stakeholder perspectives (e.g., train manager, train drivers, mechanics, engineers, management) were included to share and visualize their contribution and to anchor the results in an affinity diagram. The benefit of the affinity diagramming approach is that it allows participants to reflect on the rolling stock introductions, suspend their judgment in order to see issues from multiple perspectives and to decide on the most effective actions to take to ensure a reliable introduction of future rolling stock. This multidisciplinary orientation is also recognized in asset management literature as an important gap in, e.g. knowledge sharing (Ruitenburg et al., 2014) and decision-making (Van Dongen, 2015). For systems theorists, the source of poor performance, organizational failure, and inability to adapt is often to be found in the limited cognitive skills and capabilities of individuals compared to the complexity of the systems they are called upon to manage (Simon, 1979; Perrow, 1984). Systems thinking states that it is easy, but often misleading, to focus on specific, isolated events. It is more valuable to investigate trends, and the underlying structure shaping those trends, and to act on that information.

Figure 3 illustrates the research process in this study which incorporates the affinity diagram process based on Brassard (1989). Affinity diagramming is theoretically rooted in grounded theory. Grounded theory was developed by Glaser and Strauss (1967) as a strategy for analyzing qualitative data. Grounded theory forms an inductive process where single observations from the data are analyzed in a bottom-up approach to form more general conclusions. The theory analyzes qualitative data from transcripts or elaborated summaries of interviews. After coding of the data using open coding (Corbin and Strauss, 1990), key points of interest in the data are marked, and through a process of constant comparison of statements, low-level and high-level categories emerge from the data as groups of codes of similar content (Glaser and Strauss, 1967). These categories can identify root causes of a problematic situation that is described and elaborated through all the associated codes.

The participants in this study were selected from three rolling stock introductions. Prior to data collection, the participants received an explanation of the study. In order to identify the issues and challenges, the participants were asked to reflect on their current and past experiences in the context of rolling stock introductions framed by the boundary question. After collection of the data using open interviews, participants were asked to confirm their statements. Once the participants agreed on their statements, the authors started coding the transcriptions and related documentation using the qualitative software of Atlas.TI. This preliminary coding served as a starting point for the collective analysis with an expert group of six people. In this session, categories emerged from analyzing and comparing the preliminary coded statements, which is central to grounded theory and affinity diagramming. Categories emerge as part of the collective process of analyzing and comparing, rather than analyzing data based on predefined categories or hypotheses (Glaser and Strauss, 1967). After determining the categories, variables (e.g., degree of uncertainty) were identified for each category and inter-relationships were identified by counting the inbound and outbound arrows indicating which category influences the other more. Based on these relationships, events, patterns and underlying structures were identified which provide an indication where actions are most effective. After the expert session, findings were summarized and an affinity diagram was created for possible solutions to the boundary question.

4. Results and discussion

This section includes the case findings. After the introduction of the case, the results are presented following the stages as proposed by Brassard (1989).

4.1 Case introduction

The Nederlandse Spoorwegen (NS) is the main railway operator in The Netherlands, having the exclusive right to operate passenger trains on the Dutch main railway network until 2025. The Dutch state is their sole shareholder. NS, together with ProRail (in charge of the railway infrastructure in the Netherlands), needs to provide safe and reliable train services to its passengers. The supervision of the NS and ProRail is carried out by the Dutch Ministry of Infrastructure and Environment. Reliable train services are essential as former research showed that the reliability of service (being on time) is perceived as a decisive factor for public transport passengers' satisfaction (Bates et al., 2001; Edvardsson, 1998). However, as one of the former rolling stock introductions (SLT) showed, the NS was regularly confronted with the fact that the train, infrastructure and people were not ready at the moment the train was introduced in the timetable resulting in unreliable train services as illustrated in Figure 4 (Liefting, 2018). The figure shows that although the train was technically feasible for a commercial introduction, it takes a lot more to ensure reliable train services to its passengers from both operations and maintenance staff. Evaluations of the SLT showed (among others) an inadequate (control of) performance specifications (RAMS/LCC) and insufficient and late involvement of maintenance engineers in the system design and lay-out. Furthermore, the transfer of information and documentation from the suppliers for training and spare parts control was overdue.

The NS expected rolling stock shortages as of late 2016. In order to meet the concession requirements, set by the Ministry of Infrastructure and Environment on the main railway network, NS ordered new commuter (Flirt Fast Forward and Sprinter Next Generation) and Inter City (Inter City Next Generation) trains. Eventually, this may result in an expected 40% increase of rolling stock by 2022.

Therefore, introducing new rolling stock as well as ensuring reliable performance remains a challenge for the NS organization in the upcoming years and requires a “smooth introduction” of new rolling stock as termed by the organization itself. The term “smooth introduction” originated from the SLT introduction, but as findings in this section will show, was not equally interpreted. As one participant commented: “smooth introduction is just doing your work adequately.” Another interviewee, when asked for the definition of “smooth introduction,” said: “The objective is not only to have a train in time ready according to the contractual expectations, but the whole organization to be able to run a reliable timetable from day 1.” Other responses to the same question included: “No disruptions in train services as a result of new rolling stock by enhancing and controlling the five components of our maintenance framework: people, rolling stock, equipment and outillage, methods and concepts, tools and spare parts.” Another participant rephrased the term “smooth introduction” as: “A successful operation deployment from day one requires a train set that performs as contracted, an operator fully capable operating the train set and a maintainer fully capable of maintaining the train set. This process starts long before contracting the supplier, even if this seems far away from the moment of operational deployment of the new trains.”

But what constitutes a so-called “smooth introduction” and which factors can or should be managed to avoid or dealt with unwanted variance in performance? The next subsections present the findings of the case study based on three former rolling stock introductions: the SLT, the V250 and the FLIRT (Table 1). The SLT and the V250 are both examples of rolling stock introductions which had to be taken out of service due to unreliable performance after commercial introduction.

4.2 Case results

By adopting a system thinking approach using the affinity diagramming technique, issues and challenges were identified from different perspectives (and stakeholders), key elements were assessed and organized for being able to analyze the root causes and drivers. The remainder of this section is structured following the research process as introduced in Section 3. The research process consisted of six consecutive stages adopted and adapted from Brassard (1989):

1. Formulating the boundary question;

2. Identifying issues and challenges;

3. Categorize the issues and challenges using experts;

4. Defining variables for each category;

5. Linking categories and identifying events, patterns and underlying structures;

6. Answering the boundary question.

4.2.1 Formulating the boundary question and identifying issues and challenges (a, b)

Based on the purpose of a rolling stock introduction and the concerns in the case organization, the boundary question was formulated at the start of the study:

Which factors determine a reliable introduction of new rolling stock in an environment characterized by increasing complexity and uncertainty?

In order to identify the issues and challenges, participants were asked to reflect on their current and past experiences in the context of rolling stock introductions framed by the boundary question. 18 interviews were held with stakeholders in the case organization representing different perspectives on the topic to identify the issues and challenges of a reliable introduction. Initial coding of the transcriptions of the interviews resulted in 64 different codes based on 243 statements.

4.2.2 Categorizing issues and challenges and defining variables (c, d)

The initial coding served as input to an expert session with six stakeholders facilitated by the researchers. The 64 codes were grouped and categorized by the experts. Categories emerge as part of the collective process of analyzing and comparing, rather than analyzing data based on predefined categories or hypotheses (Glaser and Strauss, 1967). After determining the categories, variables were identified in the second round of the same session by the experts for each category (Table 2).

4.2.3 Linking categories and identifying events, patterns and underlying structures (e)

After identifying the variable for each category, the next step is to link the categories go each other. This involves the counting of the inbound and outbound arrows for every single category (Appendix). This way, the influence of the categories on each other can be identified. After identifying all inter-relationships, a hierarchy appeared listing the events (react), patterns (adapt) and underlying structures (prevent). In short, events are the things which happen on a day-to-day basis: e.g. a machine breaks down. Patterns of events are the accumulated memories of events. They reveal recurring patterns. Underlying structures can be perceived as “event generators” because they are responsible for producing the events. The purpose of the hierarchy is to identify and address issues and challenges on the right level and to focus efforts on those items which have a large impact on other items and thus, a first critical asset introduction framework emerged.

The key findings will be described below with a reference to the category and illustrated by statements from the participants. The groundedness of the system-levels (the number of statements from the participants in each system-level) showed a distribution of 43% of statements in the underlying structures-level (prevent), 32% of statements in the pattern-level (adapt) and 25% statements in the event-level (react). This indicates a strong focus of participants on issues and challenges in the “prevent” and “adapt” system levels.

4.2.3.1 React (events)

Findings showed the complexity (12) and uncertainty (6) involved in rolling stock introductions, partly as a result of the political context, national and European regulations, changing standards, complex railway systems, many stakeholders and the adoption of new (digital) technologies (3). As one participant stated: “an introduction on itself is not complicated, it was the required coherence among many entities which made it complex” (12). Infant mortalities were not only recognized in new technologies (3) but also in the organizations managing these technologies (6). One interviewee argued that infant mortalities are a fact of life, despite all the efforts and preparations. Awareness and accepting this fact, is thus essential for a reliable introduction (6). In the embedded case of the SLT introduction, the resource planning showed train drivers changing shifts to avoid driving the new (digital) train (3). Questions were also raised about the performance levels (2) and how to ensure the alignment between performance contracts between suppliers, operators and the concession agreement with the Dutch government (private-public relationships). Furthermore, the maintenance organization was characterized as a fire brigade, which makes pretty good sense in reacting on operational failures, but may have negatively influenced a good preparation for new rolling stock (2). Due to the perceived political pressures, senior management preferred decision-making with short term benefits above the long-term progress (2).

4.2.3.2 Adapt (patterns)

Findings illustrated the need to adapt since the rate of change increased (8) and the environment cannot be predicted so easily all the time. As one of the interviewees stated: “we have to stop navel gazing and restart scanning the environment more closely” (5). Participants also agreed on the (mis)alignment between the changing organization (project) and the running organization (operations) (11). As one of the participants stated: “the project organization had a heart rate of 140, the standing organization a heart rate of 40….” As failures were part of all introductions, accepting that failures occur and a silent introduction for getting used to the new equipment was highly recommended. Besides the absorptive capacity of individual entities and organizations (1), questions were raised about the collaboration and the maturity of stakeholders within the network chain (11). Improvisation capabilities were part of the operational culture, but had to be recalibrated as careful planning can also contribute to higher levels of predictability (4). However, findings also showed a disconnect between strategy (and planning) and the implementation of these strategies in the field (9).

4.2.3.3 Prevent (underlying structures)

Findings in underlying structures-level revealed several issues in mainly organizational systems. Traditional silo thinking (both intra- and interorganizational) prevented an integrated perspective on rolling stock introductions (14). There was a large consensus among participants that the culture was characterized by “management of the own stamp,” illustrating the limited perspective on the whole (15). Standardization, a well-known construct in aviation, was limited (10). A lack of investments in an early familiarization of maintenance and operations staff resulted in long term dependencies on the supplier (13). The acquisition and sharing of knowledge among stakeholders in the network chain of the introduction was lacking but considered essential (7). It was not only about the systems, but also about the data and the information required to function effectively. The participants on the whole stated the importance of integrality, as the railway system, and the introduction, must be considered as one integral system (14). As rolling stock introductions cover several years, reconfirming earlier assumptions during the introduction was required as resources changed frequently (13). The evolution of train designs had a great impact on maintenance. In the past, many working hours were related to preventive maintenance and most of the systems were located under the train. New trains have systems on the roof, more computerized systems and an increased adoption of IT in maintenance activities. Many systems are equipped with sensors that gives the opportunity of real time insight of the actual status of the trains which can eventually facilitate maintenance. For that reason, the preparation activities were focused on real-time monitoring of the diagnostic data of the train to increase knowledge (13). A common view amongst the participants was that, although technology is at its core, an introduction is in fact all about the people.

4.2.4 Answering the boundary question (f)

This subsection started with the boundary question of which factors determine a reliable introduction of new rolling stock in a context characterized by increasing complexity and uncertainty. 15 factors were identified as presented in the critical asset introduction framework (Figure 5). Based on the relationships between these factors, events, patterns and underlying structures were identified. This provides the case organization an indication where actions are most effective for reliable rolling stock introductions.

5. Theoretical and practical implications

This section discusses the theoretical and practical implications. What is surprising about the findings in this study is that technological systems, although generally perceived as the foundation of reliable train services was not considered a key factor in the exploration of a reliable introduction. It was mainly organizational issues and challenges that were stated as the key inhibitors. Therefore, the concept of “infant mortality” does not only adhere to technological systems (rolling stock), but should be extended to organizational systems as well. This supports the notion of a rolling stock introduction as a socio-technical system (Wilson et al., 2007). For a socio-technical system to remain reliable, it must somehow handle unforeseen situations in ways that forestall unintended consequences (Weick, 1987). Given the increasing complexity in rolling stock introductions, both technological as well as organizational, this might be one of the key challenges in future rolling stock introductions, and assumable, other critical asset introductions as well.

The findings of this study underline the evolution of the organizational role of the maintenance (asset management) function from a reactive, preventive, and predictive mode to a more holistic/process-oriented, complete, systematic organizational mode (Alsyouf, 2007). This was also observed by Simões et al. (2011) who identified a shift in maintenance performance measurement towards a more systematic, organizational perspective. Their literature review-based research revealed important themes related to evolution of maintenance performance management. These themes focus on the effective utilization of maintenance resources, information systems support, and human factor management. Research on human factors in asset management has mainly focused on human errors calculation models and methods. Even though social and organizational factors have received increasing attention in recent years, still studies of physical and mental stress, normal work, cultures of maintenance have been scarce (Sheikhalishahi et al., 2016). Research in other fields has repeatedly shown that it is often not only human error but also organizational factors that set up adverse conditions that increase the likelihood of system failure (Madni and Jackson, 2009). This corresponds to the system approach for human error as described by Reason (2000). The basic premise in the system approach is that humans are fallible, and errors are to be expected, even in the best organizations. Errors are seen as consequences rather than causes, having their origins not so much in the perversity of human nature as in “upstream” systemic factors. Countermeasures assume that though we cannot change the human condition, we can change the conditions under which humans work.

Although the findings of this study are specific for the case of rolling stock introductions and should be interpreted as such, this study confirms the importance to (re)consider organizational factors to be included when seeking for reliable performance in critical asset introductions. This observation in in line with other asset management research. For example, Gomes et al. (2020) found in their survey-based work among maintenance managers the beginning of the shift from a machine and plant-related performance perspective to a wider organizational orientation, which currently often leave maintenance managers still with many questions and few answers.

6. Limitations and future research

This study set out to explore the factors involved that determine how asset-intensive organizations can achieve reliable outcomes when introducing critical assets, despite random failures (e.g., as a result of early infant mortality failures) using a qualitative case study in railways. The study on three former introductions of rolling stock showed that despite intensive preparations, there were shortcomings resulting in unreliable train services and thus variances in performance. Findings showed that the introduction of new rolling stock was characterized by a strong focus on equipment and technology. However, the most obvious finding that emerged from the collective analysis was that organizational factors largely influenced the performance of the introduction. The introduction framework consists of 15 identified categories in three layers which embody a reliable introduction and may support asset-intensive organizations seeking reliable performance despite infant mortalities, to cope with an increasingly complex environment. As stated by Kumar et al. (2013), most maintenance (asset management) decision makers often come to the best conclusion using heuristics, backed up by qualitative assessment, supported by quantitative measures. This study can be considered a first qualitative contribution to decide on the factors to be considered (and managed) when introducing new critical assets.

The evaluation of the research process with the expert team showed that the collective analysis of the data contributed to a large extend to a better awareness and understanding of the factors involved. It supported discussions and showed multiple perspectives from 18 interviews and background documentation. This supports former research to adopt a multidisciplinary orientation in asset management (Ruitenburg et al., 2014; Van Dongen, 2015). The coding of the statements of the participants by the authors increased the speed of the team sessions by the experts, but in hindsight, may have framed the experts in their analysis on the data, limiting the opportunity to identify more factors involved.

As this study was a first exploration, future studies in railways should examine more closely the features and characteristics of rolling stock introductions, and identify the factors that determine the organizational capability to ensure and maintain reliability when facing the unexpected and show how this capability can be developed. Furthermore, future cases should also incorporate other industries, in order for being able to compare the results among critical asset introductions in general.

Figures

Figure 1

Asset lifecycle model

Figure 2

The “standard” bathtub curve based on Klutke et al. (2003)

Figure 3

Research process for problem identification based on the seven management and planning tools (Brassard, 1989)

Figure 4

The introduction of the Sprinter Light Train (Liefting, 2018)

Figure 5

A critical asset introduction framework for a reliable introduction with the hierarchy of events (react), patterns (adapt) and underlying structures (prevent) based on inter-relationships (the numbers represent the inbound arrows)

Figure A1

Determining inter-relationships by counting inbound and outbound arrows

Table 1

Rolling stock introductions incorporated in the explorative case study

Rolling stock introduction	Train type	First commercial introduction
SLT	Commuter train	2008
V250	Inter City train	2012
FLIRT	Commuter train	2016

Table 2

Identifying categories and extending the categories with variables

#	Category	Variable
1	Absorptive capacity	Degree of absorptive capacity
2	Performance	Level of performance (reliability)
3	Technology	Rate of technology adoption
4	Improvisation	Adoption of improvisation
5	Adaptability	Level of adaptability
6	Uncertainty	Degree of uncertainty
7	Knowledge	Disclosure of knowledge
8	Change	Ability to change
9	Strategy execution	Strength of strategy execution (alignment)
10	Standardization	Level of standardization
11	Chain collaboration	Extend of chain collaboration
12	Complexity	Degree of complexity
13	Learning	Rate of learning (maintenance and operations)
14	Integration	Power of integration (avoid silo thinking)
15	Ownership	Degree of ownership