To invest or not to invest in digital initiatives? An exploratory examination of procedures, evaluation criteria and barriers

Patrick Kraus (Business School, Hochschule Pforzheim, Pforzheim, Germany)
Julian Kappl (Business School, Hochschule Pforzheim, Pforzheim, Germany)
Dennis Schlegel (Faculty of Informatics, Hochschule Reutlingen, Reutlingen, Germany)

Digital Transformation and Society

ISSN: 2755-0761

Article publication date: 17 June 2024

Issue publication date: 30 September 2024

329

Abstract

Purpose

Due to the disruptive nature of digital transformation, firms can hardly ignore the further digitalisation of processes and business models. Implementing such initiatives triggers enormous investments in infrastructure and software, making the evaluation of digital investments crucial for a firm’s competitive situation.

Design/methodology/approach

Given the dynamics and uncertainties inherent in digital transformation, a qualitative, inductive research approach based on semi-structured interviews with high-level finance executives has been employed.

Findings

Our findings indicate widespread dissatisfaction with traditional investment appraisal methods for evaluating digital investments. Data also suggest that non-financial considerations are frequently taken into account, albeit implicitly, as participants struggled to clearly conceptualize these criteria.

Originality/value

The literature indicates important research gaps regarding the applicability and usage of traditional, predominantly financial, investment appraisal methods in digital contexts. This research enhances our understanding of digital investment evaluation, by (i) developing an exploratory conceptual framework of potential qualitative evaluation criteria and (ii) providing an in-depth and detailed understanding of the barriers to implementing investment appraisal methods.

Keywords

Citation

Kraus, P., Kappl, J. and Schlegel, D. (2024), "To invest or not to invest in digital initiatives? An exploratory examination of procedures, evaluation criteria and barriers", Digital Transformation and Society, Vol. 3 No. 4, pp. 410-423. https://doi.org/10.1108/DTS-02-2024-0004

Publisher

:

Emerald Publishing Limited

Copyright © 2024, Patrick Kraus, Julian Kappl and Dennis Schlegel

License

Published in Digital Transformation and Society. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode


Introduction

In recent years, the digital transformation of organisations has been a topic of intense discussion (Bounfour, Housel, Silkey, & Nonnis, 2023; Van Veldhoven & Vanthienen, 2023). Digital transformation, a multifaceted phenomenon, involves digitisation and digitalisation as technical and sociotechnical processes (Volpentesta, Spahiu, & De Giovanni, 2023). Digital technologies are key to this transformation and serve as major instruments for its implementation (Yang, Fu, & Zhang, 2021). In a similar vein, digital innovation requires digital technology as a prerequisite (Nambisan, Lyytinen, Majchrzak, & Song, 2017). According to a recent forecast by Gartner (2024), global information technology spending will increase to a total of five trillion USD in 2024. Due to these massive capital requirements, it is necessary to economically evaluate digital investments. While in traditional capital budgeting contexts, scholarly research has accumulated a profound stock of knowledge about common practices (Alkaraan, Cooper, Finkelstein, & Alkaraan, 2017; Harris & Al-Massri, 2011), significant research gaps exist regarding the evaluation of investments in information technology (IT) or information systems (IS). Applying traditional investment appraisal techniques in digital contexts may be problematic, as some argue that these methods are unsuitable (Irani, Ezingeard, & Grieve, 1997) due to the unique characteristics of digital investments and the difficulties in quantifying their benefits. Others report a theory-practice gap (Frisk, Bannister, & Lindgren, 2015) and a general lack of research on investment appraisal in digital contexts (Ylä-Kujala et al., 2023).

Hence, the main objective of our research is to enhance understanding of the evaluation of digital investments. Specifically, this involves (i) developing an initial exploratory conceptual framework of potential qualitative evaluation criteria for digital investments and (ii) providing an in-depth understanding of the barriers to implementing investment appraisal methods in this domain. This research aims to address critical gaps in both the accounting and the IS research field.

The remainder of this paper is structured as follows: First, we discuss the theoretical background of this research and review relevant literature. Next, we outline the research design. Subsequently, we present and critically discuss our findings, also in the context of previous research. Finally, we synthesize the contribution of this research and briefly indicate limitations and avenues for further research.

Theoretical background and literature review

Characteristics of digital investments

Regarding the terminology, the literature often refers to IT/IS investments (Irani, 2002; Irani et al., 1997) or ICT investments (Lefley, 2013; Lefley, Hynek, & Janeček, 2016). More recently, the terms digitalisation or digital transformation have gained prominence (Jones, Hutcheson, & Camba, 2021; Joppen, Lipsmeier, Tewes, Kühn, & Dumitrescu, 2019). For this reason, we prefer the term digital investments, which we use synonymously to the aforementioned terms. It is important to note that our research identified a predominance of investments in software, as will become evident later. Digital investments are distinct from traditional investments due to several key characteristics:

  1. The costs of digital initiatives are difficult to estimate due to their complexities and unforeseeable developments. Often, related costs cannot be allocated to a single initiative and thus impact various functional areas (Irani et al., 1997).

  2. The failure rate of IS projects has traditionally been very high.

  3. The benefits of digital investments can hardly be predicted due to their qualitative nature. Digital initiatives may also be of strategic nature and relate to multiple projects or areas, making it difficult to quantify their benefits monetarily.

  4. Due to innovative and rapid developments, digital investments have the risk of becoming outdated prematurely, which is hard to predict.

We argue that these characteristics need to be carefully considered when evaluating digital investment alternatives. In traditional decision situations, investment appraisal methods provide valuable support. However, some scholars argue that traditional appraisal techniques are no longer suitable (Irani et al., 1997). Others contend that while traditional methods are still valid, the context has changed due to digitalisation. We move now on to critically discuss the current state of investment appraisal in the context of digital initiatives.

Status quo of investment appraisal research in the context of digital transformation

There is a vast array of investment appraisal methods (Brealey, Myers, Allen, & Edmans, 2022). However, regarding digital investments, the literature is relatively scarce; despite the significant attention digital transformation has received from scholars (e.g. Ali, Green, & Robb, 2015).

One research stream focuses explicitly on business cases for IT investments. Ward, Daniel, and Peppard (2008) found that most organisations are dissatisfied with the approaches to creating a business case and suggest a stepwise approach that considers both financial and subjective benefits. Giannoutakis and Li (2012) developed a framework for building business cases in the context of intelligent transport systems and Colli, Stingl, and Waehrens (2022) created a framework that includes various value dimensions of Industry 4.0 initiatives, allowing for a more balanced evaluation of digital transformation projects. Joppen et al. (2019) designed a business case calculator for Industry 4.0 initiatives, and Einhorn, Meredith, and Marnewick (2022) proposed an approach that supports business case considerations throughout the IT project lifecycle.

Another research stream critically examines traditional investment appraisal methods and seeks to refine or develop specific instruments to evaluate digital investments (e.g. Irani & Love, 2002; Walther & Spitta, 2004). Irani and Love (2002) provide taxonomy of investment appraisal techniques for IT/IS investment evaluation. Walther and Spitta (2004) critically reviewed methods from the literature and examine empirical studies. Frisk et al. (2015) propose a multi-criteria approach based on contextual experience to close the theory-practice gap. Moreover, Irani and Love (2002) highlighted the importance of intangible and non-financial benefits, especially over longer periods, while Walther and Spitta (2004) argued that such qualitative factors are insufficiently considered by most investment appraisal techniques. Almeida, Ayala, Benitez, KliemannNeto, and Frank (2023) developed an investment appraisal framework for Industry 4.0 technologies, incorporating a multiple-criteria decision making approach that includes economic, financial and sociotechnical aspects within the measurement process of potential technologies. Ylä-Kujala et al. (2023) applied a net present value approach to evaluate robotic process automation technologies, relying solely on quantitative figures. While the distinctiveness of ICT and non-ICT investments is generally supported, traditional investment appraisal methods are still frequently used, which may explain why many firms are dissatisfied with their approaches (Ward et al., 2008). However, among others, Brown (2005) found that the slow adoption of various developed evaluation tools is due to (i) a lack of staff resources and specialised skills and (ii) challenges related to change management and static company cultures.

Implementing digital initiatives involves uncertainties and risks (Almeida et al., 2023; Buhulaiga & Telukdarie, 2024) and firms seem to encounter barriers during these change processes (Kumar, Salmona, Berry, & Grummert, 2023). Specifically, regarding investment appraisal, the review of the literature revealed important research gaps, which will be addressed by this paper. First, there is a lack of recent empirical research on digital investment appraisal practices, which is also supported by Ylä-Kujala et al. (2023). Consequently, there is a limited understanding of why and how traditional investment appraisal techniques are used or not, the challenges involved, and how benefits and costs are estimated. Second, it is evident that benefits of digital investments are often not purely financial but relate to various areas (Irani & Love, 2002; Walther & Spitta, 2004). Therefore, these qualitative aspects should also be considered in the evaluation of digital investments, but the literature provides little information on concrete qualitative evaluation criteria. Finally, there seems to be a considerable gap between theory (the availability of instruments) and practice (their application), with the literature reporting only selective experimental implementations of adapted tools, e.g. by Almeida et al. (2023) and Ylä-Kujala et al. (2023).

Research design and methods

Given the high degree of novelty and hence also vagueness and ambiguity (Hanelt, Bohnsack, Marz, & Antunes Marante, 2021) in the field of digital transformation and the evaluation of digital investments (Irani, 2002), a flexible research approach is required, which allows for developing a detailed and in-depth understanding of the phenomena under investigation. Based on these characteristics, a qualitative research approach has been selected (Cassell & Symon, 2012). Interview methods, in particular, represent a powerful means of generating new knowledge (Kvale & Brinkmann, 2009), and are frequently used in novel and dynamic contexts (Cannas, 2021; Faro, Abedin, & Cetindamar, 2021; Tuukkanen, Wolgsjö, & Rusu, 2022). Semi-structured interviews, specifically, are appropriate as they provide flexibility while ensuring that all relevant aspects are covered.

In total, five semi-structured interviews were conducted with experts in management positions responsible for digital investments in major family firms within the manufacturing industry in southwest Germany. The sample was purposively constructed, and the high hierarchical positions of the participants within their organisations ensured the gathering of meaningful and rich data. The sample size of five interviews is comparable to similar studies in this domain, which typically use samples of around five to ten informants (Schneider & Kokshagina, 2021; Tuukkanen et al., 2022), and is considered adequate, especially given the influential positions of participants and the objective to develop an in-depth understanding of digital investment appraisal methods. Interviews have been conducted via MS Teams, which, due the challenges imposed by the COVID-19 pandemic, is an adequate option for the collection of primary data (Gray, Wong-Wylie, Rempel, & Cook, 2020). The composition of the sample is illustrated in Table 1.

The data was transcribed and subsequently analysed using a thematic coding approach (King, 2012). This approach did not overly rely on strict protocols and procedures (Harley & Cornelissen, 2022) to support the identification of novel and fresh patterns and ideas. However, data analysis was done iteratively, involving discussions and critical evaluations by all authors, as well as phases of critical reflection on the developing coding scheme. The final coding scheme encompasses two hierarchical levels and is illustrated in Table 2.

Findings and discussion

Procedural handling of digital investments

First, it is important to understand whether there are procedural differences in the decision-making processes for digital versus traditional investment proposals. The data indicate a heterogenous view on this, as one participant states:

Basically, we do have the same proposal processes as we have with other investments. (IC2)

Here, it seems that while participants are aware of the different types of investments, the process of proposing and applying for approval is similar. Given the distinctiveness of digital investments decisions compared to non-digital ones, this necessitates a nuanced approval process, for evaluating digital investments, which is seen as problematic.

This means the type of the investment is different; it has a completely different nature. Consequently, it must be evaluated differently. (IC3)

That is a completely different steering of investments. Concurrently, it is difficult to evaluate the overall benefit of a digital solution individually. (IC2)

This relates to the fact that digital investments typically affect the firm broadly or partially, in the sense that several organisational units might experience changes in procedures. This also indicates that a variety of people and corporate functions are typically involved.

… the complexity is different. Because different interfaces, different contents, and different people are involved and need to be brought together. (IC2)

Given the distinct characteristics, impact, and relevance digital investments demonstrate, also from a technical perspective, in some firms, the approval of such investments by an additional board is mandatory. This process emphasises the interconnections inherent to digital investments and upcoming digital projects involving various stakeholders.

The approval process includes the decision of an additional board. As soon as there is a certain digitalisation share, for instance, an investment in the area of software, this IT board looks at the intended project, evaluates and approves it, whether it makes sense as proposed or not. (IC4)

… This means that for IT investments, we have an advisory board, I would name it like that. There, a group of people familiar with the subject comes together. This group discusses the proposed digital investments, but basically, this is a qualitative discussion. (IC3)

Due to the technical dynamics, complexities and compatibility with the overall hardware and software landscape, the involvement of the IT department is an important prerequisite for the success of digitalisation initiatives. Overall, it can be observed that most firms are aware of the distinctiveness of digital investments, and such investments are treated separately regarding the approval processes.

Evaluation of digital investments

Regarding the analysis and evaluation of digital investments, especially in the context of a profitability assessment, it was found that participating firms apply heterogeneous approaches tailored to their specific needs. Overall, it can be observed that firms do not apply a purely economic perspective, which might be explained by the distinctiveness of digital investments.

… then, frequently not only the economic criteria play a major role. (IC1)

I do not look at them. I do not check whether there is an advantageousness, I say, I need this. I would like to have that. Then we implement and do that. (IC1)

I do not conduct an investment appraisal for digitalisation projects. (IC1)

The elaborations of the above participant indicate a tendency towards an unstructured, unsystematic decision-making process.

Another approach is to categorise investments into projects that provide a financial benefit and those that do not. Regarding the projects indicating a financial benefit, the firm is currently trying to improve the quantification of the benefit in the sense of better understanding the input factors. Prioritisation of investments is primarily based on a business case logic, specifically using a discounted cash flow method.

Well, we do have projects that are pure cost projects; these are described simply as cost cases. We also have project that have a benefit share; they somehow generate value. We are now really trying to improve this area of benefit management. (IC4)

While it can be observed that there is an aim to apply a business case logic in decision-making and improve the related approaches, many investments still seem to be undertaken not solely based on financial logic but also on other relevant criteria. However, the data also show that there are very advanced and systematic approaches, but not necessarily in in terms of financial valuation. First, digital investments are categorised, and infrastructural enabler investments are differentiated from complementary investments.

Principally, we separate generic enablers from individual solutions, on which we then can also decide individually. The infrastructural contributions can then be regarded as some kind of digitalisation flat rate across our overall portfolio. (IC2)

In accordance with the above-identified importance of qualitative factors, it is also emphasised that these factors can be decisive.

When we cannot calculate it, then mostly soft facts or qualitative factors are used.’ (IC2)

Consequently, given the importance of qualitative factors, the method of utility analysis is regularly applied by three companies (IC2, IC3 and IC4), while IC5 is reserved, referring to the considerable effort required for the constant application of the instrument.

Ultimately, we considered the effort too high to use this instrument for every investment or even for some of the investments. (IC5)

Besides this, none of the participating firms is planning to implement further specific instruments in the process of prioritising digital investments, as these are considered per se too complex. However, none of the participants could name further instruments such as the real options model. It was also not observed that participants use specific instruments to consider uncertainties or risks of digital investments. Regarding this, participants refer to the consideration of risks during project management of digital projects, rather based on qualitative evaluations or an additional buffer, as indicated by one participant:

‘I believe that the danger is clear to the people, especially in the IT. But besides assuming a general additional buffer of 15 per cent for security reasons regarding the project duration, we do nothing, and nothing is planned in the near future. (IC4)

In general, there are few concrete indications of how risks are considered in the investment appraisal considerations for digital investments. Another participant (IC3) elaborated on the impact of a new software solution on different functional areas, such as production, and the probability of disruption of production processes. However, discussions on this remained rather holistic. As an intermediate result, it can be concluded that participating firms are aware that there is considerable potential for improvement regarding the evaluation of digital investments. Compared to non-digital investment cases, firms seem to assess the applied approaches as weaker, given the specific challenges associated with digitalisation. Few firms even refer to a learning phase; however, most of the participants are not planning to considerably improve their methodological approaches or introduce new instruments. In conclusion, an important finding has been the high importance of qualitative factors when evaluating digital investments, as opposed to economic or financial indicators. Therefore, this aspect will be discussed in more depth in the following section.

Conceptualizing qualitative criteria in the context of digital investments

While the relevance of non-financial criteria has also been addressed earlier (Irani & Love, 2002), the identification of its outstanding relevance as a common denominator among the interviewees, as well as the systematization of qualitative indicators, is an important contribution of this research. In contrast to theoretical suggestions in the Finance discipline, even when financial factors tend to indicate a negative outcome, firms may decide in favour of such investments.

Well, for us, it is indeed important to be a technological leader, and the firm is willing to make such decisions under uncertainty and under non-existent economic aspects. (IC2)

There are projects, which make sense, even when they indicate a negative discounted cash flow. As soon as the qualitative benefit is so outstanding that in the future it would lead to a negative contribution of profit, then you would do that. (IC3)

The participating firm (IC4) echoes the above sentiment:

Yes, definitely. Net present value is a key performance indicator but not the final indicator on which a decision is made. (IC4)

Data indicate that success and benefit are defined differently and not reduced to a pure profit indicator, but firms do expect economic benefits, especially in the long-term. This might be explained by the fact that the sample firms consist of family firms, which tend to have a stronger long-term focus.

While the importance of qualitative evaluation criteria is a dominant theme within the data, it must be noted that defining the specific qualitative criteria was hardly possible for many participants. This leads to the conclusion that in many cases, an implicit and intuitive consideration of such criteria is taking place, which is an important finding of this research. This is especially significant as the discipline of management control and the positions of the participants are mostly related to a prosaic, fact-oriented, and meticulous mentality. Recently there has been a call for more intuitive trial-and-error approaches in the context of digitalisation within the discipline (Schäffer & Weber, 2019).

On an aggregated basis, the following major qualitative criteria can be distilled from the data and can be considered within the evaluation process of digital investments, as shown in Figure 1.

The fear of losing competitive advantage to major competitors is a pressing issue due to the dynamics and changes induced by digital transformation (Kumar et al., 2023). The inclusion of qualitative factors may support the consideration of long-term benefits and prevent an overly short-term focus on monetary aspects, which hinders the long-term development of the firm. Especially, in the context of family firms, this long-term orientation is an important and dominant theme (Koiranen, 2002). This is reflected in the following quote:

It would have been good to start earlier and gather our own experiences because we can see the fear of making mistakes during the transformation process. The primary mistakes, however, are that we say we are not willing to make an investment of 20, 30 or 40 million EUR, because we do not know when it will amortize. But with 5 billion in revenues, not to make an investment of 40 million is almost negligent. If you then have to say afterward that we have lost our 5 years advantage and the customers criticise features of our products and we do not have the same sales growth we had in the past. (IC2)

While several general qualitative criteria could be identified, it must also be noted that the criteria could be specific to the individual situation, as one participant indicates:

This is very project-specific. Quite often, for instance, you have something like those processes just became less complicated. (IC4)

Barriers to the implementation of investment appraisal in the context of digital investments

To explain the methodological shortcomings in the evaluation of digital investments, our participants indicated several barriers to the continuous improvement, development, and application of investment appraisal approaches, as illustrated in Figure 2. In a similar vein to Jones et al. (2021), we understand a barrier as a difficulty to overcome to implement or apply something.

We have identified several clusters of barriers in the data, which refer to methodological, functional, managerial, and external barriers.

Within the functional barriers we have identified limited capacity, a frequently identified issue in comparable studies (Brown, 2005). Moreover, a lack of knowhow about digital investments has been identified as a critical issue. This may also explain the limited involvement of the management control function in digital investment decision-making. This seems to be actively addressed by participants, as the following quote indicates:

Besides that, we try to build up such knowhow. If we have not done much in this area, with all the new possibilities that exist, then there consequently is a lack of knowhow which we need to acquire. And developing this knowhow is done extensively. (IC3)

Furthermore, there are managerial barriers that lead to a limited involvement of the management control function in digital investment decision-making.

But controlling was not involved; a decision paper was prepared by the IT department and the executive board assessed the project on this basis. (IC5)

This can be considered problematic, as ensuring rational decisions can be seen as a core objective of management control (Schäffer & Weber, 2019). The limited involvement of the management control function can also be interpreted as a fundamental general management decision, as the allocation of resources, knowhow and hence capacity is at the very core of such decisions. Participants even speak to a certain degree of a disinterest from the general management function in topics related to investment appraisal, which may lead to the false application of instruments resulting in wrong decisions that considerably and sustainably harm the transformation of organisations.

The decision-maker finally determines the quality that is implemented in a company. This is determined by the number of people, the capacity which is allocated to a specific function. And if there is not enough capacity, you simply cannot do certain things. If these things are also not demanded, then finally the organisation does not care whether there is a deficiency, for instance, in a utility analysis for a digital investment or not. (IC1)

The participant from company IC1 refers to an important finding of this research, which requires further scholarly investigation to better understand how general top management behaviours affect the digital transformation of organisations. While the literature frequently describes that digital transformation has developed into a topic intensively discussed at the top management level (Autio, Mudambi, & Yoo, 2021; Margiono, 2020; Peter, Kraft, & Lindeque, 2020), it is important to keep in mind that organisations must be prepared to operationally manage the transformation. Given the limited progress and success of many digital transformation initiatives (Beer, Fridgen, Mueller, & Wolf, 2013; Lefley et al., 2016), an important reason for this could be the inherent conflict between top management attention and limited internal capacity within various organisations.

Moreover, there are indications in the data that participants experience considerable methodological barriers. A participant refers to a certain complexity and efforts in relation to investment appraisal calculations, as stated in the following quote:

‘This certainly would make sense, to a certain degree, but as I said rather on an administrative level. Not in the sense of profitability, because you could intensively calculate that up and down, but I do not have time for that. I do not want to do that. (IC1)

Generally, the limited practicality of instruments and methods is a prevalent topic, also identified regarding a variety of other management instruments in previous research (Frisk et al., 2015). This has also been observed for traditional models managing, for instance, digital innovation (Nambisan et al., 2017). This especially emphasises the requirement to develop technically useful and tailored but at the same time pragmatic and usable instruments for practitioners. This can be vividly illustrated by the following quote:

Probably, this is because all the existing methods and tools might be interesting and quite good regarding the underlying theory. But finally, if they are not considered as important by practice and nobody cares, then they are simply irrelevant. (IC1)

This indicates that investment appraisal methods are not regarded as irrelevant per se but that there is a gap between theoretical developments by scholars and the implementation of these instruments by companies. The degree of digitalisation of investment appraisal methods themselves has been identified as another critical aspect, regarding, for instance, the availability of data and system support, as indicated in the quote below:

Basically, it would be good to have more support by systems, in all areas. (IC4)

Hence, there seems to be a dual challenge: the management control function itself needs to change tremendously by means of digital technologies (Möller, Schäffer, & Verbeeten, 2020) while also coordinating the transformation of organisations in the course of digitalisation (Schäffer & Weber, 2019). A difficult issue tends to be the quantification of benefits of digital investments, as also indicated by others (Irani & Love, 2002). This seems to be addressed quite intensively by participating firms, as indicated below:

Well, I can only say the costs we have for that, to define the potential benefits, is difficult. We must develop that from year to year, and extend that step by step, so that we can show the potentials. We need another one to two years to better understand which potentials we want to use and how to prioritize that. (IC2)

Given the enormous complexities and dynamics within the field of digital transformation (Sebastian et al., 2017), participants also experience such complexities regarding technical solutions forming the base of digital transformation, for instance, software solutions. We have described this lack of market transparency as external barrier. Indeed, there is an enormous variety of potential solutions, and the participants indicate that providers of digitalisation solutions intentionally exploit the lack of transparency, and that there is a higher degree of uncertainty in such markets compared to more traditional ones.

There are certain restrictions on the markets, which consequently are used by the various providers. Market barriers are considerably higher in such digitalisation projects. (IC3)

While some of the barriers are inherent to the overall nature of digital transformation and the related complexities, others must be consistently addressed by the management control function to ensure rational decision-making. This requires adequate capacity in terms of manpower, knowhow and capabilities, relevant to digitalisation.

Conclusion

In this paper, we have shown that digital investments have specific characteristics compared to traditional investments. Our research contributes to knowledge by providing new empirical data on practitioners’ perspectives and actual behaviour in terms of evaluating digital investments. As there has been identified a paucity of research in this domain, we address an important gap in the literature, especially given the enormous investments in digital initiatives. We have presented a framework of criteria to qualitatively evaluate digital investments, as well as a structured overview of barriers to the implementation of the methods. Our model of qualitative factors enriches the field by proposing a multidimensional approach to evaluate digital investments beyond traditional financial metrics. It recognizes various dimensions, contributing a comprehensive lens through which digital investments can be qualitatively evaluated. As our model considers employee-related issues alongside technical issues, it bridges the gap between the human aspects of digital investments and their technical requirements, thereby reflecting the complexities of digital investments.

Our results have important theoretical and managerial implications. They show that traditional investment appraisal theory may need to be amended to account for the specific characteristics of digital investments. At the same time, they provide practical guidance for organisations on how to evaluate investments in digitalisation. Managers can use our model to ensure that digital investments are aligned with their departments' roles and tasks, thereby optimizing resource allocation and strategic fit of initiatives. Considering employee-related issues emphasises the need for managers to invest in training and development as part of the digital investment process. The identified barriers may help managers improve the investment appraisal process, for instance, by increasing the communication between the different functional units.

Our work certainly has some limitations. The present study is based on a sample size that may not be generalisable to all firms due to its qualitative and exploratory nature. Therefore, we recommend further corroboration of the results by using additional qualitative approaches such as case studies and quantitative research designs. Additionally, applying the identified qualitative factors to specific investment cases could enhance the model. In particular, integrating these factors with traditional financial methods offers a valuable approach.

Figures

Conceptual framework of qualitative evaluation criteria

Figure 1

Conceptual framework of qualitative evaluation criteria

Barriers for improving digital investment appraisal methods and approaches

Figure 2

Barriers for improving digital investment appraisal methods and approaches

Overview on the sample

InterviewPositionHeadcountRevenueDuration
IC1Head of Group Controlling15,000∼2 billion EUR41 min
IC2Head of R&D Controlling20,000∼5 billion EUR50 min
IC3Head of Controlling4,400∼1 billion EUR52 min
IC4Employee Operational Excellence15,000∼4 billion EUR57 min
IC5Head of Production and R&D Controlling5,400∼1 billion EUR38 min

Source(s): Own illustration

Theme-related coding scheme

Main themeSub-theme
Digitalisation projectsImportance of digitalisation
Objectives of digitalisation
Digital investmentsTreatment of digital investments
Prioritization of digital investments
Uncertainties and risks
Evaluation of digital investment appraisal
Qualitative factorsWeighting of quantitative and qualitative indicators
Decisive qualitative indicators
OutlookImprovement of digital investment appraisal
Barriers

Source(s): Own illustration

References

Ali, S., Green, P., & Robb, A. (2015). Information technology investment governance: What is it and does it matter?. International Journal of Accounting Information Systems, 18, 125. doi: 10.1016/j.accinf.2015.04.002.

Alkaraan, F. E., Cooper, C. L., Finkelstein, S., & Alkaraan, F. (2017). Strategic investment appraisal: Multidisciplinary perspectives. In C. L. Cooper, & S. Finkelstein (Eds), Advances in Mergers and Acquisitions (Vol. 16, pp. 6782). Bingley: Emerald. doi: 10.1108/s1479-361x20170000016004.

Almeida, R. P., Ayala, N. F., Benitez, G. B., Kliemann Neto, F. J., & Frank, A. G. (2023). How to assess investments in industry 4.0 technologies? A multiple-criteria framework for economic, financial, and sociotechnical factors. Production Planning and Control, 34(16), 15831602. doi: 10.1080/09537287.2022.2035445.

Autio, E., Mudambi, R., & Yoo, Y. (2021). Digitalization and globalization in a turbulent world: Centrifugal and centripetal forces. Global Strategy Journal, 11, 114. doi: 10.1002/gsj.1396.

Beer, M., Fridgen, G., Mueller, H.-V., & Wolf, T. (2013). Benefits quantification in IT projects. Wirtschaftsinformatik Proceedings, 45, 707720.

Bounfour, A., Housel, T., Silkey, T., & Nonnis, A. (2023). Digital transformation and strategic agility during the COVID-19 crisis: The role of the intangible capital conversion. Digital Transformation and Society, 2(3), 257275. doi: 10.1108/DTS-09-2022-0048.

Brealey, R., Myers, S., Allen, F., & Edmans, A. (2022). Principles of corporate finance (14th ed.). New York: McGraw-Hill.

Brown, A. (2005). IS evaluation in practice. Electronic Journal Information Systems Evaluation, 8(3), 169178.

Buhulaiga, E., & Telukdarie, A. (2024). Industry 4.0 implementation for multinationals: A case study. Digital Transformation and Society, ahead-of-print(ahead-of-print). doi:10.1108/DTS-07-2023-0057.

Cannas, R. (2021). Exploring digital transformation and dynamic capabilities in agrifood SMEs. Journal of Small Business Management, 61(4), 16111637. doi: 10.1080/00472778.2020.1844494.

Cassell, C., & Symon, G. (2012). Introduction: The context of qualitative organizational research. In G. Symon, & C. Cassell (Eds), Qualitative Organizational Research: Core Methods and Current Challenges (pp. 111). London: Sage.

Colli, M., Stingl, V., & Waehrens, B. V. (2022). Making or breaking the business case of digital transformation initiatives: The key role of learnings. Journal of Manufacturing Technology Management, 33(1), 4160. doi: 10.1108/jmtm-08-2020-0330.

Einhorn, F. D., Meredith, J., & Marnewick, C. (2022). A model to guide the effective use of a business case for IT projects. International Journal of Managing Projects in Business, 15(1), 3657. doi: 10.1108/ijmpb-03-2021-0076.

Faro, B., Abedin, B., & Cetindamar, D. (2021). Hybrid organizational forms in public sector’s digital transformation: A technology enactment approach. Journal of Enterprise Information Management, 35(6), 17421763. doi: 10.1108/JEIM-03-2021-0126.

Frisk, J. E., Bannister, F., & Lindgren, R. (2015). Evaluation of information system investments: A value dials approach to closing the theory-practice gap. Journal of Information Technology, 30(3), 276292. doi: 10.1057/jit.2014.9.

Gartner (2024). Gartner forecasts worldwide IT spending to grow 6.8% in 2024. Available from: https://www.gartner.com/en/newsroom/press-releases/01-17-2024-gartner-forecasts-worldwide-it-spending-to-grow-six-point-eight-percent-in-2024 (accessed 4 April 2024).

Giannoutakis, K. N., & Li, F. (2012). Making a business case for intelligent transport systems: A holistic business model framework. Transport Reviews, 32(6), 781804. doi: 10.1080/01441647.2012.740096.

Gray, L. M., Wong-Wylie, G., Rempel, G. R., & Cook, K. (2020). Expanding qualitative research interviewing strategies: Zoom video communications. The Qualitative Report, 25(5), 12921301. doi: 10.46743/2160-3715/2020.4212.

Hanelt, A., Bohnsack, R., Marz, D., & Antunes Marante, C. (2021). A systematic review of the literature on digital transformation: Insights and implications for strategy and organizational change. Journal of Management Studies, 58(5), 11591197. doi: 10.1111/joms.12639.

Harley, B., & Cornelissen, J. (2022). Rigor with or without templates? The pursuit of methodological rigor in qualitative research. Organizational Research Methods, 25(2), 239261. doi: 10.1177/1094428120937786.

Harris, E. P., & Al-Massri, M. (2011). Capital investment appraisal. In M. G. Abdel-Kader (Ed.), Review of Management Accounting Research (pp. 343377). London: Palgrave Macmillan.

Irani, Z. (2002). Information systems evaluation: Navigating through the problem domain. Information & Management, 40(1), 1124. doi: 10.1016/s0378-7206(01)00128-8.

Irani, Z., & Love, P. E. D. (2002). Developing a frame of reference for ex-ante IT/IS investment evaluation. European Journal of Information Systems, 11(1), 7482. doi: 10.1057/palgrave/ejis/3000411.

Irani, Z., Ezingeard, J.-N., & Grieve, R. J. (1997). Integrating the costs of a manufacturing IT/IS infrastructure into the investment decision-making process. Technovation, 17(11), 695706. doi: 10.1016/s0166-4972(97)00060-6.

Jones, M. D., Hutcheson, S., & Camba, J. D. (2021). Past, present, and future barriers to digital transformation in manufacturing: A review. Journal of Manufacturing Systems, 60, 936948. doi: 10.1016/j.jmsy.2021.03.006.

Joppen, R., Lipsmeier, A., Tewes, C., Kühn, A., & Dumitrescu, R. (2019). Evaluation of investments in the digitalization of a production. Procedia CIRP 81, 81, 411416. doi: 10.1016/j.procir.2019.03.071.

King, N. (2012). Doing template analysis. In G. Symon, & C. Cassell (Eds), Qualitative Organizational Research: Core Methods and Current Challenges (pp. 426450). London: Sage.

Koiranen, M. (2002). Over 100 Years of age but still entrepreneurially active in business: Exploring the values and family characteristics of old Finnish family firms. Family Business Review, 15(3), 175187. doi: 10.1111/j.1741-6248.2002.00175.x.

Kumar, A., Salmona, M., Berry, R., & Grummert, S. (2023). Digital transformation preparedness: An exploratory study. Digital Transformation and Society, 3(2), 127144. doi: 10.1108/dts-07-2023-0051.

Kvale, S., & Brinkmann, S. (2009). InterViews: Learning the craft of qualitative research interviewing (2nd ed.). Thousand Oaks et al: Sage.

Lefley, F. (2013). The appraisal of ICT and non-ICT capital projects: A study of the current practices of large UK organisations. International Journal of Managing Projects in Business, 6(3), 505533. doi: 10.1108/ijmpb-04-2012-0010.

Lefley, F., Hynek, J., & Janeček, V. (2016). Formal assessments, teams and influence of a project champion in the selection of ICT projects: A Czech republic and UK study. Prague Economic Papers, 25(2), 143159. doi: 10.18267/j.pep.551.

Margiono, A. (2020). Digital transformation: Setting the pace. Journal of Business Strategy, 42(5), 315322. doi: 10.1108/JBS-11-2019-0215.

Möller, K., Schäffer, U., & Verbeeten, F. (2020). Digitalization in management accounting and control: An editorial. Journal of Management Control, 31(1-2), 18. doi: 10.1007/s00187-020-00300-5.

Nambisan, S., Lyytinen, K., Majchrzak, A., & Song, M. (2017). Reinventing innovation management research in a digital world. MIS Quarterly, 41(1), 223238. doi: 10.25300/misq/2017/41:1.03, Management Information Systems Research Center, University of Minnesota.

Peter, M. K., Kraft, C., & Lindeque, J. (2020). Strategic action fields of digital transformation: An exploration of the strategic action fields of Swiss SMEs and large enterprises. Journal of Strategy and Management, 13(1), 160180. doi: 10.1108/jsma-05-2019-0070.

Schäffer, U., & Weber, J. (2019). Digitalization will radically change controlling as we know it. In U. Schäffer (Ed.), Behavioral Controlling: Anniversary Volume in Honor of Jürgen Weber (pp. 159168). Wiesbaden: Springer.

Schneider, S., & Kokshagina, O. (2021). Digital transformation: What we have learned (thus far) and what is next. Creativity and Innovation Management, 30(2), 384411. doi: 10.1111/caim.12414.

Sebastian, I., Ross, J., Beath, C., Mocker, M., Moloney, K. & Fonstad, N. (2017), “How big old companies navigate digital transformation”.

Tuukkanen, V., Wolgsjö, E., & Rusu, L. (2022). Cultural values in digital transformation in a small company. Procedia Computer Science, 196, 312. doi: 10.1016/j.procs.2021.11.066.

Van Veldhoven, Z., & Vanthienen, J. (2023). The role of trade associations in the digital transformation of their industry. Digital Transformation and Society, 2(3), 235256. doi: 10.1108/DTS-11-2022-0058.

Volpentesta, T., Spahiu, E., & De Giovanni, P. (2023). A survey on incumbent digital transformation: A paradoxical perspective and research agenda. European Journal of Innovation Management, 26(7), 478501. doi: 10.1108/EJIM-01-2023-0081.

Walther, S. G., & Spitta, T. (2004). Approaches to the ex-ante evaluation of investments into information systems. Wirtschaftsinformatik, 46(3), 171180. doi: 10.1007/bf03250934.

Ward, J., Daniel, E., & Peppard, J. (2008). Building better business cases for IT investments. MIS Quarterly Executive, 7(1), 115.

Yang, M., Fu, M., & Zhang, Z. (2021). The adoption of digital technologies in supply chains: Drivers, process and impact. Technological Forecasting and Social Change, 169, 120795. doi: 10.1016/j.techfore.2021.120795.

Ylä-Kujala, A., Kedziora, D., Metso, L., Kärri, T., Happonen, A., & Piotrowicz, W. (2023). Robotic process automation deployments: A step-by-step method to investment appraisal. Business Process Management Journal, 29(8), 163187. doi: 10.1108/BPMJ-08-2022-0418.

Corresponding author

Patrick Kraus can be contacted at: research@patrickkraus.net

Related articles