Evaluation of Information System Investments: A Value Dials approach to closing the theory-practice gap

Abstract

Failed or partially failed information systems (IS) investments continue to be a problem in both public and private sectors. Although there are many causes of such failure, lack of clarity about the goals and benefits expected and how they are to be realized is one of the major contributors. This is particularly so in the public sector where the purpose of IS investment is not to increase profits, but to accomplish social and political goals. One way of reducing failure and wastage rates is to have ex-ante evaluation. Over the past three decades IS research has proposed a plethora of new theoretical approaches to improving ex-ante evaluation, but with a few exceptions, these have had little impact on practice. This paper presents a way to address this gap between theory and practice. It proposes a way in which managers can improve IS investment evaluation by changing perspective from a focus on traditional analytic tools towards a design attitude that seeks to develop multicriteria IS evaluation approach based on contextual experience and prior knowledge. The proposed approach enables organizations to design and develop a value case using value dials as a common language that defines what constitutes the value of particular IS investments for a specific context. To assess its effectiveness, this approach was applied using a multilevel dialogical action research project within a Swedish public organization. The findings show that managers in this organization significantly improved the effectiveness of their evaluation of IS investments.

Towards the Adoption of Machine Learning-Based Analytical Tools in Digital Marketing

“…Selecting the right analytical tool that will meet the requirements of data analysis is vital to ensure return on investment and the added value of the generated information. When developing the best marketing strategy the attention of responsible staff should be shifted from traditional analytical tools to advanced and specialized analytical tools [12]. This is especially important in marketing, where hundreds of potentially relevant metrics need to be considered.…”

Five Challenges for Benefits Management in Complex Digitalisation Efforts – and a Research Agenda to Address Current Shortcomings

“…When reviewing the history of the Information systems (IS) discipline, Hirschheim and Klein (2012) position benefits realisation as an extension of the IT evaluation literature. As pointed out by Frisk, Bannister and Lindgren (2015), the available literature of IT evaluation is extensive and thus too diverse to use as a basis for our research. Rather, we are focusing on one of the specific research streams related to IT evaluation, namely what is commonly referred to as benefits management (BM) or benefits realisation (BR).…”

“…Despite this, BR approaches have been adopted in practice, where the Benefits management model (BMM) has been the most influential (Mohan, Ahlemann and Braun, 2016;Waring, Casey and Robson, 2018). However, few empirical studies on how the BM process occur in practice can be found (Doherty, 2014;Frisk, Bannister and Lindgren, 2015), especially from public sector (Juell-Skielse, Lönn and Päivärinta, 2017) and inter-organisational collaborations (Lönn, Juell-Skielse and Päivärinta, 2016).…”

“…Digitalisation may drive increased collaboration among organisations in both public and private sectors (Boonstra and de Vries, 2008;Gil-Garcia, 2012;Garmann-Johnsen and Eikebrokk, 2014;van Fenema and Keers, 2018). However, such implementation of digital technologies are not straightforward and organisations struggle to achieve the intended outcome of their investments (Doherty, 2014;Frisk, Bannister and Lindgren, 2015;Mohan, Ahlemann and Braun, 2016;Christensen, 2017;Askedal, 2019). A large proportion of ICT efforts do not deliver expected benefits on time and on budget (Flak, 2012) resulting in loss of profit for private sector and public sector failure to accomplish societal and political goals (Frisk, Bannister and Lindgren, 2015).…”

Local Government Performance, Cost‐Effectiveness, and Use of the Web: An Empirical Analysis

“…This reflects in large part the difficulties in conducting effective studies of technological interventions in complex environments, which has generated an extensive literature on Information Systems evaluation approaches over multiple decades in both the public and private sectors. Where effort has been spent on the ex ante evaluation of investment decisions, a focus is possible on stakeholder involvement and both metric‐based and qualitative analysis of the particular problem for which information systems are being proposed to solve (see Frisk, Bannister, & Lindgren, ). The goals and metrics established can then be used after implementation to understand if the investment has achieved its goals.…”