A Method for Assessing Influence Relationships among KPIs of Service Systems

Abstract

The operational performance of service systems is commonly measured with key performance indicators (KPIs), e.g., time-to-resolve, SLA compliance, and workload balance. The assumption is that healthy KPIs lead to healthy business outcomes such as customer satisfaction, cost savings, and service quality. Although the domain experts have an intuitive understanding of the causal relationships among the KPIs, the degree of influence a cause KPI has on the effect KPI is difficult to estimate based on intuition. Also, the intuitive understanding could be wrong. Further, we show how the causal relationships are intricate with aspects such as the rate of influence and conditionality in measurements. As a result, although crucial, it is nontrivial to estimate the degree of influence. Without the degree of influence, prediction of business outcomes and decisions based on them tend to be ad hoc. This paper presents a novel method for validating the intuitive direction and the polarity of a causal relationship provided by domain experts. Further, the method also estimates the degree of influence based on the measure of Pearson's correlation. Using the degree of influence and least squares regression, the method predicts values of effect KPIs. The method is evaluated by applying it on 10 widely used KPIs from 29 real-life service systems. We find that the method validates 8 of the 15 intuitive relationships and estimates the degree of influence for each of the validated relationships. Further, based on the degrees of influence and the regression model learned from the 29 service systems, the method could estimate the values of the effect KPIs with an average root-mean-squared error (RMSE) of 1.2%, in 9 additional service systems.

Optimal Strategy for Proactive Service Delivery Management Using Inter-KPI Influence Relationships

“…A possible approach for assessing the KPI relations is to depend on domain expertise for identifying the major KPIs of interest and their intuitive relationships. Shrinivasan et al [1] outline a method for validating expert opinion and leveraging it to understand relations. However it may not be feasible to identify all the relations using domain experts, especially their strength of influence or how soon the causal effects of one be seen on the other.…”

“…It is hence important to drive data-driven methods that analyzes the critical KPIs and can help move the system to a desired optimal state. However demystifying the impact of operational KPIs is non-trivial due to the following reasons:(1) each KPI relation can have multiple attributes of direction, strength and polarity [1]. Negative polarities imply that improving some KPIs could worsen others.…”

“…Measurement frameworks for IT service systems is a well-established need and in the last couple of years a fair amount of research is focusing on service quality [4]. In the paper [1] authors discuss a validation approach for KPI relationships and a regression model for predicting the values of outcome KPIs. However the work uses an aggregated KPI measure and complexity of different lags is not addressed.…”

“…The problem of selecting the right set of KPIs, such that the desired outcome state is attained within a time and a budget constraint has also not been addressed in the service system domain. Building up on previous work [1], we outline the following:…”

Improvements Towards the Identification and Quantification of Relationships Between Key Performance Indicators

“…Each KPI can have multiple attributes of direction, strength, and polarity. For example, improving some KPIs might worsen others, and the impact of a KPI might be delayed and only become apparent at a later stage [1,2]. To address these challenges, the importance of quantifying inter-relationships between KPIs has been studied.…”