Development and validation of a rule-based time series complexity scoring technique to support design of adaptive forecasting DSS

Adya, Monica; Lusk, Edward J.

doi:10.1016/j.dss.2015.12.009

Cited by 15 publications

(19 citation statements)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Decision making literature suggests that task complexity influences decision maker's strategies (Payne et al, 1990), information seeking behaviors (Bystrom and Jarvelin, 1995), DSS use (Adya and Lusk, 2016), and performance (Campbell, 1988). In the forecasting literature, a small but consistent set of studies provide similar evidence that the complexity of a time series can challenge the forecasting process and detrimentally influence forecast accuracy (Goodwin and Wright, 1993).…”

Section: Introductionmentioning

confidence: 91%

“…To address this gap, Adya & Lusk (2016), hereon referred to as A&L, developed and validated an FDSS, the Complexity Scoring Technique (CST). The CST is a rule-based DSS designed to distinguish complex time series from simple ones.…”

Section: Introductionmentioning

confidence: 99%

“…

…”

mentioning

confidence: 99%

See 2 more Smart Citations

Replication and Extension of a Forecasting Decision Support System: An Empirical Examination of the Time Series Complexity Scoring Technique

Adya¹,

Lusk²

2018

AIS TRR

Self Cite

View full text Add to dashboard Cite

Abstract:This study presents a conceptual replication of Adya and Lusk's (2016) forecasting decision support system (FDSS) that identifies the complexity or simplicity of a time series. Prior studies in forecasting have argued convincingly that the design of FDSS should incorporate the complexity of the forecasting task. Yet, there existed no formal way of determining time series complexity until this FDSS, referred to as the Complexity Scoring Technique (CST). The CST uses characteristics of the time series to trigger 12 rules that score the complexity of a time series and classify it along the binary dimension of Simple or Complex. The CST was originally validated using statistical forecasts of a small set of 54 time series as well as judgmental forecasts from 14 representative participants to confirm that the FDSS successfully distinguished Simple series from Complex ones. In this study, we (a) replicate the CST on a much larger set of data from both statistical and judgmental forecasting methods, and (b) extend and validate the series classification categories from the binary Simple-Complex used in the original CST to Very Simple, Simple, Complex, and Very Complex thus adding an ordinal link between the two previous binary designations. Findings suggest that both the replication and extension of the CST further validate it, thereby greatly enhancing its use in the practice of forecasting. Implications for research and practice are discussed.

show abstract

Section: Introductionmentioning

confidence: 91%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Replication and Extension of a Forecasting Decision Support System: An Empirical Examination of the Time Series Complexity Scoring Technique

Adya¹,

Lusk²

2018

AIS TRR

Self Cite

View full text Add to dashboard Cite

show abstract

“…According to extensive research over the last four decades a simple, effective, efficient and extensively used way of benchmarking forecasting activity is the holdback validation of the forecasting model. See Makridakis et al (1982), continuing with the work of , and recently with the complexity calibration work of Adya & Lusk (2016).…”

Section: Holdback Validation Of the Ols Modelmentioning

confidence: 99%

Analytic Procedures: A Holdback-vetting Forecasting Model

Lusk¹

2017

AFA

Self Cite

View full text Add to dashboard Cite

Introduction: Forecasting is now a best practices requirement for PCAOB audits. This is clear from AS 5 where Analytic Procedures are now a part of the Planning and Substantive Phases of the certification audits. In this regard, we are encouraged by the -On the Go Stores" AP case offered by the AICPA, and have extended their case illustration of AP treatments. Study Pré cis:In our presentation, we initially consider the OLS Regression model utilized in the AICPA case and offer a vetting protocol to rationalize the use of this forecasting model in the AP phases. Then we move to a disposition analysis stage where the forecast information is posed in relief to the actual client value so as to ascertain if Extended Procedure investigations would be warranted. Results:We offer three Confidence Intervals drawn from the OLS modeling system that are formed from the Fixed Effect, Random Effects, and finally the Excel Platform for the 95% CI parameter set. Impact:The protocol set is programmed in an open-access VBA Decision Support System which is available free as a download with no restriction on its use.

show abstract

“…We have selected the time-series set used in the Makridakis et al (1982), Collopy & Armstrong (1992) and Adya & Lusk (2016) studies. These time-series have been used in a large number of studies over the years and in this sense have been vetted for inclusion in our testing protocol.…”

Section: The Datasetsmentioning

confidence: 99%

Analytical Procedures Phase of PCAOB Audits: A Note of Caution in Selecting the Forecasting Model

Gaber¹,

Lusk²

2017

AFA

Self Cite

View full text Add to dashboard Cite

The best-practices execution of PCAOB audits requires the use of Analytical Procedures at the Planning and the Substantive Phases. This often finds the auditor using the standard OLS two-parameter linear regression forecasting model [OLSR] to project account-values from the Planning Phase to balances expected at Year-End so as to effect a variance analysis at the Substantive Phase. This is the point of departure of our study. We examine the practical effect of using the OLSR model in a time-series context of the audit. Specifically, this research report provides information on the use of the OLSR model as the model of choice in the audit context compared to the ARIMA(0,2,2)/Holt model which is usually the standard choice for an exponential smoothing model in the presence of autocorrelation of data in the time-stream; autocorrelation is the usual case for longitudinal series taken in the audit. Results: We find that there are reasons to condition the selection of the forecasting model in the Analytical Procedures context based upon autocorrelation in the data-stream. When the time-stream of data exhibits autocorrelation the OLSR model fails in a statistically significant manner to capture the next or one-period ahead client value at the same rate as does the ARIMA/Holt model. This then has implications for the False Negative Investigation Error.Keywords: big-data, Holt, forecasting confidence intervals Forecasting as the Principal Platform in Forming an Analytical Procedures Investigation Projection ContextThe Public Company Accounting Oversight Board [PCAOB] Requires the Use of Analytical Procedures [AP] in the Planning and the Substantive or Completion phases of certification audits. As noted by Arens, Elder, Beasley & Hogan (2015, p.193 This linkage between the Planning and the Substantive phase all but proscribes that there is a projection at the Planning stage, typically formed analytically using previously reported and certified past data and forecasted as an expectation for the client's Year-End value. The modelling form used to generate this projection is very often the standard linear regression model. In fact, the AICPA (2012) initiative: The Clarity Project dealing with Analytical Procedures presents a carefully detailed study called the On the Go Stores case analysis where the forecasting technique used is the standard regression model.After the forecasting projection is made the auditor may examine the difference between the AP forecasting expectation, Applied Finance and Accounting Vol. 4, No. 1; 2018 74 E, and the actual client's account value, A. This is usually referred to as the disposition phase of the AP protocol where if the magnitude of the directional difference between E & A is relatively large, then the auditor is required to consider the meaning of this difference, note in the working papers the likely reason(s), and finally to justify the decision whether or not to further investigate the account.Typically in our experience and that of our colleagues in the audit worl...

show abstract

Development and validation of a rule-based time series complexity scoring technique to support design of adaptive forecasting DSS

Cited by 15 publications

References 65 publications

Replication and Extension of a Forecasting Decision Support System: An Empirical Examination of the Time Series Complexity Scoring Technique

Replication and Extension of a Forecasting Decision Support System: An Empirical Examination of the Time Series Complexity Scoring Technique

Analytic Procedures: A Holdback-vetting Forecasting Model

Analytical Procedures Phase of PCAOB Audits: A Note of Caution in Selecting the Forecasting Model

Contact Info

Product

Resources

About