Evaluating the Performance of Iterative Proportional Fitting for Spatial Microsimulation: New Tests for an Established Technique

Abstract

Iterative Proportional Fitting (IPF), also known as biproportional fitting, 'raking' or the RAS algorithm, is an established procedure used in a variety of applications across the social sciences. Primary amongst these for urban modelling has been its use in static spatial microsimulation to generate small area microdata -individual level data allocated to administrative zones. The technique is mature, widely used and relatively straight-forward. Although IPF is well described mathematically, accessible examples of the algorithm written in modern programming languages are rare. There is a tendency for researchers to 'start from scratch', resulting in a variety of ad hoc implementations and little evidence about the relative merits of differing approaches. These knowledge gaps mean that answers to certain methodological questions must be guessed: How can 'empty cells' be identified and how do they influence model fit? Can IPF be made more computationally efficient? This paper tackles these questions and more using a systematic methodology with publicly available code and data. The results demonstrate the sensitivity of the results to initial conditions, notably the presence of 'empty cells', and the dramatic impact of software decisions on computational efficiency. The paper concludes by proposing an agenda for robust and transparent future tests in the field.

Validation of Spatial Microsimulation Models: a Proposal to Adopt the Bland-Altman Method

“…This paper is not intended as an exhaustive exposition about the relative merits and disadvantages of all the methods described in Table 1. This has been clearly and comprehensively covered by previous publications (Edwards & Tanton, 2012;Lovelace et al, 2015;Scarborough et al, 2009).…”

“…One of the chief problems encountered in validation, particularly in model calibration, is that there is often a large quantity of values to compare, each with a varying degree of agreement between actual and simulated values, so that gaining an overall assessment of model fit is challenging. On the other hand, summary measures of model fit have the disadvantage of losing information (Lovelace et al, 2015). One review concluded that a combination of approaches was necessary to overcome these shortcomings (Kopec et al, 2010).…”

“…Although the importance of validation is widely recognised (Edwards et al, 2011) o Standard Error about Identity (SEI) ( Ballas et al, 2007;Tanton & Vidyattama, 2010) o Total Absolute Error (TAE) (Voas & Williamson, 2001) o Standardized Absolute Error (SAE) (Edwards & Tanton, 2012) o Root Mean Squared Error (RMSE) (Legates & McCabe Jr, 1999) o Z-score or modified z-score (zm) (Voas & Williamson, 2001) or overall relative sum of zsquare scores (Tanton, Williamson, & Harding, 2014) o E5 (Lovelace et al, 2015) o Statistical techniques to derive confidence intervals (e.g. bootstrapping (Tanton, 2015)) o Statistical tests for difference (e.g.…”

“…The relative strengths and merits of these validation methods have been extensively explored by a number of reviews (Edwards & Tanton, 2012;Lovelace et al, 2015;Rahman et al, 2013;Scarborough et al, 2009).…”

Estimating Population Attribute Values in a Table: “Get Me Started in” Iterative Proportional Fitting

“…It is fast and requires little computational power when compared with other methods (Lovelace et al 2015); the methodology is transparent (once it is properly explained) and is reproducible; that is, with the same inputs, the outcome is the same no matter how many times it is implemented. There is also growing support for implementing IPF in a variety of statistical packages, discussed next.…”