How Much Does the Cardinal Treatment of Ordinal Variables Matter? An Empirical Investigation

Abstract

Many researchers use an ordinal scale to quantitatively measure and analyze concepts. Theoretically valid empirical estimates are robust in sign to any monotonic increasing transformation of the ordinal scale. This presents challenges for the point-identification of important parameters of interest. I develop a partial identification method for testing the robustness of empirical estimates to a range of plausible monotonic increasing transformations of the ordinal scale. This method allows for the calculation of plausible bounds around effect estimates. I illustrate this method by revisiting analysis by Nunn and Wantchekon (2011, American Economic Review, 101, 3221–3252) on the slave trade and trust in sub-Saharan Africa. Supplemental illustrations examine results from (i) Aghion et al. (2016, American Economic Review, 106, 3869–3897) on creative destruction and subjective well-being and (ii) Bond and Lang (2013, The Review of Economics and Statistics, 95, 1468–1479) on the fragility of the black–white test score gap.

The Analysis of Human Feelings: A Practical Suggestion for a Robustness Test

“…Second, the technique is simple and straightforward to execute, which is not necessarily the case for other methods for testing robustness (see, e.g., Bloem 2021). It only requires two additional regressions.…”

“…This is the issue on which we focus. Our article builds particularly upon the work of researchers such as Ferrer‐I‐Carbonell and Frijters (2004), Oswald (2008), Abul Naga and Yalcin (2008), Lv et al (2015), Ravallion et al (2016), Schröder and Yitzhaki (2017), Bond and Lang (2019), Chen et al (2019), Kaiser and Vendrik (2019), Apouey et al (2020), and Bloem (2021).…”

Weather-Related Home Damage and Subjective Well-Being

“…An alternative approach is taken by Bloem (2021), who suggests using a series of data transformations to check the robustness of results obtained using cardinal models. His research presents two key insights.…”

“…14 While it may be possible that there is no monotonic rank-preserving transformation that will reverse the sign of an estimated coefficient, this is no guarantee that valid transformations will not substantially change both the size and the significance of an estimated coefficient. The second insight is that failing to meet the theoretical criteria does not automatically invalidate the results obtained if only relatively extreme transformations substantially influence the results (Bloem 2021). Following on from these two insights, Bloem (2021) suggests applying a series of transformations to the underlying data to check the robustness of results to different underlying cardinalizations.…”

“…Firstly, Bond and Lang (2019) argue that SWB regressions are highly dependent on the way in which ordinal happiness data are transformed into cardinal data, showing that a monotonic transformation that is rank preserving at the individual level can lead to coefficient estimates of the opposite sign. Following the method suggested by Bloem (2021), we perform a series of data transformations to investigate empirically whether our results are robust to changes in the assumed underlying cardinal scale. Second, the recent literature (Borusyak and Jaravel 2018;de Chaisemartin and D'Haultfoeuille 2020;Callaway and Sant'Anna 2021;Goodman-Bacon 2021;Sun and Abraham 2021) shows that the standard two-way fixed-effect (difference-in-difference) analysis can be misleading when the impact of a treatment changes over time.…”