Orthorexia nervosa: Research based on invalid measures is invalid

Abstract

O rthorexia nervosa (ON) refers to a strong preoccupation with a healthy diet with negative emotional, cognitive, or social consequences when trying to approach this goal and when the eating behaviour deviates from these self-imposed rules. According to a recent meta-analysis published in this journal, the prevalence of ON could be as high as 27.5% [1]. If that was true, ON -which is not (yet) a recognised eating disorder in diagnostic classification systems -would be the most prevalent of all eating disorders and, in fact, of all mental disorders [2]. There are two explanations for this. Either humans have been unaware of this eating disorder pandemic or the results from that meta-analysis suffer from a systematic bias in measurement. We consider the second one more likely.Most importantly, the data that the meta-analysis is based on all came from studies that used the ORTO-15 questionnaire [3]. However, recent evidence shows this scale produces invalid scores for assessing ON [4].Even the meta-analysis's authors acknowledge the ORTO-15's limitations in the discussion section. Given that 'validity refers to the degree to which evidence and theory support the interpretations of test scores for proposed uses of tests (p. 11,[5]), no validity implies that the scores cannot be interpreted as intended. More clearly stated, the ORTO-15 does not measure ON and should not be interpreted as doing so. The logical conclusion is that any study using it (including meta-analyses) adds very limited knowledge to the scientific record or, even worse, it can be considered noise that some people will interpret as evidence. It has to be remembered that 'validity is, therefore, the most fundamental consideration in developing tests and evaluating tests (p. 11,[5]). PROBLEMS IN THE ASSESSMENT OF ORTHOREXIA NERVOSAUnfortunately, research on ON has been hampered for years due to the psychometric limitations of several of its measures. For example, one of the most prominent limitations of the ORTO-15 is its low internal reliability and absence of a first-or second-order one-factor structure [4]. Without this, item responses must not be summed to a total score, that is, computing total scores leads to uninterpretable values.Other important limitations include the content of the items of the ORTO-15 but also of other commonly used instruments. Specifically, some items are either unspecific to ON (e.g. include no clear reference to healthy eating) or pathologise a healthy or neutral approach to eating. For unspecific items, consider 'In the last three