No abstract

No abstract

Background: There is a need for valid self-report measures of core health-related quality of life (HRQoL) domains.Objective: To derive brief, reliable and valid health profile measures from the Patient Reported Outcomes Measurement Information System ® (PROMIS ® ) item banks.Methods: Literature review, investigator consensus process, item response theory (IRT) analysis, and expert review of scaling results from multiple PROMIS data sets. We developed 3 profile measures ranging in length from 29 to 57 questions. These profiles assess important HRQoL domains with highly informative subsets of items from respective item banks and yield reliable information across mild-to-severe levels of HRQoL experiences. Each instrument assesses the domains of pain interference, fatigue, depression, anxiety, sleep disturbance, physical function, and social function using 4-, 6-, and 8-item short forms for each domain, and an average pain intensity domain score, using a 0-10 numeric rating scale.Results: With few exceptions, all domain short forms within the profile measures were highly reliable across at least 3 standard deviation (30 T-score) units and were strongly correlated with the full bank scores. Construct validity with ratings of general health and quality of life was demonstrated. Information to inform statistical power for clinical and general population samples is also provided.Conclusions: Although these profile measures have been used widely, with summary scoring routines published, description of their development, reliability, and initial validity has not been published until this article. Further evaluation of these measures and clinical applications are encouraged.

Reliability is a fundamental problem for measurement in all of science. Although defined in multiple ways, and estimated in even more ways, the basic concepts seem straightforward and need to be understood by practitioners as well as methodologists. Reliability theory is not just for the psychometrician estimating latent variables, it is for everyone who wants to make inferences from measures of individuals or of groups. For the case of a single test administration, we consider multiple measures of reliability, ranging from the worst (β) to average (α, λ3) to best (λ4) split half reliabilities, and consider why model-based estimates (ωh, ωt) should be reported. We also address the utility of test–retest and alternate form reliabilities. The advantages of immediate versus delayed retests to decompose observed score variance into specific, state, and trait scores are discussed. But reliability is not just for test scores, it is also important when evaluating the use of ratings. Estimates that may be applied to continuous data include a set of intraclass correlations while discrete categorical data needs to take advantage of the family of κ statistics. Examples of these various reliability estimates are given using state and trait measures of anxiety given with different delays and under different conditions. An is provided with more detail and elaboration. The is also used to demonstrate applications of open source software to examples of real data, and comparisons are made between the many types of reliability.

We argue that it is useful to distinguish between three key goals of personality science – description, prediction and explanation – and that attaining them often requires different priorities and methodological approaches. We put forward specific recommendations such as publishing findings with minimum a priori aggregation and exploring the limits of predictive models without being constrained by parsimony and intuitiveness but instead maximising out-of-sample predictive accuracy. We argue that naturally-occurring variance in many decontextualized and multi-determined constructs that interest personality scientists may not have individual causes, at least as this term is generally understood and in ways that are human-interpretable, never mind intervenable. If so, useful explanations are narratives that summarize many pieces of descriptive findings rather than models that target individual cause-effect associations. By meticulously studying specific and contextualized behaviours, thoughts, feelings and goals, however, individual causes of variance may ultimately be identifiable, although such causal explanations will likely be far more complex, phenomenon-specific and person-specific than anticipated thus far. Progress in all three areas – description, prediction, and explanation – requires higher-dimensional models than the currently-dominant “Big Few” and supplementing subjective trait-ratings with alternative sources of information such as informant-reports and behavioural measurements. Developing a new generation of psychometric tools thus provides many immediate research opportunities.

The influence of personality on important life outcomes has been widely recognized for thousands of years (Condon, 2014), and the difficulty of its measurement has been vexing for many decades (Galton, 1884; Cattell, 1945; Goldberg, 1981; Ackerman, 2018). The challenge with objective measurement stems from the need for massive amounts of data to account for dynamic interplay between variations in thousands of narrow dispositional traits (aka individual differences in behavior) and the ever-evolving contextual factors inherent to modern living. It is a prototypical “big data” problem. Despite this, dozens of ambitious social scientists have posited a diverse array of personality assessment models. Many of these are heavily imbued with theory, nearly all are focused solely on one domain of personality (e.g., very broad dispositional traits or vocational interests) to the exclusion of others (e.g., cognitive abilities, values, or less generalizable maladaptive behaviors), and most have been derived based on surprisingly small samples drawn from populations that have come to be known as "WEIRD" (Henrich et al., 2010). Simply put, there is widespread need for models that are empirically-grounded in more (and more representative) data.In this manuscript, I demonstrate that it is possible to address the shortcomings of extant theory-driven approaches by combining recent innovations from outside of personality research to empirically derive personality assessment models. This is done by administering a large pool of widely-used public domain items from the International Personality Item Pool (Goldberg et al., 1999) to three large online samples (N > 125,000) using a planned missingness design (Revelle et al., 2016). While the existing "best practices" for developing personality assessment models tends towards several iterative rounds of data collection and analysis guided by theory culminating in publication of only the final product, I have endeavored to make a highly detailed record of all steps followed during the development of the SAPA Personality Inventory in order to encourage feedback regarding critical analytic decisions. This has unfortunately resulted in the production of a book-length manuscript but I hope that this transparency will serve to minimize (even if it does not eliminate) the influence of bias.January 10, 2018: This manuscript remains subject to review and further revision. I welcome additional feedback (by email). I will note the status of these revisions as they occur.

Separating the signal in a test from the irrelevant noise is a challenge for all measurement. Low test reliability limits test validity, attenuates important relationships, and can lead to regression artifacts. Multiple approaches to the assessment and improvement of reliability are discussed. The advantages and disadvantages of several different approaches to reliability are considered. Practical advice on how to assess reliability using open source software is provided.

The CHAI appears to be a valid, reliable, and easily administered tool that can be used to assess health activation among adults, including those with limited health literacy. Future studies should test the tool in actual use and explore further applications.