Difficulty, Discrimination, and Information Indices in the Linear Factor Analysis Model for Continuous Item Responses

Ferrando, Pere J.

doi:10.1177/0146621608314608

Cited by 30 publications

(65 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…This article uses Fleishman's (1978) power transformation (PT) method probability density function to study the impact of nonnormal latent distributions on the reliability of total scores. Second, additional research has extended the concept of item and test information to CTT under the assumptions that observed measurements are continuous, rather than polytomous, and u is linearly related to x (Ferrando, 2002(Ferrando, , 2009McDonald, 1982;Mellenbergh, 1996). Ferrando notes that a linear model provides a good approximation when item discrimination indices are relatively small in value and coarsely measured items have five or more response categories.…”

mentioning

confidence: 99%

The Reliability and Precision of Total Scores and IRT Estimates as a Function of Polytomous IRT Parameters and Latent Trait Distribution

Culpepper

2013

Applied Psychological Measurement

View full text Add to dashboard Cite

A classic topic in the fields of psychometrics and measurement has been the impact of the number of scale categories on test score reliability. This study builds on previous research by further articulating the relationship between item response theory (IRT) and classical test theory (CTT). Equations are presented for comparing the reliability and precision of scores within the CTT and IRT frameworks. This study presented new results pertaining to the relative precision (i.e., the test score conditional standard error of measurement for a given trait value) of CTT and IRT, and the new results shed light on the conditions where total scores and IRT estimates are more or less precisely measured. The relative reliability of CTT and IRT scores is examined as a function of item characteristics (e.g., locations, category thresholds, and discriminations) and subject characteristics (e.g., the skewness and kurtosis of the latent distribution). CTT total scores were more reliable when the latent distribution was mismatched with category thresholds, but the discrepancy between CTT and IRT declined as the number of scale categories increased. This article also considered the appropriateness of linear approximations of polytomous items and presented circumstances where linear approximations are viable. A linear approximation may be appropriate for items with two response options depending on the item discrimination and the match between the item location and latent distribution. However, linear approximations are biased whenever items are located in the tails of the latent distribution and the bias is larger for more discriminating items.The impact of the number of scale categories on reliability is a classic topic in psychology (Symonds, 1924), and an extensive body of research has examined the effect of the number of

show abstract

mentioning

confidence: 99%

The Reliability and Precision of Total Scores and IRT Estimates as a Function of Polytomous IRT Parameters and Latent Trait Distribution

Culpepper

2013

Applied Psychological Measurement

View full text Add to dashboard Cite

show abstract

“…For instance, we have used an empirical approach to the derivation of the environment response curve, but some other IRT models of difficulty and discriminating power ( [22,23]) could be adapted to our case.…”

Section: Discussionmentioning

confidence: 99%

On environment difficulty and discriminating power

Hernández-Orallo

2014

Auton Agent Multi-Agent Syst

View full text Add to dashboard Cite

This paper presents a way to estimate the difficulty and discriminating power of any task instance. We focus on a very general setting for tasks: interactive (possibly multi-agent) environments where an agent acts upon observations and rewards. Instead of analysing the complexity of the environment, the state space or the actions that are performed by the agent, we analyse the performance of a population of agent policies against the task, leading to a distribution that is examined in terms of policy complexity. This distribution is then sliced by the algorithmic complexity of the policy and analysed through several diagrams and indicators. The notion of environment response curve is also introduced, by inverting the performance results into an ability scale. We apply all these concepts, diagrams and indicators to two illustrative problems: a class of agent-populated elementary cellular automata, showing how the difficulty and discriminating power may vary for several environments, and a multi-agent system, where agents can become predators or preys, and may need to coordinate. Finally, we discuss how these tools can be applied to characterise (interactive) tasks and (multi-agent) environments. These characterisations can then be used to get more insight about agent performance and to facilitate the development of adaptive tests for the evaluation of agent abilities.

show abstract

“…atenuadas) con respecto a los pesos verdaderos y, además, esté efecto no será constante: la atenuación será mayor cuanto más extremos y discriminativos los ítems (Muthén y Kaplan, 1985). Dado que el peso factorial es el principal indicador de la calidad del ítem como medida (Ferrando, 2009), este problema tiene consecuencias importantes ya que llevará, posiblemente, a interpretaciones incorrectas.…”

Section: ¿Correlaciones Producto-momento O Correlaciones Policóricas?unclassified

“…Pues, en términos generales, porque no es inusual que un modelo aproximado pero simple y robusto funcione mejor que un modelo teóricamen-te más apropiado pero más complejo e inestable, y, en el caso del AF, esto es bastante frecuente (Ferrando, 2009;Ferrando y Lorenzo-Seva, 2013;Hofstee, ten Berge y Hendricks, 1998). El AF no lineal es, desde luego, bastante más complejo que el lineal y presenta potenciales problemas que conviene conocer y que discutiremos a continuación.…”

Section: ¿Correlaciones Producto-momento O Correlaciones Policóricas?unclassified

“…Así, un cuestionario formado por ítems con formato Likert de 7 puntos que tienen dificultades medias y no son excesivamente discriminativos, y que se administra a una muestra de 250 sujetos justifica el uso del AF lineal basado en las correlaciones de Pearson. Este escenario es relativamente frecuente en medidas de personalidad o actitud que evalúan rasgos no patológicos (Ferrando, 2009). En el otro extremo, un test que contiene ítems muy fáciles junto a ítems muy difíciles, todos ellos muy discriminativos, con formato de respuesta en 3 puntos y administrado a una muestra de 600 sujetos sugiere claramente el uso de la opción no lineal.…”

Section: ¿Correlaciones Producto-momento O Correlaciones Policóricas?unclassified

See 1 more Smart Citation

El análisis factorial exploratorio de los ítems: algunas consideraciones adicionales

2014

Self Cite

View full text Add to dashboard Cite

Resumen: El presente artículo puede considerarse como una ampliación al trabajo de Lloret et al. (2014) en la que se discuten, de forma ampliada, dos tópicos de especial relevancia en análisis factorial de ítems: (a) la decisión acerca de la matriz de correlación más apropiada en cada caso, y (b) la determinación de soluciones finales semi-confirmatorias, que sean realistas, interpretables y que utilicen la información disponible por el investigador. La presentación de los dos tópicos no es neutral, sino que refleja las posiciones de los autores, por lo que debe ser también evaluada críticamente por parte del lector. En ambos casos se ofrecen recomendaciones prácticas. Sin embargo, el trabajo va especialmente dirigido a los investigadores aplicados con cierta orientación metodológica que quieren ir un poco más allá de las recomendaciones actualizadas propuestas anteriormente. Palabras clave: correlaciones producto-momento; correlaciones policóri-cas; modelo de respuesta graduada; estructura simple; rotaciones semiespecificadas; matriz diana; rotaciones ortogonales y oblicuas.Title: Exploratory Item Factor Analysis: Some additional considerations. Abstract: This article can be considered to be an extension of a previous work by Lloret et al. (2014) in which we discuss in more depth two particularly relevant topics in item factor analysis: (a) how to chose the most appropriate correlation matrix, and (b) how to arrive at a realistic and interpretable semi-confirmatory solution by making use of the available information. The discussion of these topics strongly reflects the views of the authors. So, we encourage the interested reader to take it critically. For both topics we offer practical recommendations. However, the article is mainly intended for applied researchers with a certain methodological penchant and who want to go beyond standard recommendations. Key words: Product-moment correlations; Polychoric correlations; Graded Response Model; Simple Structure; Semi-Specified rotations; Target Matrix; Orthogonal and Oblique Rotations.Pocas técnicas psicométricas han generado tanta controversia y tenido tantos altibajos como el análisis factorial exploratorio (AFE). Con respecto al primer punto, ha sido y sigue siendo con mucho, la técnica más usada en análisis de ítems, pero también la más criticada. Con respecto al segundo, el uso cada vez más generalizado de los modelos de ecuaciones estructurales lo arrinconó hace unos años al limbo de las técnicas casi obsoletas. Un AFE podía tener cierta justificación en las etapas iniciales de un estudio. Sin embargo, a partir de aquí, la técnica 'correcta' era el más riguroso análisis factorial confirmatorio (AFC). La realidad, sin embargo, es tozuda: la estructura de la mayor parte de los tests es inherentemente compleja y no casa bien con las restrictivas hipó-tesis del AFC. Parece pues que el AFE sigue siendo necesario para analizar las repuestas a los ítems.Los psicómetras, por su parte, no han dejado de investigar en AFE y de hacerlo evolucionar. Sin embargo, debido a ...

show abstract

Difficulty, Discrimination, and Information Indices in the Linear Factor Analysis Model for Continuous Item Responses

Cited by 30 publications

References 32 publications

The Reliability and Precision of Total Scores and IRT Estimates as a Function of Polytomous IRT Parameters and Latent Trait Distribution

The Reliability and Precision of Total Scores and IRT Estimates as a Function of Polytomous IRT Parameters and Latent Trait Distribution

On environment difficulty and discriminating power

El análisis factorial exploratorio de los ítems: algunas consideraciones adicionales

Contact Info

Product

Resources

About