The selection of upper and lower groups for the validation of test items.

Abstract

It is not intended here to discuss the general problem of item validation, but only that aspect of it thai arises when an upper and lower group are selected to serve as standard groups in the differentiation of test items. It is argued that the more indubitably it is known that the upper group is superior to the lower group, the more definitely can it be concluded that an item is valid by finding that the upper group is more successful in passing it than the lower group. If, in two situations, one in which the upper and lower groups are differentiated with high certainty and the other with little certainty, the proportion of passes (i.e., right answers) in the upper groups are equal and equally superior to the proportion of passes in the lower groups, we should believe that the item represented in the first situation is more valid than that of the second situation.Having available an initial group which is normally distributed with reference to a desired criterion, we set the problem of selecting upper and lower portions of this group which will be most efficient in the study of items, and their selection or rejection. The items in question are capable of two grades only, right or wrong. We further limit the issue by not here considering the interrelationship of items, a matter of first importance when the final test to be constructed is to contain more than one item. It is granted that the problem as set is too constricted to be "real," but it is, nevertheless, believed that its solution is commonly pertinent to the handling of real item selection problems.The writer has stated 1 that twenty-seven per cent should be selected at each extreme to yield upper and lower groups which are most indubitably different with respect to the trait in question. This article does not alter that conclusion but does provide a more available and somewhat improved derivation.Let us be given graduated scores on a test or trait from a sample of size N. For simplicity we shall consider N to be even, so that we may 1 Reported by Milton B. Jensen: The Objective Differentiation of Three Groups

Evaluation of Automatically Generated Pronoun Reference Questions

“…The groups of high and low proficiency are defined as the top 27% of the test takers and bottom 27% of the test takers respectively. The threshold value of 27% is utilised to maximise two characteristics; those two groups must be as different as possible to discriminate clearly, and the number of test takers in each group must be as large as possible to achieve reliability (Popham, 1981;Kelley, 1939). We computed the item discrimination index for each question item and the average of them.…”

El Pensamiento Probabilístico de los Profesores de Biología en Formación

“…Hemos calculado este índice en los doce ítems relativos al reconocimiento de la aleatoriedad. Para lo cual, hemos seleccionado dos grupos de estudiantes en función de sus puntuaciones totales de acuerdo el criterio de Kelley (1939), quien mostró que el índice es más sensible cuando se forman dos grupos: uno con el 27% superior y el otro con el 27% inferior de los estudiantes, que han sido calificados en función del porcentaje de ítems reconocidos como aleatorios. Una vez identificados los grupos, se calculan las proporciones de aciertos del ítem considerado en los dos grupos, p s y p i y se define el índice de discriminación como: D = p s -p i Donde, p s es la proporción de estudiantes del grupo superior que responden correctamente al ítem; y donde p i es la proporción de estudiantes del grupo inferior que responden correctamente al ítem.…”

Razão de mortalidade proporcional de Swaroop e Uemura: necessidade de revisão periódica de sua definição

“…Para determinar o tamanho dos grupos a serem comparados, adotou-se o critério proposto por Kelley 9 (1939) para formação de grupos em análi-ses por itens de testes educacionais; segundo esse autor, a maior discriminação entre os grupos é alcançada quando cada um deles, em uma e outra das extremidades da ordenação do resultado global do teste, inclui 27% do total de observações. No presente caso, essa percentagem corresponde, por aproximação ao inteiro, a 9 países.…”

“…In order to determine the size of the groups to be compared, the criterion proposed by Kelley 9 (1939) for the selection of groups for analysis, by means of items in educational tests, was adopted; according to that author the greatest discrimination between groups is attained when each of them, at both extremes of the ordering of the overall result of the test, includes 27% of the total number of observations. In the present case this percentage corresponds, by approximation to the whole number, to 9 countries.…”