Statistics with external criterion
Contents
Statsf reports become "ECStatsf" reports when an external criterion analysis has been selected. The discussion found in this topic is based on results from the LenguaBIg dataset. The "Core" score was used as the external criterion, and the second subtest, "Trial items only", was selected for the analysis.
option |
The response option selected by the student. Response options may be letters (upper or lower case) or digits (from 0 to 9). If the option corresponds to the keyed-correct answer, then the option is underlined, as are all the statistics for the option. |
wt. |
The number of scoring points given for selecting the option. |
n |
The number of students who selected the option. |
p |
The proportion of students who selected the option. Multiply this by 100 to get the corresponding percent. In the example above, 56% of the students selected option D on I21. |
pb/ec |
The point-biserial correlation with the external criterion score. To compute this, all those who selected the option are given a "score" of 1 while those who did not are given a score of 0. These scores are then correlated with the external criterion score. |
b/ec |
The biserial correlation with the external criterion score. Note that it is possible for b/ec to be greater than 1.00. |
avg/ec |
The average external criterion score for those who selected the option. On I21 (above), the 3,072 students who selected option D, the keyed-correct option, had an average external criterion score of 27.49. The "ba" flag is showing because this avg/ec was below the mean of the external criterion scores, a fact which is also flagged by the negative "z" of -0.12. The "aa" flag on option C, a distractor, indicates that the 2,271 students who selected C, with an average external criterion score of 29.65, were above average students, at least as measured by their external criterion scores. These are generally unwanted outcomes; the flags are there to draw attention to a potential problem with I21. |
z |
avg/ec expressed as a "z score". |
r/ec |
The Pearson product-moment correlation between the item and the external criterion score. When an item has only one option with a non-zero "wt." value, r/ec will be the same as pb/ec. |
The "Statistics" table for an ECStatsf report is similar to the "Summary statistics" found in an ordinary Statsf report, as seen in the next topic.
The "correlation bands" plot the r/ec values found for the items. In this example, only one item, I6, had a reasonable correlation with the external criterion (generally we'd want these correlations to be at least 0.30). As seen in the upper part of this topic, I21's r/ec was -0.13; items with negative r/ec values are found in the ".00:" correlation band.