The item correlation coefficients which form part of Lertap's Stats1f and Stats1b reports are based on correlating item scores with what's referred to as an "internal criterion": the number produced by scoring the remaining items in the subtest or scale to which the item belongs.

 

It is possible to replace the internal criterion with another score. This other score is referred to as an "external criterion". An external criterion may be used as part of the process of validating test items.

 

In Lertap, the external criterion score must correspond to a column in the Scores worksheet. The analysis begins by having the user pick the Scores column that has the score to be used as the external criterion. Once this is done, Lertap asks the user to pick out the subtest which has the items to be correlated with this score. This is done by displaying Sub worksheets.

 

Users of the external criterion analysis feature will often have a criterion measure which needs to be imported to the Scores worksheet. In this case, the criterion measure should be recorded in a column in the Data worksheet. Once it's there, the Move menu on the Lertap toolbar will allow the measure to be copied over to the Scores worksheet.

 

The effects of part-whole inflation may be examined by using an external criterion analysis. Lertap's item correlation coefficients are always corrected for part-whole inflation (sometimes referred to as part-whole contamination); to see what they'd be without such correction, define a subtest's score, as found in the Scores worksheet, to be the external criterion. (In some texts and other item analysis programs, part-whole inflation is at times referred to as "spuriousness".)

 

An example of the output corresponding to an external criterion analysis may be seen via a click here.

 

Related tidbits:

 

See the "Using an external criterion" section of the manual's Chapter 8.

 

An easy-to-read technical paper with more details about how Lertap 5 calculates item correlations is available here.

 

Also see "Using Lertap in a Test Validity Study", click here to branch out to it.