SCheck Results for Uni Class B

The results below were produced by Wesolowsky's SCheck program.

 

Two of the 11 pairs below, {36, 70} and {95, 128}, were not flagged by Lertap's RSA.  The two programs agreed on the other 9 pairs.  (But be sure to read the text below the table -- the two programs agree exactly if Lertap's options are changed.)

 

    UniClassB_SCheckOut

 

When Lertap's RSA analysis was initially run, producing the results seen on the previous page, four items were omitted due to "overly-popular distractors", that is, distractors (incorrect answers) that were selected more frequently than the items' correct answers.  Excluding such items is recommended when Lertap RSA runs.  However, if the correct answer is selected almost as often as the "popular distractor", and if the item's discrimination is at least 0.20, then leaving the item in the analysis may well be appropriate.  (Note that SCheck will include all items by default.)

 

In this case, if all four initially excluded items, I2, I12, I14, and I20, are included, then Lertap RSA analysis will include 10 of the pairs identified by SCheck, missing out only the {95, 128} pair.

 

In turn, if the EEIC setting in Lertap's System worksheet is changed to 6 from its usual setting of 8, then the {95, 128} pair will also be added to Lertap's list, resulting in total agreement with SCheck. (See the 'Tidbit' below.)

 

Harpp & Hogan (1993) suggested an EEIC=6 setting but modified it to EEIC=8 in Harpp, Hogan, & Jennings (1996).  (Reference list is here).

 

The EEIC=8 setting can be viewed as conservative in that it makes it more difficult for a pair's results to be flagged as being possibly indicative of collusion.

 

The table below is Lertap's RSA cases report with EEIC=6, and with all 34 test items included in the analysis.

 

UniClassBrevisedRSA

 


 

Tidbit: the Windows version of Lertap 5 allows for some RSA settings to be changed on-the-fly, making it unnecessary to open the System worksheet.