These results are from a high-stakes test of topics having to do with characteristics of the English language. It was given to over 5,000 junior high school students in an Asian country.

 

Students had to have a score of at least 50% to pass the test. However, in our discussion of this sample, the pass rate will not be our focus. (Please see the M.Nursing sample for mention of cut scores, pass rates, and mastery testing.)

 

Here, we'll concentrate on using multiple subtests, and an external criterion analysis, in order to see how Lertap may be used to check out the quality of trial, or "pretest", or "pilot" items. These are questions embedded in the exam which are not scored; these questions are themselves under test, as it were, to see if they're good enough.

 

 

 

 

This 50-item test had 40 core items, and 10 trial, or "pretest", items.

 

The trial items were I6, I15, I16, I21, I23, I29, I31, I35, I39, and I49. These items were not scored. They were included to see how well they would perform; if they turned out well, they would be considered for inclusion in a future version of the test.

 

The first subtest, "Core", has an *exc line which serves to exclude the trial items. The second subtest, "Trial", has only the trial items, while "All", the last subtest, uses all 50 items.

 

The "Guide" worksheet was used to make it easier to write the CCs lines; what is particularly useful is the "map" which links items to columns.

 

The "C or T" column in this "Guide" indicates if an item was a "Core" item, or a "Trial" item.

 

The first trial item is I6, found in c10 of the Data worksheet. (Note that c10 is the first entry in the *exc line.) The map also makes it easier to write the *key line for the trial items. But fear not: you don't have to have a worksheet like our "Guide". It has nothing to do with what Lertap does -- it's just a helper for making CCs lines easier to put together.

 

Okay then, let's get some results, eh? We'll go through the 3 Steps, and meet you in the next topic.