Before getting into the syntax for CCs cards used to analyse cognitive items, let's come to terms with some terms.

 

Each cognitive item may use up to twenty-six (26) response codes.  Response codes are also known as alternatives, or as options.  A true/false item may use {T and F} as response codes, or {t, f}, or {1, 2}.  A cognitive item with four possible responses may use codes of {A, B, C, D}; or {a, b, c, d}; or {1, 2, 3 , 4}.

 

Associated with each response code is a weight, the number of points a person gets for choosing the corresponding option.  For example, if the right answer to an item is A, then people who select A will get a certain number of points; people who select one of the item's other responses will (usually) get no points.

 

Okay?  Now then ....

 

Let's say we've given a 5-item cognitive test, with answers appearing in columns 2, 3, 4, 5, and 6 of the Data worksheet.  We'd like Lertap to spin its magic, to analyse our data.  In order to do this, we'll need to enter some lines in the CCs worksheet.  Put on a fresh pot of coffee, pour yourself a cup, and have a look at the examples below.

 

Example C1:

This set of two CCs cards might be all that's required to get Lertap to analyse the data:

*col (c2-c6)

*key ACCDB

Anyone who selects A on item 1, C on items 2 and 3, D on item 4, and B on item 5 will get a score of 5 -- one point for each answer.  Why?  There are five items; the right answers, the "keyed-correct" answers, are shown on the *key card above.  Unless you say otherwise, Lertap awards one point for each right answer.

 

Example C2:

We'll add a *sub card in order to have Lertap label some of its reports:

*col (c2-c6)

*sub Name=(Followup TV9 news quiz), Title=(NewsQuiz)

*key ACCDB

The *sub card is usually optional for cognitive subtests.  Here we're using one just to give a Name and a Title to the subtest.  The Name will appear as a heading at the top of Lertap's item analysis reports, such as Stats1f and Stats1b.  The Title will appear at the top of one of the Scores columns, making it a bit easier to interpret the Scores report.  The Name can have any length, but Title should be no longer than 8 characters.  If Name and/or Title are not given on a *sub card, Lertap will create default labels: Name=(Test 1), and Title=(Test1).

 

Example C3:

Next we'll use a *sub card in order to turn on certain scoring options:

*col (c2-c6)

*sub Title=(NewsQuiz), PER, SCALE

*key ACCDB

Now the *sub card has three control words, Title, PER, and SCALE.  PER gets Lertap to create a percentage score for each test taker, being the student's score expressed as a percentage of the maximum possible score.  For example, if the maximum score is 5, and a student got three items correct, PER=60%.  The SCALE control word adds the student's z-score to the Scores report; on a test with a mean of 3, standard deviation of 1, a student test score of 4 would correspond to a z-score of +1.00.

 

Example C4:

To switch Lertap into its mastery scoring and report mode, include the word MASTERY on the *sub card, as shown here:

*col (c2-c6)

*sub Title=(NewsQuiz), Mastery

*key ACCDB

Using the MASTERY control word on *sub causes two things to happen.  Each student will have her/his percentage score automatically included in the Scores report, just as happens when the PER control word is used.  More importantly, the MASTERY control word gets Lertap to substantially alter one of its main statistical reports.  The Stats1ul report will include a summary group statistics table, a variance components analysis, and two classification accuracy indices (please refer to Chapter 7 of the manual for details, and also take in a 2007 journal article dealing with the use of cut scores).

 

Lertap assumes the mastery cutoff percentage to be 70%.  This can be reset quickly, as shown below:

*col (c2-c6)

*sub Mastery=80%, Title=(NewsQuiz)

*key ACCDB

The cutoff percentage has now been set to 80%.  More generally, it is possible to have the default level of 70% set to any value by making a change in Lertap's System worksheet.  It is also possible to set the mastery level at a raw test score.  For example, Mastery=30 will set the cutoff at a test score of 30:

*col (c2-c6)

*sub Mastery=30, Title=(NewsQuiz)

*key ACCDB

(Click here to read more about the System worksheet, and click here to read a bit more about the use of the Mastery control word.)

 

Example C5:

This example reflects a common situation:

*col (c2-c6)

*sub Res=(1,2,3,4), Title=(NewsQuiz)

*key 13342

The RES control word is telling Lertap that the item response codes are digits, not letters.  Unless you tell it otherwise, Lertap assumes that cognitive items have four options, with response codes of {A, B, C, D}.  If this is not the case, you must use an Res=() declaration on a *sub card, as exemplified above.  Note that the *key card has been changed -- if the response codes were digits, then the *key card will give the digit corresponding to the right answers.  (Also note: RES= is the same as Res=, which is the same as res=, which can even be the same as Responses=; Lertap really only looks at the first letter of the control words, and it doesn't care if letters are upper or lower case.)

 

Here are some other examples of valid Res=() declarations:

Res=(T,F)

(The subtest's *key card must contain Ts and Fs.)

Res=(A,B,C,D,E,F,G,H,I,J)

Res=(a,b,c,d)

(The subtest's *key card must contain lower-case letters.)

Res=(1,2,3,4,5,6)

(The subtest's *key card must have digits.)

Res=(A,B,C,D)

(Not required! This is the default setting for cognitive items.)

 

Critical note: the response codes seen in the Res= declaration tell Lertap what to look for when it reads the information in the Data worksheet's rows.  If the response codes are upper-case letters, such as {A,B,C,D}, then Lertap will expect to find upper-case letters in the relevant columns of the Data worksheet.  Nasty things can happen when, for example, the item responses seen in Data columns are lower-case letters, such as {a,b,c,d}, and the *sub card has Res=(A,B,C,D).  This is a mis-match.  Res=(A,B,C,D) tells Lertap to look for upper-case letters, but none will be found.  Things will come a-crashing.  (There's a bit more on this towards the end of the CCs sheet topic.)

 

Example C6:

Here's one more example of the *sub card in action:

*col (c2-c6)

*sub Title=(NewsQuiz), CFC, Wt=.5

*key ACCDB

CFC means "correction for chance", another scoring option entertained by Lertap.  This control word isn't used all that often; it usually results in penalising students if they appear to be guessing (see Chapter 10 of the manual for more discussion).  The Wt= declaration applies when the CCs worksheet defines more than one subtest, that is, when there are two or more *col cards.  In this case, Lertap will usually generate a total test score by summing the subtest scores; the Wt= assignment controls how this is done.  If Wt=0 then the subtest will not be included in the total test score.

 

Critical note: Wt=1 is the default action -- if there's more than one subtest, and no Wt= assignment is found on *sub cards, then Wt=1 is assumed.  And note: when using Wt=, it's best to put it at the end of the *sub line.

 

Now we will exemplify the use of the other control cards for cognitive tests.

 

Example C7:

We'll add an *alt card:

*col (c2-c6)

*sub Name=(Followup TV9 news quiz), Title=(NewsQuiz)

*key ACCDB

*alt CDDDC

The *alt card is optional.  Here it's telling Lertap that the last response code used by the first and last items is C, whereas the last response code used by all other items is D.  Since there is no explicit Res= declaration, Lertap assumes Res=(A,B,C,D).  (Note that this format of the *alt card differs from that shown in the manual.  The format changed way back in the year 2005.  In the old format this *alt card would have been *alt 34443.)

 

To read more about the practical effects of using *alt, please see the very end of this topic.  A very practical real-life example may be enjoyed by having a look at this little paper.

 

Example C8:

We'll use a *wts card:

*col (c2-c6)

*sub Name=(Followup TV9 news quiz), Title=(NewsQuiz)

*key ACCDB

*wts 31121

The *wts card is optional.  It indicates the number of points to be given for the correct answer, and it's only required when some of the items are worth more than one point.  In this example, the correct answer to the first item, A, is worth 3 points, while the correct answer for the fourth item, D, is worth 2 points.  All other items are worth one point.

 

If an item is to be worth more than 9 points, a *mws card has to be used.  *mws cards are mentioned below.

 

Example C9:

Both *alt and *wts cards in use:

*col (c2-c6)

*sub Name=(Followup TV9 news quiz), Title=(NewsQuiz)

*key ACCDB

*alt CDDDC

*wts 31121

You understand this one, don't you?  The right answer to the first item is A.  It (the first item) uses three response codes, (A,B,C).  A correct answer on the first item is worth 3 points.

 

Q: if I answer D on the fourth item, how many points do I get?  Two.

 

What's the maximum score I can get over these five items?  Eight.

 

If I answer C on the last item, how many points do I get?  None; the right answer is B.

 

If I don't answer the third item, what happens?  I get sent home early with instructions to have extra peanuts with my beer.  (In truth: nothing.  A non-answer to a cognitive item usually gets "scored" as a zero.)

 

Example C10:

Using Lertap's Big Gun, the *mws card:

*col (c2-c6)

*sub Name=(Followup TV9 news quiz), Title=(NewsQuiz)

*key ACCDB

*mws c2, 1, 0, 0, *

*mws c3, 0, 0, 1, 0

*mws c4, 0, 0, 1, 0

*mws c5, 0, 0, 0, 1

*mws c6, 0, 1, 0, *

This example is really the same as Example 9.  We want to ease you into the idea of *mws cards by starting with an "easy" example.

 

Keep in mind that the default Res=(A,B,C,D) applies to this example, there being nothing to the contrary on the *sub card.

 

The *mws c2 card refers to the item whose responses are found in column 2 of the Data worksheet.  This is, of course, the first item.  Of the four potentially-possible responses to this item, (A,B,C,D), the *mws c2 card says that the first response is to get one point; the second and third responses are to get zero points, and the fourth response is in fact not used by this item -- hence the asterisk.

 

Look at the *mws cards above.  They have the same format: the column number of the item in question, followed by the number of points corresponding to each of the item's response codes.  If the item does not use one or more of the response codes, an asterisk is used.

 

Example C11:

More about the *mws card:

*col (c2-c6)

*sub Name=(Followup TV9 news quiz), Title=(NewsQuiz)

*key ACCDB

*mws c2, 1, 0, 0, *

*mws c6, 0, 1, 0, *

This example is the same as the last one.

 

We hear you saying "No it's not, come on now!  The last example used five *mws cards; now there are only two".

 

Sure.  You're right.  What we should say is that this example accomplishes the same item scoring as the last example.  Look at the three cards we've eliminated:

*mws c3, 0, 0, 1, 0

*mws c4, 0, 0, 1, 0

*mws c5, 0, 0, 0, 1

These cards say that the items found in columns 3, 4, and 5 of the Data sheet use all four response codes, have one correct answer, and award one point for the correct answer.

 

But this is the default.  Lertap assumes all items will use all response codes, have one correct answer, and will award one point when the correct answer is selected.  There's no need for *mws cards for these items -- their scoring is standard stuff.

 

So, what's special about the items in c2 and c6?  They don't use one of the response codes.  Now, this really isn't a big deal.  Lertap would process the c2 and c6 items even if we didn't mention the fact that these items use just three response codes; Lertap's various reports would simply show that the fourth option, with a response code of "D" in this case, was not selected by anyone, and the Stats1b report would flag "D" as a poorly-performing distractor.  Such things as test scores and coefficient alpha will not be not affected.

 

But why not do the job right?  Lertap allows items to have a different number of options.  The *alt card and the *mws card both allow you to set the record right, to inform Lertap that some items do not use all of the subtest's response codes.  Use these cards and Lertap's reports will look a bit cleaner.  

 

Example C12:

Still more about the *mws card:

 

*col (c2-c6)

*sub Name=(Followup TV9 news quiz), Title=(NewsQuiz)

*key ACCDB

*alt CDDDC

*mws c4, 0.5, 0, 0.5, 0

 

This example is quite typical.  The item whose responses are coded in column 4 of the Data sheet is now being double-keyed.  If someone selects the first response they get half a point.  And, if someone selects the third response, they also get half a point.  There are two "right" answers, each worth half a point. This example exemplifies one way in which partial credit may be addressed in Lertap.

 

*mws c4, 1, 0, 1, 0

 

Here again there are two right answers, but now they're each worth one point.

 

*mws c4, 0.50, -0.50, 2.00, -0.75

 

Things are getting real fancy now.  The best answer is the third one, for which a whopping two points are awarded.  The first answer is worth half a point.  The second and fourth answers now have negative scoring weights; a person selecting the second option loses half a point, whereas someone going for the fourth option will lose three-quarters of a point.

 

Example C13:

Some buildings do not have a 13th floor, and we don't have a 13th example, either.

 

Example C14:

Re-scoring all items at once:

*col (c2-c6)

*sub Name=(Followup TV9 news quiz), Title=(NewsQuiz)

*key ACCDB

*mws call, 1, 0, 1, 0

The items found in all of the subtest's columns are to be scored with one point for the first and third responses, with no points for the second and fourth responses.  This sort of scoring is not at all common for cognitive items, not at all -- but if you want to do it, you can.

 

In the world of Lertap, *mws cards are the most potent cards going.  They're dynamite.  They completely override whatever information has come on preceding CCs cards.

 

A special form of the *mws card may be used when it's desired to quickly remove an item from a subtest. Click here to read about it.

 

In Lertap Version 5.25, another special form of the *mws card was introduced.  It has this form:

 

*mws c12, 0, 1, 0, 1, other=1

 

To give credit to everyone for an item, even if they didn't answer the item, a card such as the following might be used:

 

*mws c12, 1, 1, 1, 1, other=1

 

The card above gives one point for each of the item's permitted answers, and it even gives people one point if they didn't answer the item.

 

Click here to read more about "other".

 

There are indeed times when, as in Example 10, a *mws card is used for each item.  It may be only 2% of the Lertap-using world which will have an example of this sort, but it does happen -- we've seen it.  In such a case, does the *key card make sense?  No.  But Lertap requires each and every cognitive subtest to have a *key card, so put one in (please).

 

Example C15:

Items which require multiple responses. Not covered above is the matter of scoring cognitive test items that require students to select more than one response in order to get an item correct. The example we have for this is not here, not in this topic. But we do have it covered, and invite you to click here to see our example.

 

Keep in mind that the manual has three chapters on CCs cards.  Between what's written there, and what's appeared in this help topic, we hope you'll have an adequate to good grasp of CCs cards.  But drop us a note if you've got questions: lertap5@gmail.com.