Cognitively affected, no doubt

Sixty fortunate people participated in a workshop designed to introduce them to Lertap 2, the mainframe version used before desktop computers came on the scene.

At the end of the workshop each participant answered a 37-item "test". The first twenty five items were cognitive ones, multiple-choice in format; some of these items used up to six options, A through F, while others used fewer options. These items were meant to reflect how well each person had mastered the content of the workshop.

The twenty five cognitive items were followed by ten Likert-style affective questions, items which gave respondents the opportunity to say what they thought about the software.

The last two items, Q36 and Q37, were open-ended. The first of these asked people to indicate how long they had been using computers, while the second requested information on how long they had been using tests as part of their work.

Partial responses for five of the respondents are shown in the table below:

This table is really a snapshot of an Excel worksheet. We have had Excel hide many of the worksheet's columns so that the table fits on this page, while still letting you see how the various responses were entered: the multiple-choice responses were coded as upper-case letters; the Likert responses were coded as digits; the "years-using" responses were real numbers.

The twenty-five multiple choice item responses occupy columns 2 through 27 of the worksheet; the ten Likert responses are found in columns 28 through 37; years using computers is in column 38; years using tests is in column 39.

The next table displays the lines of control code used to define the cognitive and affective subtests:

If you've followed the examples preceding this one, some of the content of these lines may be familiar.

The cognitive subtest, "Knwldge", is more complex than the Class quiz and Entrance exam examples because its items use more options, up to F of them (up to six of them), and because not all items use all options, or alternatives. The *alt line indicates the last alternative used by each item— for example, the first entry in the *alt line, C, means that the first item used A, B, and C as its options.

The lines for the second subtest, "Comfort", will be familiar if you've already looked at the CEQ survey example. The Comfort subtest's Likert items used four positive stems, and six negative ones.

What's the Wt=0 thing doing on the *sub lines? It's telling Lertap that each subtest is to be excluded from the calculation of a total score— ordinarily, Lertap will add up the score from the subtests to produce a Total score, as seen in the Entrance exam example. But in this case one of the tests is cognitive, while the other is affective— it doesn't make much sense to add these test scores together, so we've used Wt=0 to tell Lertap to forget about making a total score.

The statistics section of the Scores report produced by Lertap looked like this:

There was a fairly comfortable correlation, 0.80, between Knwldge and Comfort, suggesting that those who got high scores on the multiple-choice items tended to have a more favourable opinion of the software.

Chapter 2 of the manual has a more detailed discussion of the Lertap Quiz results, including methods for using Lertap to see if there was a relationship between the two subtest scores and the years-using questions— for example, we might be interested in determining the extent of the relationship (if any) between the Comfort subtest score, and years using computers.