Here are links to some fresh videos made in 2010. iPhone and iPad users: beware. These enthralling movies use Flash.
Note inserted 9 May 2012: boring! Video (1) is good if you suffer from insomnia. (It has some good points, but is very long and the speaker sounds like he was on some sort of tranquilizer.) Here's another option, a PowerPoint presentation which should open in your browser.
(1) SMA Inggris cooks tour (about 45 minutes; dated 11 August 2010)
The scene: this "story" involves the use of Lertap 5 for processing results for a "high-stakes" test, one where high reliability was desired but not quite realized. Lertap is used to answer the question: Why wasn't the reliability higher?
The data set used in the video is from a 50-item English test, an exam found in the battery of national academic achievement tests used each year in Indonesian high schools. (SMA refers secondary education; Inggris = English.)
References: the first half of this video draws mostly from Chapters 2, 7, and 10 of the manual. The second half, starting at 27:20 minutes into the movie, uses quintile plots for "visual eye-tem analyses". There's a good reference for these plots, a very captivating read, indeed: use the links seen in the "Visual item analysis with quintile plots" paragraph, found on the "Erudite epistles" page.
Software: Windows 7 and Excel 2010 were used in the video. Results would be the same with Windows XP, Windows Vista, and Mac OSX, using Excel 2003, Excel 2007, or, for the Mac, either Excel "X", or Excel 2004.
(2) SMA Inggris item fixes (about 14 minutes; dated 12 August 2010)
The scene: this "story" involves adjusting the scoring of three multiple-choice items so that they have more than one correct answer. Multiple subtests are set up so that the resulting new test scores can be compared to the original ones. Also demonstrated in the story is the easiest way to exclude items from scoring.
The data set used in the video is from the same 50-item English test used in the story above.
The themes featured in the story may be seen here.
References: giving an item more than one correct answer involves the use of *mws control "cards". These are discussed in Chapter 3 of the manual, and also in Lelp (where you'd want to look at Example C10). Excluding an item may be accomplished in numerous ways; see Lelp. Scoring a test in multiple ways is accomplished by using multiple subtests; each subtest's *col control "card" will be the same.
(3) Trigonometry exam demonstration (about 20 minutes; dated 10 August 2010)
This video repeats much of what's seen in (1) but with a different data set, a 40-item test with particularly good statistics (reliability of 0.92, and only two problematic items). Both types of quintile plots are extensively demonstrated and discussed. Well into the video, at the 14:28 mark, a discussion of the "IStats" report begins, with mention of such statistics as the "SMC", "tetrachoric correlations", "eigenvalues", and "principal components".
References: For more about IStats, see Chapter 10 of the manual, and, especially, Lelp. The "Erudite epistles" page has a link to much more about eigenvalues; go to that page, scroll down, and find "About eigenvalues, scree tests, and coefficient alpha".
(4) Affective scale demonstration (about 16 minutes; dated 18 August 2010)
The scene: this "story" is based on an affective instrument with 48 Likert items (strongly disagree to strongly agree). The instrument had very low reliability when it was originally scored by a hasty graduate student. Detecting how to correct the scoring involved the use of a Lertap principal-components analysis (had the student paid attention to the wording of the original items, the p-comps analysis would have been unnecessary). The reliability of the instrument jumped to 0.90 once the new scoring procedure was in place.
The themes featured in the story may be seen here.
References: reversing the scoring of Likert items involves the use of a *pol control "card", as discussed in Chapter 6 of the manual, and also in Lelp. Lertap's support for principal components is mentioned in Lelp (where reference is also made to factor analysis); a captivating, inspiring discussion of principal components, scree plots, and their relationship to reliability, as indexed by coefficient alpha, may be found in this paper.
(5) Looking for group differences (about 29 minutes; dated 12 October 2010)
The scene: this group adventure tour uses a 50-item cognitive test from Indonesia, involving almost 17,000 high school students. The "Breakout score by groups" option is applied to see if there were differences in test performance by gender. Answering this question involves an analysis of variance, followed by a boxplot. This is followed, at the 11-minute mark, by use of "Ibreaks", the "Item responses by groups" option, used to see if there may have been response differences, by gender, on each of the 50 items. Response plots are created, more analysis of variance tables are drooled over, and then differential item functioning (DIF) methods are applied to index the extent of gender response differences, per item. (To see an example of how to create empirical DIF graphs, scroll in to just beyond the 24-minute mark.)
References: Lelp! Look to Lelp for the friendly web pages which underpin this tour. The "Breakout score by groups" option is here, and the material on "Ibreaks" is here. A truly top-flight, first class pdf file to download to your portable e-document reader can be had herewith.
