What a revue

I have been using Lertap since May 2003 and have come to depend on it for the scoring and analysis of several of our examination programs. This is a remarkably powerful item analysis tool that is easy to use, especially as it is based on Excel, a system we use often. It is able to cope with data in a variety of formats using Excel's data importing capability, and Lertap's "spreader", to distribute candidate responses into separate cells for analysis.

Lertap produces a variety of useful reports and graphs, all easily printed using Excel's standard print options. Furthermore, you can sort the Lertap results, for example to locate the most difficult items on a test form, without compromising the overall summary statistics. Subtests are easily created within Lertap and the analysis includes intercorrelations between the subtests and the test as a whole. Analyses of subgroups can also be done using the newly-enhanced *tst control card.

Perhaps most important is the excellent support that Dr. Larry Nelson provides for Lertap users. The Lertap manual and website are both entertaining and comprehensive. Answers to virtually all Lertap questions can be found using these resources. On those rare instances when an answer proves elusive, Dr. Nelson is quick to respond to inquiries and he has on several occasions made changes to Lertap in response to our suggestions.

I highly recommend Lertap to anyone needing a user friendly test and survey analysis package.

Fae Mellichamp
Psychometric Director
Professional Testing, Inc.
1705 Metropolitan Blvd., Suite 102
Tallahassee, Florida 32308
(850) 386-4444, fax (850) 386-2404
USA

My company works through the non-profit Stupski Foundation to provide assessment support to local school districts here in the U.S. I want to pass along the positive response I get every time I demonstrate what Lertap can do. The educators I work with are especially pleased with the charting capabilities. The graphs make it much easier for them to understand item performance (especially the quintile breakouts...it is like I can see the light bulbs go on over their heads and they immediately start looking a the content of the item and work to relate the content to the data). I really think Lertap's output provides a more intuitive way of communicating item data to front-line educators.

Denis W. Jarvinen, Ph.D.
Director of Research and Evaluation
MetriTech, Inc.
800.747.4868
djarvine@metritech.com

I've been a user of LERTAP for more than 25 years and have been witness to LERTAP's growth from a mainframe based application to its current form. LERTAP has continually been a fixture in my array of computing tools. I have used it both as a research tool and as a teaching tool. It has an easy to learn syntax, generates all of the basic psychometric statistics and includes a host of additional information. There is no other single piece of psychometric software as complete and as versatile as LERTAP on the market. LERTAP is a must for every psychometrician's bag of tools.

Dwight D. Harley, Ph.D.
Director of Psychometrics and Adjunct Associate Professor
Division of Studies in Medical Education
Faculty of Medicine and Dentistry
University of Alberta
Edmonton, Alberta, Canada
dharley@ualberta.ca

Larry: I have used Lertap for processing student level assessment data and running initial runs of PISA, TIMMS, and NAEP data as needed. I have found it to be one of the most helpful and reliable pieces of software in my portfolio. Thanks for your work in gradually improving it and bringing it closer and closer to simulating IRT analyses within the classical approach.

John A. Dossey
Distinguished University Professor of Mathematics Emeritus
Illinois State University

LERTAP is a very effective computer program that provides data for item and test analysis within the framework of classical test theory. For an introductory treatment of educational assessment using multiple choice questions and Likert type attitude scales it is easy to use, with informative tables and graphs and very attractively presented. With laptop computers becoming readily available to every classroom teacher, this computer program is now accessible to all teachers who would wish to monitor their use of classroom assessment procedures.

John P. Keeves
Professorial Fellow
School of Education
Flinders University
Adelaide, South Australia

Lertap is an excellent, easy to use program for classical item analysis, and by far the best tool for this kind of work on the Mac. Their tech support is magnificent, a real plus for any software program. When I was having problems setting up an analysis, I even got help from out in the field, from the wilds of northern Thailand, no less. Highly recommended.

Michael 'Rube' Redfield
Associate Professor
Faculty of Human Sciences
Osaka University of Economics
Japan

I'm a human performance consultant with a PhD in Measurement, Evaluation and Statistics in Social Sciences from the University of Chicago (1973). Since then, I have had only rare opportunities to use my item analysis skills. However, I recently had to help someone put together a standard cognitive assessment, so I needed some software and a refresher course.

I am happy to say that Lertap provided everything I required. What I appreciate most is the author's ability to communicate the essence of the statistical issues and applications. His writing style and sense of humor do a great job of pulling the topic off of its grandiose, theoretic and academic pedestal down onto the common ground upon which most mortals walk. Related to that is his willingness (and ability) to separate the important from the trivial , e.g., by not pretending that the differences among a half dozen types of discrimination statistics are of much practical use in most situations. I also have found the various levels of statistical summaries to be very nicely thought out— their practical value reflects a firm understanding of applied statistics.I also want to say something I never thought I would ever hear myself say about a stat program: I found it fun to use! I don't think I have ever before cracked a smile upon receiving an error message.

A great software package!

John Eggert, Ph.D.
President
The Idea Leadership Company
Chicago, Illinois USA
Jeggert@IdeaLeadership.com

Después de varios años utilizando diversas versiones del programa Lertap a nuestro proceso de selección para ingreso a la Facultad de Ingeniería de la UCV, donde se toman seis pruebas con un total de cien preguntas a varios miles de participantes, puedo dar fe de que el programa funciona de manera confiable y segura, al mismo tiempo que es facil de utilizar. La salida del programa es de grán utilidad para los docentes y para quienes administran la prueba. Por estas razones no dudo en recomendarlo a quienes requieran realizar análisis clásico de reactivos para la verificación de la consistencia interna de las subpruebas y de la prueba completa.

Carlos E. González G.
Master en Física
Universidad Central de Venezuela
Profesor del Dpto de Física Aplicada FIUCV
Miembro de la Comisión de Ingreso de la FIUCV
Caracas, Venezuela
dejgonzalez@cantv.net

Lertap is an EXCELLENT tool for test scoring and item analyses. The learning curve is reasonable and the technical support is superb! Any statistics or analyses which Lertap does not already do, can usually be added easily into the Excel worksheet. The only drawback is that, by using Excel, one is limited to 256 columns. But for tests with less than 250 items (and six identifiers) Lertap is a remarkably well-designed software program.

Barbara Foster, Ph.D.
University of Texas
Southwestern Medical Center
Dallas, Texas USA

Item analysis is critical for evaluating and improving in-house examinations in medical school. These examinations are often of poor quality because of limited time for item writing. Our studies suggest that feedback of item analysis data will help item writers improve quickly and contributes to higher quality items. Small medical schools, like ours, place a high value on classical item analysis because sample size requirements of IRT methods are often prohibitive.

Lertap 5 allows us to generate CTT data quickly and easily for both our OSCE checklists / rating scales and MCQs. Our use of Lertap began with the development of quality assurance procedures for our OSCE measures. We also saw the value of using LERTAP to supplement data from our existing procedures for analyzing written examinations. There are currently very few software packages which can provide data for the extended option sets (5 to 26 options) used in EMQs. This was seen as a major weakness since this format is increasingly used in medical education certifying tests. Since LERTAP can now provide data for 10+ options, we have the opportunity to evaluate the quality of our EMQ options sets.

Thus, even with access to a wide variety of programs, LERTAP has become the program of choice for generating data both for research and item writer feedback. One of the more useful features is that it allows us to provide a variety of information, trace lines, choice means, and point biserials for modifying distractors. From May 2005, we also hope to extend use of this program to analyzing our short answer questions (SAQs), for which we have developed procedures jointly with the Trinidad and Tobago Ministry of Education.

Jerome De Lisle
Department of Measurement and Evaluation
Measurement & Evaluation Officer
Centre for Medical Sciences Education
Faculty of Medical Sciences
University of the West Indies, St. Augustine
Trinidad and Tobago, West Indies
jeromedelisle@yahoo.com

En la Comisión de Exámenes del Departamento de Idiomas hemos contado con Lertap desde los años 80, cuando el entonces Centro de Informática y Comunicaciones corría Lertap 2 en una mainframe sobre datos que nosotros enviábamos en papel y que allá digitaban. Esto nos permitía obtener al cabo de unos seis meses una apreciación estadística de la calidad de nuestros exámenes departamentales. Este tipo de exámenes se sigue administrando a unos 600 estudiantes dos veces por trimestre para evaluar su comprensión de lectura en inglés técnico-científico.

Hace un par de años adquirimos Lertap 5 para tener dentro del propio departamento un medio de hacer el análisis de ítemes por nosotros mismos. El sistema es amistoso y, puesto que corre en Excel, se hace fácil familiarizarse con él. Ahora contamos con los resultados del análisis apenas un par de semanas después de que los profesores nos devuelven las hojas de respuesta. Además nos satisface saber que las mejoras del paquete corrigen la inflación en las correlaciones punto-biseriales que se daban en las versiones anteriores y proveen un mapa de bandas de dificultad y discriminación para cada examen.

Recientemente nos afanábamos por producir gráficos de huella (trace line plots) para 90 ítemes en un estudio sobre el uso de tres opciones en exámenes de selección simple. Nos tomó varias horas de ensayo y error hacer los gráficos para apenas 25 de estos ítemes. Cuando descubrimos que Lertap 5.4.6 incorpora entre sus nuevas capacidades la de hacer justo lo que necesitábamos, bajamos la versión de prueba. Lertap hizo los 90 gráficos en alrededor de tres minutos.

Los materiales de apoyo (manual, atención en línea) son muy buenos.

Recomendamos Lertap a cualquier profesional de la educación que desee contar con un software de análisis clásico de ítemes.

Carlina Rojas, Noela Cartaya, Yris Casart, y Gilberto Berríos (jefe)
Departamento de Idiomas
Universidad Simón Bolívar
Caracas, Venezuela
gberrios@usb.ve