On Giving Test Takers a Choice Among Constructive Response Items (CT-96-03)
by Xiang Bo Wang

Executive Summary

During the past five years, there has been increased emphasis on assessing the "generative" or "constructive" process of learning. More and more testing agencies have started to incorporate constructive response items into their tests, a subset of which might be chosen by test takers.

Law School Admission Council (LSAC) is currently investigating the feasibility and advisability of administering a computerized Law School Admission Test (LSAT). Providing test takers with the option of choosing from among a set of items is certainly possible within a computer delivered assessment system. For example, in the future, LSAT takers might be given the opportunity to choose from among a list of reading comprehension and writing sample topics. However, issues such as how test takers make such choices or how such choices impact their performance have only recently been discussed. The need to understand the implications of providing test takers with choices is the impetus for this study.

Based on the data from the College Board's 1989 National Advanced Placement (AP) Chemistry Examination (The College Board, 1990) and a survey of AP chemistry teachers in Hawaii, this study investigated the relationship among the essay choices made by national test takers on five essay items, the test takers' ability levels, AP chemistry curriculum, choosing methods, and performance. Major findings are: (1) the five essays under investigation were chosen in dramatically different ways; (2) the more frequently chosen essays belonged to the core chemistry content, while the least frequently chosen item addressed a highly specialized chemistry topic; (3) there was a negative correlation between the popularity of essays and their mean scores; (4) the order in which the essays were presented seemed to have a significant effect on choice patterns of all test takers; (5) across the entire ability range, test takers who chose items selectively performed significantly higher than those who chose sequentially, possibly due to fatigue or lethargy; and (6) except for the extremely low-ability test takers, all test takers of all different ability levels seemed to choose in a similar way.
 

On Giving Test Takers a Choice Among Constructive Response Items (CT-96-03)

Research Report Index