Print

Murphy, Anne Davidson

College of Education and Social Services

2007

Ed. D.

U.S. educational policy and practice has placed new emphasis on the inclusion of students with disabilities in high-stakes standardized assessments (Hehir, 2005; No Child Left Behind, 2001). In order to participate, some students with disabilities take tests with accommodation. This exploratory study asked whether test item designs with narrative and/or visual elements were more accessible to students with disabilities in Grades 3-8 - who received oral accommodation (e.g., read-aloud) on a nationally-standardized, Mathematics test. Data were collected with the TerraNova Survey E (CTBIMcGraw-Hill, 2005) in the spring of 2005 using a stratified, random sample designed to reflect the nation's population of K -12 students (N=12,380). All students who were reported to receive oral accommodation were included in the study sample (N=243). A comparison group was randomly selected after matching by school and scale score range (N=273). Item types were identified: (a) both narrative and visual, (b) narrative and no visual, (c) visual and no narrative, and (d) neither narrative nor visual. A review of item difficulty determined that, in general, a range of classical item difficulty was present at each grade for each item type (minimum overall=0.25; maximum overall=0.90). A pre-experimental, posttest-only, nonequivalent-groups design (Creswell, 2003) was then conducted using a repeated measures analysis of variance (Tabachnick & Fidel, 2001). First, orally-accommodated students' performance was examined by item type; then performance on the item types was examined by student group, with gender, race, and school as covariates.
For the oral accommodation group in Grades 5 and 7, there was a significant, within-subjects effect for perfonnance across the item types with moderate effect sizes. A main effect for group status resulted in all grades. In Grades 5, 6, and 7, there was a sipificant main effect for performance across item types with small to moderate effect sizes. Pairwise comparisons resulted in some significant differences in performance on the four item types. The orally-accommodated students in Grades 5 and 6 performed significantly better on item designs with both visual and narrative elements as compared with narrative alone. Findings suggest that items with both visual and narrative elements may allow some students who received oral accommodation to have better access to the test's construct. Item design elements, like those described in the study, could play a meaningful role in the way students demonstrate ability, since differential boosts in performance for students with disabilities may relate to item design, not just accommodation (Sireci, Scarpati, & Li, 2006). A research agenda is proposed that describes how mixed-methods item analyses could aid in the effort to develop universally-designed educational practices (Bowe, 2003; Thomspon, Johnstone, & Thurlow, 2002).