The Development of Listening and Reading Comprehension Screening Measures to Inform Instructional Decisions for End-of-Second-Grade Students

Abstract

The premise of the Simple View of Reading is that reading comprehension is the product of two components – decoding and language comprehension. Each component is necessary but not sufficient. To support teachers in identifying end-of-second-grade students who may have difficulties in one or both of the components, parallel listening comprehension and reading comprehension screening measures were developed and investigated in two preliminary pilot studies and one large-scale administration. The first pilot study, conducted with 41 end-of-second-grade students, established administration times for the listening comprehension screening (LCS) and the reading comprehension screening (RCS) and confirmed the appropriateness of the 75 items on each of the measures. The second pilot study, conducted with 12 end-of-second- grade students with varying reading levels, demonstrated that the LCS and RCS could differentiate readers with good comprehension from readers with poor comprehension. The large-scale administration, conducted with 699 end-of-second-grade students, aided in the development of shorter final versions of the LCS and RCS and provided data to determine the score reliability and validity of the final versions of the measures, each of which had 42 items.

Item response theory (IRT) was used to identify the most apposite and discriminating items for use on the final versions of the LCS and RCS. Score reliability (Cronbach’s alpha) on the final LCS was estimated to be .89 and was estimated to be .93 on the final RCS. Various sources provided content and criterion-related validity evidence. In particular, criterion-related validity evidence included strong correlations with the Gates-MacGinitie Reading Tests and strong sensitivity, specificity, and positive predictive indices. Construct validity evidence included group differentiation and a confirmatory factor analysis (CFA), all of which supported a single underlying construct on the LCS and a single underlying construct on the RCS. In a subset of 214 end-of-second-grade students from the larger study, partial correlation and structural equation modeling (SEM) analyses supported the discriminant validity of the LCS and RCS as measures of comprehension. The listening and reading comprehension screening measures will assist second-grade teachers in identifying student learning needs that cannot be identified with reading-only comprehension tests.