Online assessment: a study of the validation and implementation of a formative online diagnostic tool in developmental mathematics for college students

Abstract

This Research and Design (R&D) study models the methodology necessary to replicate an online assessment instrument designed to assess student skills and facets of thought while understanding Multiple Meanings and Models of Fractions (MUL) in college level developmental mathematics. The researcher used cognitive research done in the area of fractions to design this instrument that both documents and assesses facets of thought or reasoning strategies used by students. The final facet cluster is a table that ranks these facets from least to most problematic, documenting the student facets of thought across the content objective MUL. Over 500 student and teacher participants were used in the design and development of Fraction Diagnoser. All participants were affiliated with college developmental mathematics in Texas, representing four colleges and universities. Forty-eight student participants were individually interviewed to ascertain facets of understanding on the topic of MUL. Seven teacher participants were individually interviewed as to the effectiveness of Fraction Diagnoser in the classroom after the final step of the R&D cycle. Content experts were used to design the questions assessing skills and facets. Fraction Diagnoser was built using the Borg and Gall R&D cycle as its blueprint. Nine of the ten steps of the R&D cycle were used in the development of the instrument, excluding just the final product revision due to cost and time restraints. According to Borg and Gall (1996), a dissertation R&D should be limited to a few steps, but all of the steps used for this R&D allowed for the researcher to completely address all of the research questions. During the steps of the R&D cycle, validation and reliability analyses were done to statistically address the effectiveness of Fraction Diagnoser. Final interviews with the teacher participants supported findings in recent research on the effective use of online assessment. Implications for practice and recommendations for further study were also addressed.