| dc.description.abstract | Research-based decision making has been advanced as a way for
professionals to make a determination about the effectiveness of a potential
treatment. However, informed consumers of research need to be able to
determine what constitutes evidence-based practices and what criteria can be
used to determine if evidence-based practices have been met.
This study was a synthesis of research that involved a critical review of
the empirical research studies reported in Volume 47 of the Journal of Speech,
Language, and Hearing Research (JSLHR) published in 2004. This
methodological research synthesis evaluated (a) the research designs used in
the JSLHR studies, (b) information and rationale used to inform population
validity assessment decisions, and (c) the extent to which the sampling designs,
population validity rating, data analysis procedures, and the specification of
generalizations and conclusions provide sufficient evidence to determine an
overall rating of conclusion validity. Results indicated that less than one-fifth of the 105 research synthesis
population of studies used experimental research designs. Additionally, the vast
majority of the research synthesis population of studies (83.8%) were
observational research designs.
Only five studies out of the research synthesis population of studies
(4.8%) were determined to have high population validity. In contrast, 84.8
percent of the research synthesis population of studies were found to have low
population validity. That is, the studies did not contain adequate information or
description of the essential sampling concerns.
The vast majority or 75.3 percent of the research synthesis population of
studies were rated as having low conclusion validity. Approximately one-fifth of
the 105 research synthesis study population (22 studies or 20.9%) were found to
have moderate conclusion validity while less than five percent of the total studies
(4 of 105 studies or 3.8%) were found to have high conclusion validity.
A meaningful relationship between population validity ratings and
conclusion validity ratings was established. Since 81 of 105 studies have
identical ratings for both population and conclusion validity, the accuracy of the
prediction model developed for this study is 77.1 percent. | en |
