An empirical comparison of univariate and multivariate repeated measures analysis techniques when applied to motor performance data : a Monte Carlo study

Abstract

The purpose of this study was to examine, through Monte Carlo methods, the empirical alpha levels and the statistical power obtained when selected data from motor learning research was subjected to univariate and multivariate analyses. Specific purposes of this investigation were to compare results obtained from analysis of the group by trials interaction using five statistical techniques, including four variations of univariate ANOVA and one multivariate method. The univariate procedures utilized in this study were (1) the traditional repeated measures (RM) ANOVA, (2) RM ANOVA with degrees of freedom (df) adjusted with the sample estimate to Box's (1954) correction factor, (3) RM ANOVA with df adjusted with Huynh & Feldt's (1973) correction factor, and (5) the Geisser & Greenhouse (1958) conservative test. The multivariate test used in this study was Wilks' A. The five procedures were compared using four different dependent error measures, absolute error (AE), variable error (VE), constant error (CE), and total variability (E), with fixed sample sizes, n = 10, 20, or 30, and under nine degrees of violation of covariance homogeneity. For each of the nine conditions of type of covariance by sample size by dependent error measure, 5000 data sets were generated and empirical alpha levels were compared for the five statistical tests. The relative statistical power of the five tests was also compared. Results revealed that the multivariate test maintained the nominal alpha level under all conditions while violations of the assumptions inherent to ANOVA appeared to coincide with inflated Type I error levels when using uncorrected RM ANOVA. The Geisser & Greenhouse conservative test was indeed conservative as all empirical alpha levels fell more than two standard errors of a proportion below the nominal alpha levels. The remaining two adjusted univariate tests ranged from the estimate for Box's test being slightly conservative when assumptions were met to the Huynh & Feldt test being slightly liberal when assumptions were not met.