A Comprehensive Empirical Investigation on ANOVA Effect Sizes
Abstract
The present journal article formatted dissertation was a comprehensive investigation of ANOVA effect sizes. In the first study, the author examined the extent to which ANOVA practices have changed in comparison with a methodology review conducted 15 years ago, which include the examination of validity assumptions, sample sizes, and effect size indices. The author reviewed all articles published in 2012 in three APA journals (Journal of Applied Psychology (JAP), Journal of Counseling Psychology (JCP), and Journal of Personality and Social Psychology (JPSP). Results indicated that the use of ANOVA is proportionally less than previously indicated, but still very popular in practice. Researchers still rarely verify whether the validity assumptions are satisfied, but reporting effect size statistics is on the increase.
In the second study, the author examined the accuracy and robustness of estimates of practical significance (i.e., η^2, partial η^2, ε^2, partial ε^2 , ω^2, and partial ω^2,) in a 2 × 3 two-way fixed-effects ANOVA. The study extended the exploration of these effect sizes in the presence of assumption violations and is generalized to the more common case of multi-factor ANOVAs. The results revealed that: the classical forms were more stable; ε^2 and ω^2 were not always better estimates than η^2; sample sizes, group size ratio, heterogeneity of variance, population effect sizes, pairings, and degrees of freedom all affected the effect sizes estimate.
In the third study, the author examined the accuracy and robustness of estimates
of Intraclass Correlation Coefficient (ICC) in a 2 × 3 two-way mixed ANOVA. Results indicated that the accuracy and robustness of estimation were mainly affected by two components: sampling error due to random-effects and sampling error due to random sampling of a sample. ICC estimates are robust across different studies as long as the number of levels for the random effect is the same. Researchers should be cautious to utilize the ICCs’ estimates when the design differs from the design investigated here.
Citation
Zhou, Yuanyuan (2015). A Comprehensive Empirical Investigation on ANOVA Effect Sizes. Doctoral dissertation, Texas A & M University. Available electronically from https : / /hdl .handle .net /1969 .1 /156155.