Approaches to Identifying the Optimal Number of Factors in Multilevel Exploratory Factor Analysis

Abstract

In exploratory factor analysis (EFA), there are numerous methods to extract the optimal number of factors, and these methods can be generally divided into two categories, namely, the model-selection-based approaches and the eigenvalue-based approaches. The model-selection-based approaches exploit the commonly used model-fit indexes and selection criteria, include the Root Mean Square Error of Approximation (RMSEA), the Comparative Fit Index (CFI), the Tucker-Lewis index (TLI), the Standardized Root Mean square Residual (SRMR), the overall model-fit chi-squared test, the Akaike information criterion (AIC), the Bayesian information criterion (BIC), and the sample-size adjusted BIC (SBIC/SABIC). On the contrary, the eigenvalues-based approaches rely on eigenvalues, include the “eigenvalues > 1” criterion, the scree plot test and parallel analysis (PA). The purpose of this dissertation is to examine the accuracy of these two approaches in identifying the number of factors in multilevel exploratory factor analysis (MEFA) through two studies, separately. In both studies, multilevel data were designed and simulated. In the first study, we used both the model-based and the design-based approaches for the analysis. The results of Study 1 disclosed: (a) as the model-based approach showed most of the commonly used fit indexes and selection criteria were effective in identifying the correct number of factors at the within level, except for level-specific SRMR and AIC, while most of them tended to extract fewer factors at the between level, (b) as the design-based approach showed, most of them were able to identify the true model as the best solution, except for SRMR, AIC, and ΔAIC. In the second study, we used different eigenvalues extraction techniques for calculating eigenvalues, such as principal axis factoring (PAF), iterated principal axis factor (Iterated PAF), and maximum likelihood (ML). The results of Study 2 revealed: (a) the “eigenvalues > 1” criterion was not effective to search for the optimal number of factors, (b) PA approach performed well in recovering the correct number of factors at the within level, while the performance of PA was related to sample size and ICC at the between level. In addition, PAF performed the best followed by Iterated PAF and then ML at both levels.