Reassessment of innovative methods to determine the number of factors: A simulation-based comparison of exploratory graph analysis and next eigenvalue sufficiency test. - PsycNET
https://psycnet.apa.org/doiLanding?doi=10.1037%2Fmet0000527
Brandenburg, N., & Papenberg, M. (2022). Reassessment of innovative methods to determine the number of factors: A simulation-based comparison of exploratory graph analysis and next eigenvalue sufficiency test. Psychological Methods. Advance online publication. https://doi.org/10.1037/met0000527
Next Eigenvalue Sufficiency Test (NEST; Achim, 2017) is a recently proposed method to determine the number of factors in exploratory factor analysis (EFA). NEST sequentially tests the null-hypothesis that k factors are sufficient to model correlations among observed variables. Another recent approach to detect factors is exploratory graph analysis (EGA; Golino & Epskamp, 2017), which rules the number of factors equal to the number of nonoverlapping communities in a graphical network model of observed correlations. We applied NEST and EGA to data sets under simulated factor models with known numbers of factors and scored their accuracy in retrieving this number. Specifically, we aimed to investigate the effects of cross-loadings on the performance of NEST and EGA. In the first study, we show that NEST and EGA performed less accurately in the presence of cross-loadings on two factors compared with factor models without cross-loadings: We observed that EGA was more sensitive to cross-loadings than NEST. In the second study, we compared NEST and EGA under simulated circumplex models in which variables showed cross-loadings on two factors. Study 2 magnified the differences between NEST and EGA in that NEST was generally able to detect factors in circumplex models while EGA preferred solutions that did not match the factors in circumplex models. In total, our studies indicate that the assumed correspondence between factors and nonoverlapping communities does not hold in the presence of substantial cross-loadings. We conclude that NEST is more in line with the concept of factors in factor models than EGA. (PsycInfo Database Record (c) 2022 APA, all rights reserved)
Impact Statement
Exploratory factor analysis (EFA) is a method to develop hypotheses concerning common factors governing correlations among variables. This makes EFA a valuable instrument in various fields of psychology (such as test development). A key problem in EFA is to determine the optimal number of factors that fits observed correlations and keeps resulting models parsimonious. Contemporary research on this problem does not provide consensus on the optimal solution. Next Eigenvalue Sufficiency Test (NEST; Achim, 2017) and exploratory graph analysis (EGA; Golino & Epskamp, 2017) are recently proposed methods to approach this problem. Both were shown to determine accurately the number of factors in simulated factor models in which variables indicated one factor each. In our report, we compare NEST and EGA with simulated factor models in which each variable indicated multiple factors to varying degrees. These conditions suit validation of methods to detect factors because the premise of an unknown number of factors implies that one may not assume how many factors link to individual variables. We conducted two simulation studies: In Study 1, we show that methods detect factors less accurately when variables indicated multiple factors each and highlight that EGA suffered stronger than NEST. In Study 2, we simulated circumplex models—a particular class of factor models—and show that NEST achieved high accuracy while EGA was strikingly inaccurate. We discuss reasons for the methods' performances and argue that the signal that EGA detects is incongruent on a statistical level with the understanding of factors in factor analysis. (PsycInfo Database Record (c) 2022 APA, all rights reserved)
https://psycnet.apa.org/doiLanding?doi=10.1037%2Fmet0000527
Brandenburg, N., & Papenberg, M. (2022). Reassessment of innovative methods to determine the number of factors: A simulation-based comparison of exploratory graph analysis and next eigenvalue sufficiency test. Psychological Methods. Advance online publication. https://doi.org/10.1037/met0000527
Next Eigenvalue Sufficiency Test (NEST; Achim, 2017) is a recently proposed method to determine the number of factors in exploratory factor analysis (EFA). NEST sequentially tests the null-hypothesis that k factors are sufficient to model correlations among observed variables. Another recent approach to detect factors is exploratory graph analysis (EGA; Golino & Epskamp, 2017), which rules the number of factors equal to the number of nonoverlapping communities in a graphical network model of observed correlations. We applied NEST and EGA to data sets under simulated factor models with known numbers of factors and scored their accuracy in retrieving this number. Specifically, we aimed to investigate the effects of cross-loadings on the performance of NEST and EGA. In the first study, we show that NEST and EGA performed less accurately in the presence of cross-loadings on two factors compared with factor models without cross-loadings: We observed that EGA was more sensitive to cross-loadings than NEST. In the second study, we compared NEST and EGA under simulated circumplex models in which variables showed cross-loadings on two factors. Study 2 magnified the differences between NEST and EGA in that NEST was generally able to detect factors in circumplex models while EGA preferred solutions that did not match the factors in circumplex models. In total, our studies indicate that the assumed correspondence between factors and nonoverlapping communities does not hold in the presence of substantial cross-loadings. We conclude that NEST is more in line with the concept of factors in factor models than EGA. (PsycInfo Database Record (c) 2022 APA, all rights reserved)
Impact Statement
Exploratory factor analysis (EFA) is a method to develop hypotheses concerning common factors governing correlations among variables. This makes EFA a valuable instrument in various fields of psychology (such as test development). A key problem in EFA is to determine the optimal number of factors that fits observed correlations and keeps resulting models parsimonious. Contemporary research on this problem does not provide consensus on the optimal solution. Next Eigenvalue Sufficiency Test (NEST; Achim, 2017) and exploratory graph analysis (EGA; Golino & Epskamp, 2017) are recently proposed methods to approach this problem. Both were shown to determine accurately the number of factors in simulated factor models in which variables indicated one factor each. In our report, we compare NEST and EGA with simulated factor models in which each variable indicated multiple factors to varying degrees. These conditions suit validation of methods to detect factors because the premise of an unknown number of factors implies that one may not assume how many factors link to individual variables. We conducted two simulation studies: In Study 1, we show that methods detect factors less accurately when variables indicated multiple factors each and highlight that EGA suffered stronger than NEST. In Study 2, we simulated circumplex models—a particular class of factor models—and show that NEST achieved high accuracy while EGA was strikingly inaccurate. We discuss reasons for the methods' performances and argue that the signal that EGA detects is incongruent on a statistical level with the understanding of factors in factor analysis. (PsycInfo Database Record (c) 2022 APA, all rights reserved)
******************************************
Kevin S. McGrew, PhD
Educational & School Psychologist
Director
Institute for Applied Psychometrics (IAP)
https://www.themindhub.com
******************************************
Kevin S. McGrew, PhD
Educational & School Psychologist
Director
Institute for Applied Psychometrics (IAP)
https://www.themindhub.com
******************************************
