Quantoids corner-bifactor and second-order FA comparisons-Guest post by Matthew Reynolds

The following is a guest blog post by Matthew Reynolds, one of Tim Keith's Doctoral Student in Educational Psychology (School Psychology & Quantitative Methods) at the University of Texas at Austin, Department of Educational Psychology.

This is an excellent post by a future quantoid to be reckoned with in the field of school/educational psychology research. Kudos to Dr. Tim Keith for suggesting that one of his doctoral students make a quest blog post. This is the first such doctoral student virtual scholar post. If there are other professors who would like to entertain the idea of doctoral students being assigned articles to review and prepare for guest posts on IQ's Corner, then drop me an email.... iap@earthlink.net

• Chen, F. F., West, S. G., & Sousa, K. H. (2006). A comparison of bifactor and second-order models of quality of life. Multivariate Behavioral Research, 41, 189-225. (click to view)
  

Although not directly related to intelligence, this article compares two confirmatory factor analytic (CFA) models frequently used in psychometric research of intelligence: bifactor and second-order models. Chen et al. (2006) describe the bifactor model as having a general factor that accounts for the communality in all items and domain specific factors that account for influences above and beyond the general factor. The second-order model is described as having interrelated first-order factors with a general factor that accounts for those relations.

Study 1 compared the two models by applying the factor structure to a quality of life measurement from the AIDS Time-Oriented Health Outcome Study. Study 2 was a Monte Carlo study investigating whether there was enough power to detect differences in the bifactor and second-order model. Previous research had suggested that it was empirically impossible to distinguish between the two in typical samples used in social science research (i.e. Mulaik & Quartetti, 1997).

Results from Study 1:

• Bifactor and second-order factor models were imposed on a 17 item health-care related quality of life survey. The models had a general overall quality of life factor and four domain specific factors. The four domain-specific factors included cognition, vitality, mental health, and disease worry.

• The results from the bifactor model suggested that the mental health factor did not provide unique information above and beyond the general factor. Therefore, the model was re-specified without a mental health factor.

• The second-order factor model was specified with four first-order factors and a general quality of life factor that accounted for the relations among the first-order factors. The residual variance for the mental health factor, however, was statistically significant suggesting that there was some unique contribution of this factor (although the general factor accounted for 91.4% of the variance in that factor). Note this finding was different from the bifactor model. In the bifactor model the mental health factor did not provide unique information. Therefore, to be consistent with the bifactor model the authors also re-specified the second-order model so that only three factors, and the subtests related to mental health factor loaded directly on the second-order factor.

• The results comparing the two different models showed that both the bifactor and second-order factor models provided adequate fit. Because the second-order model is a more constrained version of the bi-factor model, the likelihood ratio test (i.e., chi-squared difference test) was used to compare the fit of the models. The second-order model fit worse than did the bifactor suggesting that the constraints applied to the bifactor model to get to the second-order model were too restrictive. Also, a power analysis suggested that there was adequate power to detect the difference.

• Next, the authors used these models to predict social functioning. Both models resulted in almost identical standardized estimates. This finding was rather reassuring in regards to the interpretability of the ability factors.

Study 2:

• The findings suggested that even with a sample size of 200 there appears to be enough power to detect differences between the bifactor and second-order models.

Discussion:

• The authors concluded that the bifactor model offers several advantages over the second-order model. One advantage was that it identified three factors instead of four. I am not quite convinced that this is necessarily an advantage. Two, they noted that researchers may miss potential non-significant first-order factor variances when looking at their results. I thought this was a good point by the authors; however, I also have had the same concern about using bifactor models. For example, a not-so-careful researcher may not consider the non-significant domain specific factor loadings as well as a non-significant domain specific factor variance.

• The second advantage was that the bifactor model fit better. That is, the relations between the general factor and the items could not be fully mediated by the first-order factors.

• Third, they stated that the bi-actor model is easier to interpret when predicting external criteria because the domain factors are represented as common factors in bifactor models whereas they are residualized factors in the higher-order model. Although true, I think the point is rather minor.

• Last, and perhaps most importantly, they conclude that BOTH models are useful in research. I agree completely with this point as CFA models should be consistent with theoretical models.

• In general, the article provides great information for those interested in hierarchical factor analysis, and it is provided in a straightforward manner. I think that the advantages of the bifactor model were a bit overstated. I do agree that it is useful to examine both models in research, especially since the second-order model can be derived from the bifactor model.

• In my own research, one weakness of the bifactor model has been related to empirical under-identification. I believe that perhaps it runs into some of the same difficulties as the multi-method multi-trait models in that they are over-parameterized. A recent study that used the bifactor model to test for method effects also found that the bifactor model may fit well even when it is an incorrect model (Maydeu-Olivares & Coffman, 2006).

• In terms of research in psychometric intelligence, the interpretation of the two models is slightly different as well. For example, in a bifactor model all of the effects of the general factor are direct. In intelligence research it seems to me that the contemporary theories are more consistent with the higher order model in which the general factor explains the interrelations of the broad abilities and its relation to test performance is mediated through the broad abilities.

• To make this more germane to intelligence researchers I have included some output of analyses that I performed using the Holzinger & Swineford correlation matrix reported in their 1937 study. Shown are the specifications, the models with standardized loadings, and the unstandardized loadings, variances, and the total effects shown separately. Just as a warning, the models are not in publication form, but suffice for a demonstration. I hope these models help to clarify how the second-order model is in fact a more constrained version of the bifactor model. See Yung, Thissen, and McLeod (1999) for a more technical account.

• Last, as an aside, I thought I would share the last two sentences from the Holzinger & Swineford 1937 article in Psychometrika. In this article the authors introduced the bifactor model:

• “The Bi-factor analysis illustrated above is not only very simple, but the calculation is relatively easy as compared with other methods. The total time for computation, done by one person, was less than ten hours for the present example.”

• I just ran a bifactor model in Amos 5, and other than setting the model up, the actual computational time took 0.29 seconds. You have to appreciate all of the time and patience that researchers have put in over the years to get us where we are today!

Technorati Tags: psychology, educational psychology, neuropsychology, statistics, factor analysis, confirmatory factor analysis, quality of life, QOL, g, general intelligence

powered by performancing firefox