"Carroll" Analysis of 50 CHC-designed tests: Part 1

As promised in March 14 post (So much data…so little time: In honor of Jack Carroll), this is a first attempt to present previously unpublished research results for dissemination and hopefully some comment-driven discussion on this blog. This activity will be spread across multiple posts, so be patient. I don’t want anyone exceeding their RDA (recommended daily allowance) of data.

The Analysis
Following my personal Fairbanks, Alaska tutoring session with Jack Carroll on his self-written suite of Schmid-Leiman (SL) EFA DOS-based software, I ran an analysis (in Dec 2003 – Note to self - publish soon, or results will perish/languish on your hard drive) as per the steps described in Chapter three of his seminal treatise (Human Cognitive Abilities). I ran the analyses on 50 individual test variables from the WJ III and WJ III Diagnostic Supplement, a battery of tests designed as per CHC theory. The analysis was on all norm subjects from ages 6 through adulthood (click here to go to page where you can download ASB2, which is a technical abstract that describes the WJIII norm sample)

As I’ve written elsewhere (CHC Theory: Past, Present and Future), Jack, during his later years, had clearly moved beyond sole reliance on his SL-EFA procedures and had embraced confirmatory factor analyses (CFA) methods. Jack had blended the two methodologies as can be seen in his last published book chapter (The Higher-stratum Structure of Cognitive Abilities: Current Evidence Supports g and About Ten Broad Factors in Nyborg's The Scientific Study of General Intelligence [2003]). As an aside, while visiting Jack in Fairbanks, I found his computer disks were full of unpublished EFA+CFA analyses that he had graciously completed for other researchers or, that represented his analysis of correlation matrices that had been included in manuscripts he had been asked to review for a number of journals. It was clear his factor analytic approach had clearly evolved to one of first obtaining results from his EFA SL approach and then using those results as the starting point for CFA refinement and model testing.

Following Jack’s lead, after running his SL-EFA procedures on the 50 variable WJIII/DS dataset, I used the final SL-EFA solution as the initial starting point for further model “tweaking” via CFA methods. I believe this use of CFA methods is often referred to as model generation CFA, to keep it distinct from model confirmation CFA. Yes, the inferential statistics are shot to hell when doing this.

The Results: Phase I
I’ve now posted a portion of the results (yes…I’m going to meter these results out over time). By clicking here, you will be taken to a pdf file that includes: (a) a listing and description of the WJ III and DS tests included in the analyses (you will need this as your key to the results) and (b) a summary, and my interpretation, of the first-order factor results. Please note, as frequently pointed out by Dr. Carroll, that there is a difference between the order and stratum of a factor. I’ve made my interpretations as per his logic regarding the breadth of the factors.

For now, I’m just posting these first-order findings “as is”….so all you quantoids can download/print the results and run your fingers through the numbers and generate hypotheses to your hearts content. My factor interpretations are represented by the CHC factor codes and names (along the top and down the left hand column). If you are unfamiliar with the CHC factor codes and terms you should visit (and print for reference) the CHC Definitions at the IAP CHC Definition Project page. Burn the names and codes into your brain. It is critical you learn CHC-SL (CHC as a second language).

Note that in the summary of the results I’ve had to propose some new narrow ability factors to account for some of the findings (comments, including possible definitions for these “newbies” is very much welcomed). Also, if you are new to CHC theory, it is recommended that your first read CHC Theory: Past, Present and Future to secure the necessary background information to interpret and understand the findings and the discussion to follow.

Comments

• Please note that my posts (at least initially) are going to focus on the structure of the CHC theory and not so much on the WJ III instrument and tests --- but this does not preclude other folks from taking a test validity approach to interpretation. Isn’t this fun?

• Please note that I’m holding back what I think are the most exciting results—the second- and third-order factor results. Yes, the final solution is a hierarchical solution with three levels. The higher-order results, IMHO, although speculative, are very exciting.

• I’ll post comments and interpretations over the next few days (weeks?). Consider this an on-line instructional exercise.

• Let the games begin.