More on the problem with the -1 SD [15 SS (3 ss)] IQ subtest discrepancy rule-of-thumb

In a prior post I raised concerns about the use of the 1 SD (15 SS/3 ss) rule-of-thumb for evaluating differences between two IQ subtest scores that are part of the same composite or cluster. My central point was that this simplistic rule-of-thumb fails to incorporate information regarding the cohesiveness or inter-correlation of the tests within a cluster. More importantly, some human ability domains are more cohesive/tight (e.g., Gc) than others (Gv), and the resulting correlation between two compared tests require the use of the SD (diff) formula that incorporates the correlation between the tests within a domain that are to be compared.

I presented estimated SD (diff) values for select subtest comparisons within the WISC-IV and WJ III in different construct domains. The estimates used the SD (diff) formula that includes the correlation between the measures to be compared.

Knowing that some folks don't like formula's and estimates, I decided to make the point more concrete with real data. A picture is worth a thousand words (or equations).

In the prior post I reported an estimated SD (diff) for the comparison of the WJ III Verbal Comprehension and General Information Gc tests of 9.9, based on their average correlation (across all norm subjects) of .78.

Today I went to the WJ III NU norm data and subtracted all General Information SS's from Verbal Information SS. I then calculated summary stats and generated the histogram below. [Click on image to enlarge]



Beautiful...don't you think? A normal distribution centered on zero (Mean = -0.5) and with an actual data-based SD of 9.8 (9.8 is almost identical to the 9.9 value resulting from the equation method).

Study the graph. It clearly shows that if clinicians want to determine if the WJ III Verbal Comprehension and General Information SS's are 1 SD different (1 SD[diff], technically), then a difference of approximately 10 points is what an examiner should look for...not 15! If an examiner uses the inaccurate rule-of-thumb (i.e, difference of 15 points is 1 SD), in reality the examiner, in the case of these two WJ III Gc tests, is actually requiring a difference of -1.5 SD (diff)....or 15 points.

See prior post for lengthier discussion of the logic, equations, and danger in invoking a subtest difference rule-of-thumb of -1 SD=15 (or, -1 SD = 3 for scaled scores).



- iPost using BlogPress from my Kevin McGrew's iPad

intelligence IQ tests IQ testing IQ scores CHC intelligence theory CHC theory Cattell-Horn-Carroll human cognitive abilities psychology school psychology individual differences cognitive psychology neuropsychology neuroscience psychology special education educational psychology psychometrics psychological assessment psychological measurement IQs Corner general intelligence intelligent IQ testing validity composite IQ scores difference scores

Generated by: Tag Generator