Does the WAIS-III measure the same intellectual abilities in MR/ID individuals?

I have had a number of people send me copies of this article (see abstracts and journal info below), especially those who do work related to Dx of MR/ID in Atkins death penalty cases.

The abstract is self-explanatory--the authors conclude that the WAIS-III four-factor structure is not validated in an MR/ID population. I can hear a lawyer now--"so Dr. __________, according to MacLean et al. the WAIS-III doesn't measure the same abilities in individuals with MR/ID...so aren't your results questionable?"

A close read of the article suggests the results should be take with a serious grain of salt. In fact, the discussion is primarily a discussion of the various methodological and statistical reasons why the published 4-factor model may not have fit.

As is often the case when dealing with samples of convenience (the authors own words), especially samples of individuals at the lower end of the ability continuum, the variables often show significant problems with non-normality and skew. This is present in this sample. Given that we are dealing with SEM-based statistics, the problem is actually one of not meeting the assumption of multivariate normality. The variables also showed restricted SD's---restricted range of talent, a condition that dampens correlations in a matrix.

While doing extensive modeling research at the Institute for Community Integration at the University of Minnesota, an institute devoted to individuals with MR/ID/DD, I was constantly faced with data sets with these problems. As a result, I was constantly faced with model fit statistics that were much lower than the standard acceptable rules-of-thumbs for model fit statistics...which reflected the less than statistical and distributional robustness of such sample data. The best way to overcome the resultant low model fits (after trying transformations of the variables to different scales), was to compare the fit of competing models. The best fitting model, when compared to competing models, may still show a relatively poor absolute fit value (when compared to the standard rules of thumb), but by demonstrating that it was the best when compared to alternatives, the case could be made that it was still the best possible model given the constraints of the sample data.

This leads to the MAJOR flaw of this study. Although the authors discuss the sample problems above, they only tested one model...the WAIS-III four-factor model. They then looked at the absolute value of the fit statistics and concluded that the 4-factor model was not a good fit. I see this as a major flaw. Since the standard rules-of-thumb for absolute magnitude of fit stats may no longer hold in samples with statistical and distributional problems, they should have specified competing models (e.g., two-factor; CHC-model, single factor, etc.) and then compared the relative model fit statistics before rendering a conclusion.

Finally, as the authors correctly point out, the current results, even with the flaws above, may simply reflect the well-established finding that the differentiation of cognitive abilities is less for lower functioning individuals, and more for higher functioning. This is Spearman's Law of Diminishing Returns (SLODR) [Click here for an interesting recent discussion of SLODR]

Bottom line for the blogmaster--I judge the authors conclusions to be overstated for the reasons noted above, particularly the failure to compare the 4-factor model to alternative models. It is very possible that the 4-factor model may be the best fitting model given the statistical and distributional constraints of the underlying sample data.


Abstract

Intellectual assessment is central to the process of diagnosing an intellectual disability and the assessment process needs to be valid and reliable. One fundamental aspect of validity is that of measurement invariance, i.e. that the assessment measures the same thing in different populations. There are reasons to believe that measurement invariance of the Wechsler scales may not hold for people with an intellectual disability. Many of the issues which may influence factorial invariance are common to all versions of the scales. The present study, therefore, explored the factorial validity of the WAIS-III as used with people with an intellectual disability. Confirmatory factor analysis was used to assess goodness of fit of the proposed four factor model using 13 and 11 subtests. None of the indices used suggested a good fit for the model, indicating a lack of factorial validity and suggesting a lack of measurement invariance of the assessment with people with an intellectual disability. Several explanations for this and implications for other intellectual assessments were discussed.

- iPost using BlogPress from my Kevin McGrew's iPad

intelligence intelligence testing Atkins cases ICDP blog psychology school psychology neuropsychology Forensic psychology criminal psychology criminal justice death penalty capital punishment ABA IQ tests IQ scores adaptive behavior AAIDD mental retardation intellectual disability SLODR WAIS-III measurement invariance

Generated by: Tag Generator