Canadian-US IQ "difference/IQ test methdology flaw" reports -- wait a minute.

Recently and article was published in the Journal of Psychoeducational Assessment that raised concerns about the accuracy of the Canadian norms for the Wechsler Adult Intelligence Scale--Fourth Edition (WAIS-IV).  Social media picked up on the story with various attention-getting headlines such as "Common IQ methodology may be flawed" and "Flawed IQ scoring system: Important difference in American, Canadian scoring systems".  The article citation and abstract are below:

• Harrison, A. G., Holmes, A., Silvestri, R., & Armstrong, I. T. (2015). Implications for Educational Classification and Psychological Diagnoses Using the Wechsler Adult Intelligence Scale–Fourth Edition With Canadian Versus American Norms. Journal of Psychoeducational Assessment, 0734282915573723.

Abstract:  Building on a recent work of Harrison, Armstrong, Harrison, Iverson and Lange which suggested that Wechsler Adult Intelligence Scale–Fourth Edition (WAIS-IV) scores might systematically overestimate the severity of intellectual impairments if Canadian norms are used, the present study examined differences between Canadian and American derived WAIS-IV scores from 861 postsecondary students attending school across the province of Ontario, Canada. This broader data set confirmed a trend whereby individuals’ raw scores systematically produced lower standardized scores through the use of Canadian as opposed to American norms. The differences do not appear to be due to cultural, educational, or population differences, as participants acted as their own controls. The ramifications of utilizing the different norms were examined with regard to psychoeducational assessments and educational placement decisions particularly with respect to the diagnoses of Learning Disability and Intellectual Disability.

I have not studied the Harrison et al. study in depth, but would like to share, with his permission, portions of an email  shared with me by Dr. Larry Weiss, Vice President, Global Research & Development, Pearson Clinical and Talent Assessment (the publisher of the WAIS-IV).  Dr. Weiss and I briefly talked about this controversial paper at the recent NASP conference.

 Dr. Larry Weiss comments ( email 3-18-15 - shared with permission)

"To follow up on our discussion about the Harrison el al. paper, they found that a large percent of Canadian college students obtained scores below the average range on the WAIS-IV FSIQ when using the Canadian norms.  They considered this finding to be highly unexpected for a sample of college students, and questioned the validity of the WAIS-IV Canadian norms.  However, the authors of that study did not adequately take into account that 75% of their sample had clinical diagnoses.   

To demonstrate the impact of clinical status on IQ test scores, my research team drew a sample of American subjects matched to the Harrison sample on clinical status and educational level.  We then scored the American sample on U.S. norms, and found that the percentage obtaining below average FSIQ scores was almost identical to that reported by Harrison et. al. using Canadian norms.  This demonstrates that the Harrison et al. findings are not unique to the Canadian norms, but are due to the mixed clinical status of their sample.  Details of our matched sample analysis will appear in an upcoming issue of the Journal of Psychoeducational Assessment in an article by Miller, Weiss, Beal, Saklofske, Zhu, & Holdnack."

Although I will not comment on the specific WAIS-IV study and methodology in question, I can point out that in a special ASB (ASB #12 Use of the Woodcock-Johnson III NU Tests of Cognitive Abilities and Tests of Achievement with Canadian Populations) where I and others reported the performance of a sample of Canadians on the WJ III NU battery, when the Canadian subjects (who were a randomly selected representative sample--not a largely clinically preselected sample), where matched on critical demographic variables to a sample of US subjects, we found that "while some minor score differences are reported across the two samples, the study findings generally support the use of the U.S.-based WJ III NU norms with Canadian school-age populations."  In other words, when comparable (demographically matched) Canadian and US subjects were compared on the WJ III NU cognitive battery, no significant Canadian-US IQ scores, beyond some minor exceptions, were found.

Readers should wait until the Miller et al. (in press) response paper is released before jumping to any quick conclusions.

[Conflict of interest disclosure:  I am a coauthor of the WJ III and WJ IV, a direct competitor to the Wechsler batteries]