Research Brief: Where kids live may influence "how" they do math

Casey, B. M., Dearing, E., Vasilyeva, M., Ganley, C. M., & Tine, M. (2011). Spatial and Numerical Predictors of Measurement Performance: The Moderating Effects of Community Income and Gender. Journal of Educational Psychology, 103(2), 296-311.

Spatial reasoning and numerical predictors of measurement performance were investigated in 4th graders from low-income and affluent communities. Predictors of 2 subtypes of measurement performance (spatial–conceptual and formula based) were assessed while controlling for verbal and spatial working memory. Consistent with prior findings, students from the affluent community outperformed students from the low-income community on all measures examined. More importantly, the study revealed different patterns of relations between cognitive skills and academic performance in the 2 communities. Specifically, spatial skills were related to measurement performance in the affluent but not in the low-income community. These findings demonstrate that socioeconomic context impacts not only children's levels of performance but also their capacity to apply basic cognitive skills, like spatial reasoning, to their academic performance.


- iPost using BlogPress from my Kevin McGrew's iPad

General intelligence: To g or not to g? Dr. Joel Schneider comments

Last week there was a spirited exchange on CHC listserv regarding the status of the theoretical construct of general intelligence (g). Dr. Joel Schneider provided a very thought provoking response that included some of his recent writings on the subject. I asked Joel if I could share on IQs Corner, and he agreed. Below are his comments "as is." As the reader will learn from some of his comments, he was responding to other individuals who made some statements about g on the list.


Yes, opinion polling is not the way to do science but ultimately science IS about consensus-building. A single researcher can produce evidence so compelling that the entire field is forced to change its mind. When it comes to g, however, there is no compelling evidence about what it is or is not. Here are three excepts from a chapter I wrote that is in preparation:

"Spearman’s (1904) little g caused a big stir when it was first proposed and has, for over a century now, been disrupting the natural state of harmony that would otherwise prevail amongst academics. Many a collegial tie has been severed, many a friendship has soured, perhaps even engagements broken off and marriages turned into dismal, loveless unions because of the rancor this topic provokes. I have seen otherwise mild-mannered professors in tweed jackets come to blows in bars over disagreements about g — okay…not really…but I have seen some very sarcastic emails exchanged on professional listservs!"

"It turns out that these two groups [the mono-g-ists and the poly-G-ists] are not merely on opposite sides of an intellectual debate — they are members of different tribes. They speak different dialects, vote for different candidates, and pray to different gods. Their heroic tales emphasize different virtues and their foundation myths offer radically different but still internally consistent explanations of how the world works. If you think that the matter will be settled by accumulating more data, you have not been paying attention for the last hundred years."

"The theoretical status of g will not cease to be controversial until something extraordinary happens to the field. I do not pretend to know what this might be. Maybe a breakthrough from biology will resolve the matter. Maybe divine intervention. Until then, I feel no need to join either tribe. I will remain agnostic and I will not get too excited the next time really smart people eagerly announce that finally, once and for all, they have proof that the other side is wrong. This has happened too many times before."

Shifting topics:

You are right, I have estimated a person's intelligence and said something about it out loud. In like manner, I have said about different people, "She's nice." "He's a jerk!" "He's funny!" "She's impressive." "He's a good person." I agree with Spearman that "intelligence" is a pre-scientific folk concept, just as nice, jerk, funny, and good are folk concepts. There is nothing wrong with these terms. They communicate pretty clearly what I want to say. However, I do not believe that there is an underlying personality variable called "goodness" or "impressiveness." Such terms probably do have an indirect relationship to more fundamental cognitive structures, though.

Here is an excerpt from an early draft of the forthcoming chapter I wrote with Kevin McGrew. Almost of all of this section was removed because the chapter was starting to look like it was going to be over 200 pages. Editing the chapter down to 100 pages was painful and many parts we liked were removed:

Is g an ability?

The controversy about the theoretical status of g may have less fire and venom if some misunderstandings are cleared up. First, Spearman did not believe that performance on tests was affected by g and only g. In a review of a book by his rival Godfrey Thomson, Spearman (1940, p. 306) clarified his position.

“For I myself, no less than Thomson, accept the hypothesis that the observed test-scores, and therefore their correlations, derive originally from a great number of small causes; as genes, neurones, etc. Indeed this much seems to be accepted universally. We only disagree as to the way in which this derivation is to be explained.”

Second, Spearman (1927, p. 92) always maintained, even in his first paper about g (Spearman, 1904, p. 284), that g might consist of more than one general factor. Cattell (1943) noted that this was an anticipation of Gf-Gc Theory. Third, Spearman did not consider g to be an ability, or even a thing. Yes, you read that sentence correctly. Surprisingly, neither does Arthur Jensen, perhaps the most (in)famous living proponent of Spearman’s theory. Wait! The paper describing the discovery of g was called “‘General Intelligence’: Objectively Determined and Measured.” Surely this means that Spearman believed that g was general intelligence. Yes, but not really. Spearman thought it unproductive to equate g with intelligence, the latter being a complex amalgamation of many abilities (Jensen, 2000). Spearman believed that “intelligence” is a folk concept and thus no one can say anything scientific about it because everyone can define it whichever way they wish. Contemplating the contradictory definitions of intelligence moved Spearman (1927, p. 14) to erupt,

“Chaos itself can go no farther! The disagreement between different testers—indeed, even between the doctrine and the practice of the selfsame tester—has reached its apogee. […] In truth, ‘intelligence’ has become a mere vocal sound, a word with so many meanings that finally it has none.”

Spearman had a much more subtle conceptualization of g than many critics give him credit for. In discussing the difficulty of equating g with intelligence, or variations of that word with more precise meanings such as abstraction or adaptation, Spearman (1927, p.88) explained,

“Even the best of these renderings of intelligence, however, always presents one serious general difficulty. This is that such terms as adaptation, abstraction, and so forth denote entire mental operations; whereas our g, as we have seen, measures only a factor in any operation, not the whole of it.”

At a conference in which the proceedings were published in an edited volume (Bock, Goode, & Webb, 2000), Maynard Smith argued that there isn't a thing called athletic ability but rather it is a performance category. That is, athletic ability would have various components such as heart volume, muscle size, etc. Smith went on to argue that g, like athletic ability, is simply a correlate that is statistically good at predicting performance. Jensen, in reply, said, "No one who has worked in this field has ever thought of g as an entity or thing. Spearman, who discovered g, actually said the very same thing that you're saying now, and Cyril Burt and Hans Eysenck said that also: just about everyone who has worked in this field has not been confused on that point." (Bock, Goode, & Webb, 2000, p. 29). In a later discussion at the same conference, Jensen clarified his point by saying that g is not a thing but is instead the total action of many things. He then listed a number of candidates that might explain why disparate regions and functions of the brain tend to function at a similar level within the same person such as the amount of myelination of axons, the efficiency of neural signaling, and the total number of neurons in the brain (Bock, Goode, & Webb, 2000, p. 52). Note that none of these hypotheses suggest that g is an ability. Rather, g is what makes abilities similar to each other within a particular person’s brain.
In Jensen’s remarks, all of the influences on g were parameters of brain functioning. We can extend Jensen’s reasoning to environmental influences with a thought experiment. Suspend disbelief for a moment and suppose that there is only one general influence on brain functioning: lead exposure. Because of individual differences in degree of lead exposure, all brain functions are positively correlated and thus a factor analysis would find a psychometric g-factor. Undoubtedly, it would be a smaller g-factor than is actually observed but it would exist.

In this thought experiment, g is not an ability. It is not lead exposure itself, but the effect of lead exposure. There is no g to be found in any person’s brain. Instead, g is a property of the group of people tested. Analogously, a statistical mean is not a property of individuals but a group property (Bartholomew, 2004). This hypothetical g emerges because lead exposure influences all of the brain at the same time and because some people are exposed to more lead than are others.

In the thought experiment above, the assumptions were unrealistically simple and restrictive. It is certain that individual differences in brain functioning is influenced in part by genetic differences among individuals and that some genetic differences affect almost all cognitive abilities (Exhibit A: Down Syndrome). Some genetic differences affect some abilities more than others (e.g., William’s Syndrome, caused by a deletion of about 26 genes on chromosome 7, is associated with impaired spatial processing but relatively intact verbal ability). Thus, there are general genetic influences on brain functioning and there are genetic differences that effect only a subset of brain functions.

The fact that there are some genetic differences with general effects on cognitive ability (and there are probably many, Plomin, 20??) is enough to produce at least a small g-factor, and possibly a large one. However, there are many environmental effects that effect most aspects of cognitive functioning. Lead exposure is just one of many toxins that likely operate this way (e.g., mercury & arsenic). There are viruses and other pathogens that infect the brain more or less indiscriminately and thus have an effect on all cognitive abilities. Many head injuries are relatively focal (e.g., microstrokes and bullet wounds) but others are more global (e.g., large strokes and blunt force trauma) and thus increase the size of psychometric g. Poor nutrition probably hampers the functioning of individual neurons indiscriminately but the systems that govern the most vital brain functions have more robust mechanisms and greater redundancy so that temporary periods of extreme malnourishment affect some brain functions more than others. Even when you are a little hungry, the first abilities to suffer are highly g-loaded and evolutionarily new abilities such as working memory and controlled attention.

Societal forces probably also increase the size of psychometric g. Economic inequality ensures that some people will have more of everything that enhances cognitive abilities and more protection from everything that diminishes them. This means that influences on cognitive abilities that are not intrinsically connected (e.g., living in highly polluted environments, being exposed to water-borne parasites, poor medical care, poor schools, cultural practices that fail to encourage excellence in cognitively demanding domains, reduced access to knowledgeable mentors among many many others) are correlated. Correlated influences on abilities cause otherwise independent cognitive abilities to be correlated, increasing the size of psychometric g. How much any of these factors increase the size of psychometric g (if at all) is not yet known. The point is that just because abilities are influenced by a common cause, does not mean that the common cause is an ability.

There are two false dichotomies we should be careful to avoid. The first is the distinction between nature and nurture. There are many reasons that genetic and environmental effects on cognitive abilities might be correlated, including the possibility that genes affect the environment and the possibility that the environment alters the effect of genes. The second false choice is the notion that either psychometric g is an ability or it is not. Note that if we allow that some of psychometric g is determined by things that are not abilities, it does not mean that there are no truly general abilities (e.g., working memory, processing speed, fluid intelligence, and so forth). Both types of general influences on abilities can be present.

In this section, we have argued that not even the inventor of g considered it to be an ability. Why do so many scholars write as if Spearman believed otherwise? In truth, he (and Jensen as well) often wrote in a sort of mental shorthand as if g were an ability or a thing that a person could have more of or less of. Cattell (1943, p. 19) gives this elegantly persuasive justification:

Obviously "g" is no more resident in the individual than the horsepower of a car is resident in the engine. It is a concept derived from the relations between the individual and his environment. But what trait that we normally project into and assign to the individual is not? The important further condition is that the factor is not determinable by the individual and his environment but only in relation to a group and its environment. A test factor loading or an individual's factor endowment has meaning only in relation to a population and an environment. But it is difficult to see why there should be any objection to the concept of intelligence being given so abstract a habitation when economists, for example, are quite prepared to assign to such a simple, concrete notion as "price" an equally relational existence.


intelligence IQ tests IQ testing IQ scores CHC intelligence theory CHC theory Cattell-Horn-Carroll human cognitive abilities psychology school psychology individual differences cognitive psychology neuropsychology neuroscience psychology special education educational psychology psychometrics psychological assessment psychological measurement IQs Corner general intelligence g

Generated by: Tag Generator

Research Bytes: Working memory training in elderly and impact of noise on performance

Click on images to enlarge








- iPost using BlogPress from my Kevin McGrew's iPad

FYiPOST: Neuropsychology - Volume 25, Issue 4

A new issue is available for the following APA journal:

Neuropsychology Volume 25, Issue 4, (Jul)

Anomia as a marker of distinct semantic memory impairments in Alzheimer's disease and semantic dementia.

Page 413-426

Reilly, Jamie; Peelle, Jonathan E.; Antonucci, Sharon M.; Grossman, Murray

Evidence for higher reaction time variability for children with ADHD on a range of cognitive tasks including reward and event rate manipulations.

Page 427-441

Epstein, Jeffery N.; Langberg, Joshua M.; Rosen, Paul J.; Graham, Amanda; Narad, Megan E.; Antonini, Tanya N.; Brinkman, William B.; Froehlich, Tanya; Simon, John O.; Altaye, Mekibib

Language production strategies and disfluencies in multi-clause network descriptions: A study of adult attention-deficit/hyperactivity disorder.

Page 442-453

Engelhardt, Paul E.; Ferreira, Fernanda; Nigg, Joel T.

The prospective course of postconcussion syndrome: The role of mild traumatic brain injury.

Page 454-465

Meares, Susanne; Shores, E. Arthur; Taylor, Alan J.; Batchelor, Jennifer; Bryant, Richard A.; Baguley, Ian J.; Chapman, Jennifer; Gurka, Joseph; Marosszeky, Jeno E.

Executive functions and social competence in young children 6 months following traumatic brain injury.

Page 466-476

Ganesalingam, Kalaichelvi; Yeates, Keith Owen; Taylor, H. Gerry; Walz, Nicolay Chertkoff; Stancin, Terry; Wade, Shari

Executive functions, information sampling, and decision making in narcolepsy with cataplexy.

Page 477-487

Delazer, Margarete; Högl, Birgit; Zamarian, Laura; Wenter, Johanna; Gschliesser, Viola; Ehrmann, Laura; Brandauer, Elisabeth; Cevikkol, Zehra; Frauscher, Birgit

Genetic architecture of learning and delayed recall: A twin study of episodic memory.

Page 488-498

Panizzon, Matthew S.; Lyons, Michael J.; Jacobson, Kristen C.; Franz, Carol E.; Grant, Michael D.; Eisen, Seth A.; Xian, Hong; Kremen, William S.

Sex differences in neuropsychological performance and social functioning in schizophrenia and bipolar disorder.

Page 499-510

Vaskinn, Anja; Sundet, Kjetil; Simonsen, Carmen; Hellvin, Tone; Melle, Ingrid; Andreassen, Ole A.

A neuropsychological investigation of multitasking in HIV infection: Implications for everyday functioning.

Page 511-519

Scott, J. Cobb; Woods, Steven Paul; Vigil, Ofilio; Heaton, Robert K.; Schweinsburg, Brian C.; Ellis, Ronald J.; Grant, Igor; Marcotte, Thomas D.

Functional disruption of the brain mechanism for reading: Effects of comorbidity and task difficulty among children with developmental learning problems.

Page 520-534

Simos, Panagiotis G.; Rezaie, Roozbeh; Fletcher, Jack M.; Juranek, Jenifer; Passaro, Antony D.; Li, Zhimin; Cirino, Paul T.; Papanicolaou, Andrew C.

The prosthetics of vigilant attention: Random cuing cuts processing demands.

Page 535-543

O'Connor, Charlene; Robertson, Ian H.; Levine, Brian

---

Research Bytes: Brain complexity, predicting job success, neuroscience/creativity, fluid IQ and personality

Bassett, D. S., & Gazzaniga, M. S. (2011). Understanding complexity in the human brain. Trends in Cognitive Sciences, 15(5), 200-209.

Although the ultimate aim of neuroscientific enquiry is to gain an understanding of the brain and how its workings relate to the mind, the majority of current efforts are largely focused on small questions using increasingly detailed data. However, it might be possible to successfully address the larger question of mind–brain mechanisms if the cumulative findings from these neuroscientific studies are coupled with complementary approaches from physics and philosophy. The brain, we argue, can be understood as a complex system or network, in which mental states emerge from the interaction between multiple physical and functional levels. Achieving further conceptual progress will crucially depend on broad-scale discussions regarding the properties of cognition and the tools that are currently available or must be developed in order to study mind–brain mechanisms.
Article Outline



Ziegler, M., Dietl, E., Danay, E., Vogel, M., & Buhner, M. (2011). Predicting Training Success with General Mental Ability, Specific Ability Tests, and (Un)Structured Interviews: A meta-analysis with unique samples. International Journal of Selection and Assessment, 19(2), 170-182.


Several meta-analyses combine an extensive amount of research concerned with predicting training success. General mental ability is regarded as the best predictor with specific abilities or tests explaining little additional variance. However, only few studies measured all predictors within one sample. Thus, intercorrelations were often estimated based on other studies. Moreover, new methods for correcting range restriction are now available. The present meta-analyses used samples derived from a German company in which applicants for different apprenticeships were tested with an intelligence structure test, specific ability tests as well as a structured and an unstructured interview. Therefore, intercorrelations between different assessment tools did not have to be estimated from other data. Results in the final examination, taking place at least 2 years after the original assessment, served as criterion variable. The dominant role of general mental ability was confirmed. However, specific abilities were identified that can be used as valuable additions. Job complexity moderated some of the relationships. Structured interviews were found to have good incremental validity over and above general mental ability. Unstructured interviews, on the other hand, performed poorly. Practical implications are discussed.


Sawyer, K. (2011). The Cognitive Neuroscience of Creativity: A Critical Review. Creativity Research Journal, 23(2), 137-154.

Cognitive neuroscience studies of creativity have appeared with increasing frequently in recent years. Yet to date, no comprehensive and critical review of these studies has yet been published. The first part of this article presents a quick overview of the 3 primary methodologies used by cognitive neuroscientists: electroencephalography (EEG), positron emission tomography (PET), and functional magnetic resonance imaging (fMRI). The second part provides a comprehensive review of cognitive neuroscience studies of creativity-related cognitive processes. The third part critically examines these studies; the goal is to be extremely clear about exactly what interpretations can appropriately be made of these studies. The conclusion provides recommendations for future research collaborations between creativity researchers and cognitive neuroscientists.


Djapo, N., KolenovicDjapo, J., Djokic, R., & Fako, I. (2011). Relationship between Cattell's 16PF and fluid and crystallized intelligence. Personality and Individual Differences, 51(1), 63-67.

The aim of the study was to explore the relationship between the five global factors and 16 dimensions of Cattell’s personality model and fluid and crystallized intelligence. A total of 105 third graders (45.7% males) of three high schools participated in the research. Fluid intelligence was measured by Raven’s Advanced Progressive Matrices and crystallized intelligence was measured by the Mill Hill Vocabulary Scale. Personality traits were measured by the Sixteen Personality Factor Questionnaire. Anxiety is correlated neither with fluid nor with crystallized intelligence. Extraversion and Self-Control are negatively correlated with fluid intelligence whereas Tough-Mindedness is positively correlated with it. Independence is positively correlated with crystallized intelligence and Tough-Mindedness is negatively correlated with it. Regression analysis reveals that all broad personality factors, except anxiety, are significant predictors of fluid intelligence. When combined together, these factors account for 25% of the variance of fluid intelligence scores. The regression model with crystallized intelligence as a criterion variable is not statistically significant. The study results are consistent with the Chamorro


- iPost using BlogPress from my Kevin McGrew's iPad

Special issue of Assessment journal on the WAIS-IV and WMS-IV research

The journal Assessment just published a special issue on the WASI-IV/WMS-IV. I love the journal cover (see above)


Frazier, T. W. (2011). Introduction to the Special Section on Advancing WAIS-IV and WMS-IV Clinical Interpretation. Assessment, 18(2), 131-132.

Bowden, S. C., Saklofske, D. H., & Weiss, L. G. (2011). Augmenting the Core Battery With Supplementary Subtests: Wechsler Adult Intelligence Scale-IV Measurement Invariance Across the United States and Canada. Assessment, 18(2), 133-140.

Brooks, B. L., Holdnack, J. A., & Iverson, G. L. (2011). Advanced Clinical Interpretation of the WAIS-IV and WMS-IV: Prevalence of Low Scores Varies by Level of Intelligence and Years of Education. Assessment, 18(2), 156-167.

Drozdick, L. W., & Cullum, C. M. (2011). Expanding the Ecological Validity of WAIS-IV and WMS-IV With the Texas Functional Living Scale. Assessment, 18(2), 141-155.


Gregoire, J., Coalson, D. L., & Zhu, J. J. (2011). Analysis of WAIS-IV Index Score Scatter Using Significant Deviation from the Mean Index Score. Assessment, 18(2), 168-177.

Holdnack, J., Goldstein, G., & Drozdick, L. (2011). Social Perception and WAIS-IV Performance in Adolescents and Adults Diagnosed With Asperger's Syndrome and Autism. Assessment, 18(2), 192-200.

Holdnack, J. A., Zhou, X. B., Larrabee, G. J., Millis, S. R., & Salthouse, T. A. (2011). Confirmatory Factor Analysis of the WAIS-IV/WMS-IV. Assessment, 18(2), 178-191


- iPost using BlogPress from my Kevin McGrew's iPad

intelligence IQ tests IQ testing IQ scores CHC intelligence theory CHC theory Cattell-Horn-Carroll human cognitive abilities psychology school psychology individual differences cognitive psychology neuropsychology neuroscience psychology special education educational psychology psychometrics psychological assessment psychological measurement IQs Corner general intelligence WAIS-IV WMS-IV Wechsler Adult Intelligence Scale Wechsler Batteries

Generated by: Tag Generator

More evidence on importance of brain rhythm@TheNeuroScience, 6/27/11 2:40 PM

Interesting new study (with mice) that highlights the growing awareness of the importance of brain rhythm synchronization...especially the importance of synchronized rhythmic communication between brain networks important for controlled attention and executive functions. This study shows an interesting possible link between increased movement (running) and brain rhythm.  The IQ Brain Clock blog has made countless posts re: brain rhythm research and human cognitive and motor function.  Here is a link to most of them - http://tinyurl.com/3kas7aj

"Run Forrest run" -- from one of my favorite movies (Forrest Gump)

Neuro Science (@TheNeuroScience) 6/27/11 2:40 PM Brain rhythm associated with learning also linked to running speed, UCLA study ... http://sns.mx/queRy3

Sent from Kevin McGrew's iPad

Kevin McGrew, PhD

Educational Psychologist

FYiPOST: Psychometrika, Vol. 76, Issue 3 - New Issue Alert

Sunday, June 26 Dear Valued Customer, We are pleased to deliver your requested table of contents alert for Psychometrika. Good news: now you will find quick links to the full text of the article. Access the article with only one click! Volume 76 Number 3 is now available on SpringerLink Register for Springer's email services providing you with info on the latest books in your field. ... More! Important News! Follow @Springernomics on Twitter In this issue: Item Selection in Multidimensional Computerized Adaptive Testing—Gaining Information from Different Angles Chun Wang & Hua-Hua Chang Abstract    Full text PDF On the Bayesian Nonparametric Generalization of IRT-Type Models Ernesto San Martín, Alejandro Jara, Jean-Marie Rolin & Michel Mouchart Abstract    Full text PDF Investigating the Performance of Alternate Regression Weights by Studying All Possible Criteria in Regression Models with a Fixed Set of Predictors Niels Waller & Jeff Jones Abstract    Full text PDF Statistical Significance of the Contribution of Variables to the PCA solution: An Alternative Permutation Strategy Mariëlle Linting, Bart Jan van Os & Jacqueline J. Meulman Abstract    Full text PDF Factor Analysis via Components Analysis Peter M. Bentler & Jan de Leeuw Abstract    Full text PDF Cohen's Linearly Weighted Kappa is a Weighted Average of 2×2 Kappas Matthijs J. Warrens Abstract    Full text PDF A Joint Modeling Approach for Reaction Time and Accuracy in Psycholinguistic Experiments T. Loeys, Y. Rosseel & K. Baten Abstract    Full text PDF Book Review M.D. RECKASE (2009) Multidimensional Item Response Theory (Statistics for Social and Behavioral Sciences). Hua-Hua Chang & Chun Wang Abstract    Full text PDF Book Review Claeskens, G. & Hjort, N. L. (2009). Model Selection and Model Averaging. Alex Karagrigoriou Abstract    Full text PDF Erratum Erratum to: The Generalized DINA Model Framework Jimmy de la Torre Abstract    Full text PDF Do you want to publish your article in this journal? Please visit the homepage of Psychometrika for full details on:    - aims and scope    - editorial policy    - article submission Request a free sample copy If you are not a current subscriber to the journal, click here to read a free sample copy online. Subscribers to a Springer publication are entitled to read the full-text articles online in SpringerLink. For registration information please contact your librarian or send us an e-mail: In the Americas: springerlink-ny@springer.com In all other countries: springerlink@springer.com Alert information

IAP 101 Psychometric Brief # 9: The problem with the 1/1.5 SD SS (15/22) subtest comparison "rule-of-thumb"

In regard to my prior "temp" post, I wrote so much in my NASP listserv response that I have decided to take my email response, correct a few typo's, and post it now as blog post. I may return to this later to write a lengthier IAP 101 Research Brief or report.

Psychologists who engage in intelligence testing frequently compare subtest scores to determine if they are statistically and practically different...as part of the clinical interpretation process. Most IQ test publishers provide sound statistical procedures (tables or software for evaluating the statistical difference of two test scores; confidence band comparison rules-of-thumb).

However, traditional and clinical lore has produced a common "rule-of-thumb" that is problematic. The typical scenario is when a clinician subtracts two test SS's (M=100; SD=15) and invokes the rule-of-thumb that the difference needs to be 15 SS points (1 SD) or 22/23 points (1.5 SD). This is not correct.

SS difference scores do NOT have an SD scale of 15! When you subtract two SS's (with mean=100; SD=15) the resultant score distribution has a mean of zero and an SD that is NOT 15 (unless you transform/rescale the distribution to this scale) The size of the difference SD is a function of the correlation between the two measures compared.

The SD(diff) is the statistic that should be used, and there are a number of different forumla for computing this metric. The different SD(diff)'s differ based on the underlying question or assumptions that is the basis for making the comparison.

One way to evaluate score differences is the SEM band overlap approach. This is simple and is based on underlying statistical calculations (averaged across different scenarios to allow for a simple rule of thumb) that incorporates information about the reliability of the difference score. Test publishers also provide tables to evaluate the statistical significance of differences of a certain magnitude for subtests, such as in the various Wechsler manuals and software. These are all psychometrically sound and defensible procedures.......let me say that again...these are all psychometrically sound and defensible procedures. I repeat this phrase as the point I make below was recently misinterpreted at a state SP workshop as me saying there was something wrong with tables in the WISC-IV...which is NOT what I said and is NOT what I am saying here).

However, it is my opinion that in these situations we must do better and there is a more appropriate and better metric for evaluating differences between two different test scores, ESPECIALLY when the underlying assumption is that the two measures should be similar because they form a composite or cluster. This implies "correlation"...and not simple comparison of any two tests.

When one is attempting to evaluate the "unity" of a cluster or composite, an SD(diff) metric should be used that is consistent with the underlying assumption of the question. Namely, one is expecting the scores to be similar because they form a factor. This implies "correlation" between the measures. There is an SD(diff) calculation that incorporates the correlation between the measures being compared. When one uses this approach, the proper SD(diff) can vary from as small as approximately 10 points (for "tight" or highly correlated Gc tests) to as high as approximately 27 pts (for "loose" or weekly correlated tests in a cluster).

The information for this SD(diff) metric comes from a classic 1957 article by Payne and Jones (click here) (thanks to Joel S. for brining it to my attention recently). Also, below are two tables that show the different, and IMHO, more appropriate SD(diff) values that should be used when making some example test comparisons on the WISC-IV and WJ-III. (Click on images to enlarge)

As you see in the tables, the 15 (3 if using scaled scores) and 22 (4.5 if scaled scores) rules-of-thumb will only be correct when the correlation between the two tests being compared is of a moderate magnitude. When the correlation between tests being compared is high (when you have a "tight" ability domain) the appropriate SDdiff metric to evaluate differences can be as low as 9.9 points (for 1 SDdiff) and 14.8 (for 1.5 SDdiff) for the Verbal Comp/Gen Info test from the WJ-III Gc cluster or 2.2 scaled score (1 SDdiff) and 3.3 (1.5 SDdiff) when comparing WISC-IV Sim/Vocab.

In contrast, when the ability domain is very wide or "loose", one would expect more variability since traits/tests are not as correlated. In reviewing the above tables one concludes that the very low test correlations for the tests that comprise the WJ-III Gv and Glr clusters produce a 1 SDdiff that is nearly TWICE the 15 point rule of thumb (27-28 points).

I have argued this point with a number of quants (and some have agreed with me) but believe that the proper SS(diff) to be used is not "one size fits all situations." The confidence band and traditional tables of subtest significant difference approaches are psychometrically sound and work when comparing any two tests. However, when the question becomes one of comparing tests where the fundamental issue revolves around the assumption that the tests scores should be similar because they share a common ability (are correlated), then IMHO, we can do better...there is a better way for these situations. We can improve our practice. We can move forward.

This point is analogous to doing simple t-tests of group means. When one has two independent samples the t-test formula includes a standard error term (in the denominator) that does NOT include any correlation/covariance parameter. However, when one is calculating a dependent sample t-test (which means there is a correlation between the scores), the error term incorporates information about the correlation. It is the same concept.....just applied to group vs individual score comparisons.

I urge people to read the 1957 article, review the tables I have provided above, and chew on the issue. There is a better way. The 15/22 SS rule of thumb is only accurate when a certain moderate level of correlation exists between the two tests being compared and when the comparison implies a common factor or ability. If one uses this simplistic rule of thumb practitioners are likely using a much too stringent rule in the case of highly correlated tests (e.g., Gc) and being overly liberal when evaluating tests from a cluster/composite that are low in correlation (what I call ability domain cohesion--click here for prior post that explains/illustrates this concept). The 15/22 SS rule of thumb is resulting in inaccurate decisions regarding the unusualness of test differences when we fail to incorporate information about the correlation between the compared measures. And, even when such differences are found via this method (or the simple score difference method), this does not necessarily indicate that something is "wrong" and the cluster can't be computed or interpreted. This point was recently made clear in an instructional video by Dr. Joel Schneider on sources of variance in test scores that form composites.

If using the recommended SDdiff metric recommended here is to much work, I would recommend that practitioners steer clear of the 15/22 (1/1.5 SD) rule-of-thumb and instead use the tables provided by the test publishers or use the simple SEM confidence band overlap rule-of-thumb. Sometimes simpler may be better.


- iPost using BlogPress from my Kevin McGrew's iPad

Temp post for NASP thread: Evaluating significance of subtest difference scores

There is a hot thread on the NASP listserv this morning where I am making a reply that references some tables/slides. I am posting them here so people can view them. I also am making reference to a classic 1957 article by Payne and Jones for which I am providing a link here.

I have been trying to find time to work this material into a formal IAP 101 Brief or Report....but simply can't find the time. Maybe this will get that going.

Sorry if this post doesn't make sense "as is".....I hope to cycle back later to write a more complete explanation.

Double click on images to enlarge.









- iPost using BlogPress from my Kevin McGrew's iPad

Research Byte: Age-related cognitive decline due more to processing speed and less to executive control?

Note that the final model (above) is consistent with prior research I included in McGrew (2005) and which is also online here. There is clearly strong evidence for the primary role of processing speed (Gs) playing a indirect role on cognition mediated via working memory.





- iPost using BlogPress from my Kevin McGrew's iPad

intelligence IQ tests IQ testing IQ scores CHC intelligence theory CHC theory Cattell-Horn-Carroll human cognitive abilities psychology school psychology individual differences cognitive psychology neuropsychology neuroscience psychology special education educational psychology psychometrics psychological assessment psychological measurement IQs Corner general intelligence working memory executive function cognitive control processing speed working memory Gsm Gs aging cognitive control

Generated by: Tag Generator

How good are aptitude tests@PsychNews, 6/17/11 5:01 PM

Psychology News (@PsychNews) 6/17/11 5:01 PM Can Aptitude Tests Really Predict Your Performance? http://bit.ly/mMp8Vk

Sent from Kevin McGrew's iPad

Kevin McGrew, PhD

Educational Psychologist

APH position paper on IQ testing with blind or visually impaired

A new position paper (from the Accessible Tests Department of the American Printing House for the Blind) on IQ testing with individuals who are blind or visually impaired is now available here.


- iPost using BlogPress from my Kevin McGrew's iPad

CHC Intelligence Theory v2.0: Grand Model and Broad and Narrow Ability Definitions Revised

Dr. Joel Schneider and I have crafted an abridged summary of our "tweaking" of the CHC taxonomy of broad and narrow ability definitions (CHC v2.0) published in the 3rd edition of Flanagan and Harrison's Contemporary Intellectual Assessment (CIA; 2012) book. The book chapter is extensive and does not included a table of revised definitions. Nor does it include a grand figure.Thus, we have developed such a summary and make it available here.  Also, the slides are available for viewing via  SlideShare.

Please be careful in the use of the definitions. In our chapter we expand on the definitions and include a section on "unresolved issues"....as the taxonomy is fluid and evolving and should not be seen as cast in stone. Purchasing the book and reading the complete chapter, as well as a ton of other excellent chapters in CIA-3, is strongly recommended.


- iPost using BlogPress from my Kevin McGrew's iPad

intelligence IQ tests IQ testing IQ scores CHC intelligence theory CHC theory Cattell-Horn-Carroll human cognitive abilities psychology school psychology individual differences cognitive psychology neuropsychology neuroscience psychology special education educational psychology psychometrics psychological assessment psychological measurement IQs Corner general intelligence CHC definitions CHC v2.0 Contemporary Intellectual Assesment

Generated by: Tag Generator

FYiPOST: Brain imaging study of preschoolers with ADHD detects brain differences linked to symptoms

http://www.sciencedaily.com/releases/2011/06/110609112915.htm

Sent with MobileRSS HD

Sent from Kevin McGrew's iPad

Kevin McGrew, PhD

Educational Psychologist

IAP home office-revised

Spent the morning redoing the IAP home office to be more efficient and to provide more work space. All ready to "get r' done". Yep...one can never have enough computers or monitors...only one missing from the picture is my iPhone...which was used to take the picture. Picture does not include at least 3 semi-dead laptops in closet :)

Research byte: Working memory model of ADHD

Click on images to enlarge











- iPost using BlogPress from my Kevin McGrew's iPad

On the road-blogging lite or zero till end of week

Research brief: Gender differences in intelligence on the WAIS-III (Irwing, in press)

There has been no shortage of contemporary research on gender differences in cognitive abilities (click here for prior IQs Corner posts), and g (general intelligence) in particular. Irwing has a new article "in press" that contributes to this literature, both by reinforcing some prior findings...but also being at variance with other. The introduction provides a nice brief overview of some of the reasons (primarily methodological) for difference on the male-female g-difference research.

Double click on images to enlarge.








- iPost using BlogPress from my Kevin McGrew's iPad

intelligence IQ tests IQ testing IQ scores CHC intelligence theory CHC theory Cattell-Horn-Carroll human cognitive abilities psychology school psychology individual differences cognitive psychology neuropsychology neuroscience psychology special education educational psychology psychometrics psychological assessment psychological measurement IQs Corner general intelligence WAIS-III gender differences

Generated by: Tag Generator