Cognitive assessment and RTI: Shinn response, correction, plus...

My prior FYI post regarding the Kearns and Fuch's LDA presentation resulted in some spirited exchange over on the NASP listserv. The most detailed response came from Dr. Mark Shinn (see his mug shot to the left). Below are Mark's listserv comments "as is" (with some slight formatting changes by the blogmaster).

Mark was also gracious enough to provide me a full conflict of interest disclosure statement which can be found at the bottom of this post. Finally, I think I have a correction to make. In an email response on the NASP list, I suggested that Fuch's was a student of Dr. Stan Deno, who is widely considered as the father of CBM. Doug was a student at the U of M prior to my arrival, so I am not aware of the complete history. But, I now believe that Doug was not involved with Dr. Deno during the development of CBM. I believe Doug's doctoral mentor was the late and great Bruce Balow. However, Doug has been involved in researching various aspects of CBM as it relates to LD identification. Enough said....I don't have time to run down the lineage of all fellow U of M scholars.

Also, I want to make a statement re: one of the products that Dr. Shinn mentions in his COI statement...namely...AimsWeb. Of all the tools for continuous monitoring I've been most impressed with the AimsWeb product...just my two cents. Finally, I'm done commenting on this thread. Folks who want to track further developments should attend to the NASP listserv.

Mark Shinn responded to a members "exciting" response to my prior blog post re: the Kearns & Fuchs LDA presentation.

• Before going overboard with excitement, I'd encourage a careful read of the presentation.
• This is not about the role/importance of cognitive assessment in LD identification. In particular, this is not a presentation about ATIs.
• On slide 60, regarding the "potential concerns" note: "Many of the studies did not identify cognitive deficits at all" "When they did, they did not always use cognitive assessment"
• This is among a number of other weaknesses.
• Slide 62 states "The Use of Cognitive Assessment Has Potential (Their Emphasis) Benefit--"May" is not the same as "Does" and this review doesn't provide much of a compelling argument as to how or why I could go on and on, but it would not be a good use of time. Note, however, among a number of concerns...
• The authors seem to confuse the p value with impact...the lower the p, the greater the effect (slides 34, 42. Minnesota statisticians would be chastising beyond belief. Effect sizes were reported only on Slide 55. Only 10 of the 36 studies were judged to be of high quality while 14 were judged to be of low quality--not excluded, but still interpreted anyway. Subjects were unspecified, but IF the topic was the role of cognitive assessment in SLD, then one would presume that the studies would have SLD students as subjects. A few clearer are not targeted on SLD. For example, on slide 48, the students are 14 students with low WMRT scores. Slide 41 lists subjects as ADHD. Hmmmm.
• Dependent measures...Visuo-spatial working memory circles, Span Boards, Raven's Head movements (slide 34). Perhaps most importantly, the presentation reports results of cognitive "interventions," not cognitive assessments. Let's see Slide 31 Do cognitive interventions have a positive effect on cognitive outcomes?
• Slide 34 Findings: Students in intervention had greater Slide 35 Performance on cognitive tasks can be improved with a working memory intervention improvement in Slide 36 Cognitive interventions have a positive effect on cognitive outcomes Slide 37 Do studies with hybrid cognitive+academic interventions produce academic gains and on and on and on...
• As a final note, what is the difference between a "cognitive" intervention and an "academic intervention?" Seems like an artificial contrivance.

Dr. Shinn's COI statement: Mark R. Shinn, Ph.D. has commercial relationships with three companies. He serves as a paid consultant to Pearson Assessment as Chief Scientist for AIMSweb, a company created by Gary Germann and Steven Jennen that was sold to the Psychological Corporation in 2006. His responsibilities include contributing to software/product development and improvement and field testing. He does not receive royalties or commissions on AIMSweb sales. He also is a consultant to Glencoe Publishing, a McGraw-Hill Company, for their Jamestown Reading Navigator (JRN) product. JRN is a reading intervention for at risk and very low-performing adolescents. His role has been to assist JRN in the use of CBM maze as part of their reading progress monitoring systems. He is scheduled to receive royalties (one-quarter of 1%) should the product achieve profitability. He also currently serves as an unpaid contributor (without royalties) to VMath, published by Voyager, a math intervention for at risk students Grade 3-12. HIs role has been to assist VMath in the development and use of CBM Math Computation as part of their math progress monitoring system.


Technorati Tags: psychology, educational psychology, school psychology, IQ, IQ testing, intelligence, CBM, NASP, AimsWeb