Math screening and prgressing monitoring - Guest post by John Garruto

The following is a guest post by John Garruto, school psychologist with the Oswego School District and member of the IQs Corner Virtual Community of Scholars. John reviewed the following article and has provided his comments below. [Blog dictator note - John's review is presented "as is" with only a few minor copy edits by the blog dicator

Fuchs, L.S., Fuchs, D., Compton, D.L., Bryant, J.D., Hamlett, C.L., & Seethaler, P.M. (2007). Mathematics Screening and Progress Monitoring at First Grade: Implications for Responsiveness to Intervention. Exceptional Children, 73(3), 311-330.

Abstract

• The predictive utility of screening measures for forecasting math disability (MD) at the end of 2nd grade and the predictive and discriminant validity of math progress-monitoring tools were assessed. Participants were 225 students who entered the study in 1st grade and completed data collection at the end of 2nd grade. Screening measures were Number Identification/Counting,Fact Retrieval, Curriculum-Based Measurement (CBM) Computation, and CBM Concepts/Applications. or Number Identification/Counting and CBM Computation, 27 weekly assessments were lso collected. MD was defined as below the 1 Oth percentile at the end of 2nd grade on calculations nd word problems. Logistic regression showed that the 4-variable screening model produced ood and similar fits in accounting for MD-calculation and MD-word problems. Classification ccuracy was driven primarily by CBM Concepts/Applications and CBM Computation; CBM oncepts/Applications was the better of these predictors. CBM Computation, but not Number dentification/Counting, demonstrated validity for progress monitoring.

John Garruto speaks (in his own words)

• This article begins with a summary of research that has already been done to date regarding math disabilities. One of the most important conclusions reached through analysis of past studies is that screening outcomes will likely differentiate math computation from math reasoning skills--looking at math as a universal entity would likely be erroneous.
• Examples of pre-existing research included reviewing the influence of various screeners in predicting outcome measures. Screeners across studies included (but were not restricted to): Number knowledge, digit span backwards, missing number, etc. Outcome measures included various group administered normed reference tests (such as the Stanford Achievement Test) or individually administered normed reference tests (such as the WJ-R). Most of the screening predictors had some degree of predictability on outcome measures (usually ranging from about .42-.72). Nor surprisingly, number knowledge was the highest predictor while missing number and digit span backwards also seemed to account for some degree of variance across studies. These findings correlate with the CHC (Cattell-Horn-Carroll) framework advocating that crystallized intelligence, fluid reasoning, and short-term memory are all important predictors of math performance (See Flanagan, Ortiz, Alfonso, & Mascolo, 2006).
• Fuchs et al. used the WJ-III to obtain a profile of their subject sample at the beginning of first grade. They used four CBM techniques in order to predict performance on the dependent measures at the end of second grade (Number ID/Counting, Fact Retrieval, CBM-Computation, and CBM-Applied Problems.) Outcome measures included the WRAT for calculation and Jordan’s Story Problems (using local norms to establish the percentile from a neighboring school district) for word reasoning. Using ROC curves, they indicated that their screeners had a good fit in targeting the number of students who would be identified as math disabled (however, there were some concerns--30 students were identified as MD (math disabled-calculation) by screening even though not by normative measure...so called “false alarms”. Seven students were identified as not MD-calculation even though the curriculum indicated that they were..so called “misses”. The numbers were 36 false alarms and 7 misses for MD-reasoning.
• Of the different screening measures-all but Number ID/Counting predict MD, with CBM-Applied Problems being the best predictor for both. The authors further hypothesize that screening measures that include a wide variety of problems (such as with applied problems) likely is a characteristic that helps with predictive power.
• I have a few thoughts on this study. Clearly, looking at the previous research has flagged that we probably should continue to focus on cognitive factors as guides to help with hypothesis generation (although clearly prior knowledge still seems to reign as most important in prediction..although that was not the focus of this study.
• I personally had concerns in the use of the WRAT and then Jordan’s Story Problems. One uses a nationally normed set of data, while the other used local norms from a nearby location. The authors indicated the sample was “local but representative”, although given all other outcome measures were nationally normed instruments--this posed as a problem for me.
• I was heartened to learn that three CBM measures performed adequately in predicting the presence of a MD and was also enlightened on the importance of applied problems (or math reasoning)--a construct that I think has been very much overlooked in much of the RTI literature on math in contrast to number of digits correct. However, when I see that there were over thirty “false alarms” for prediction--this is a cause for genuine concern. The implications go beyond providing “tutoring for those who do not need it.” It can lead to an attribution of “disability” when one might not necessarily exist..a Type I error. This can lead one to lower our expectations of students who may not necessarily present with deficiencies.
• My other concern was the difference in scores in the studies. Although the WRAT is noted to correlate at .71 with the WJ-III, the mean SS for the WJ-III at the beginning of first grade was 91 for MD and 71 for the WRAT at end of second grade. For word problems, the mean score for MD on WJ-III was 92 and about 66 on the Jordan. The standard deviations are pretty large for the WJ-III and WRAT (about 7-11 points) while smaller on the Jordan (about three points), but again I become concerned at how very dissimilar these profiles are. A large part of it can be explained by the calculation demands at beginning first (absolutely nothing outside of writing numbers) and end second (subtraction-possibly with regrouping)-although the magnitude is still very great. I might ask a quantitative expert to help me pull that apart further as this “applied practitioner” has been known to “stumble on the stats” at times.

Technorati Tags: psychology, educational psychology, education, school psychology, psychological measurement, psychological testing, screening, progress monitoring, RIT, mathematics, arithmetic, Gq, quantitative

Powered by ScribeFire.