"A test does not change from one time to another: people do. There may be considerable change on some traits, but relatively little on others. Test-retest studies evaluate the tendency for change in people, not some aspect of test quality. A test that does not reflect such changes in human traits would be an insensitive measure of those traits" (McGrew, Werder, & Woodcock, 199, p. 99).
Over on the NASP Listserv Dr. Gary Canivez asked the following question, in response to a post regarding changes in scores on the K-ABC and WJ---"Does anyone have references for long-term stability of WJ or KABC-2 scores? I'd be interested in references for such studies."
There was a very sophisticated test-retest study reported in the WJ-R Technical Manual (McGrew, Werder & Woodcock, 1991) (click here to view/download). Unfortunately it is in a test technical manual...a document that is too often ignored once a test is purchased. Additonal information can be found in the following article.
And then, of course, a person's state (concentration, anxiety, fatigue, etc.) at any testing moment can impact test performance...and test's sensitive to these states (e.g., Gsm, Gs) will likely reflect these temporary state fluctuations. This reflects state variance...which is NOT a problem with the measure...the measure is accurately reflecting how the person is doing at that time. School psychologists (and others who do psychological testing), unfortunately, typically receive measurement training that only talks about simple test-retest reliability studies and results....a diservice to our profession. We need to understand our instruments better. Properly designed test-retest studies, where the retest interval is varied, can help identify and portion the difference sources of test score variance to the appropriate sources of stable or unstable change score variance.
Read the WJ-R technical manual excerpt in particular. It is the most readable and understandable of the two sources I've included in this post.
Technorati Tags: psychology, school psychology, educational psychology, neuropsychology, psychometrics, reliability, test-retest, measurement, WJ-R, WJ III, K-ABC, KABC-II, Woodcock Johnson
Over on the NASP Listserv Dr. Gary Canivez asked the following question, in response to a post regarding changes in scores on the K-ABC and WJ---"Does anyone have references for long-term stability of WJ or KABC-2 scores? I'd be interested in references for such studies."
There was a very sophisticated test-retest study reported in the WJ-R Technical Manual (McGrew, Werder & Woodcock, 1991) (click here to view/download). Unfortunately it is in a test technical manual...a document that is too often ignored once a test is purchased. Additonal information can be found in the following article.
- McArdle, J. J., FerrerCaja, E., Hamagami, F., & Woodcock, R. W. (2002). Comparative longitudinal structural analyses of the growth and decline of multiple intellectual abilities over the life span. Developmental Psychology, 38(1), 115-142. (click to view)
And then, of course, a person's state (concentration, anxiety, fatigue, etc.) at any testing moment can impact test performance...and test's sensitive to these states (e.g., Gsm, Gs) will likely reflect these temporary state fluctuations. This reflects state variance...which is NOT a problem with the measure...the measure is accurately reflecting how the person is doing at that time. School psychologists (and others who do psychological testing), unfortunately, typically receive measurement training that only talks about simple test-retest reliability studies and results....a diservice to our profession. We need to understand our instruments better. Properly designed test-retest studies, where the retest interval is varied, can help identify and portion the difference sources of test score variance to the appropriate sources of stable or unstable change score variance.
Read the WJ-R technical manual excerpt in particular. It is the most readable and understandable of the two sources I've included in this post.
Technorati Tags: psychology, school psychology, educational psychology, neuropsychology, psychometrics, reliability, test-retest, measurement, WJ-R, WJ III, K-ABC, KABC-II, Woodcock Johnson
