Why should anyone trust the data analyses and interpretation I have posted to date regarding my Carroll Analyses of 50 CHC-designed tests (3-17-05 and 3-18-05 )? I’ve asked this question of myself many times and had to resolve it to my satisfaction before launching this blog. The tipping point came when I recently read an interview in the Educational Researcher (Vol 33, No 3, April 2004 ) with one of the leading and respected educational researchers of our times—Dr. Gene Glass.
First---who is Gene Glass and why are his thoughts worth merit?
As stated in this interview article:
- Gene V. Glass is a Regents’ Professor of both Educational Leadership & Policy Studies and Psychology in Education at Arizona State University. He has won the Palmer O. Johnson Award for best article in the American Educational Research Journal (AERJ) not once, but twice (1968 and 1970). He has served as President of the American Educational Research Association (AERA) in 1975, co-editor of AERJ (1984–1986), editor of Review of Educational Research (1968–1970) and Psychological Bulletin (1978–1980), is executive editor of the International Journal of Education and the Arts (since 2000), and serves as editor of Education Policy Analysis Archives (since 1993) and Education Review (since 2000).
- Dr. Glass has also served on the editorial boards of 13 journals and has published approximately 200 books, chapters, articles, and reviews. Dr. Glass is perhaps best known for his role in the development of the quantitative research synthesis technique known as meta-analysis. Not shabby! And, Dr. Glass has been a vocal advocate of on-line e-journal publications, and, more recently, the publication and posting of non-peer reviewed manuscripts (he has written) for dissemination to other scholars.
So…what did he say and why should it make you consider the results and interpretations I post as credible? Or, as stated by Glass in his ER interview (note – all italics are emphasis added by me):
- "Now, the questions in most people’s mind are: Won’t there be chaos if everybody 'publishes' anything they want? How will we be able to separate wheat from chaff, truth from error, pure gold from garbage? There are answers, of course, and more questions."
According to Glass (in the ER interview), and consistent with my belief in the need for more timely dissemination of scholarly insights:
- “Some papers that I have made public through my own Website are downloaded a half dozen times a day, often from places that have no access to the traditional journals. That’s reaching a wider, larger audience than paper journals reach.”
Furthermore, my beliefs and experiences coincide with those of Dr. Glass:
- "Let’s get down to brass tacks. Scholarly communications are not essentially about paper vs. Internet “packets”; they are about commercialization vs. open access to knowledge. My experiences with publishing e-journals over the last decade have taught me that many more people than we ever dreamed want access to educational research: parents, teachers, professionals of many types, students and scholars far from the United States who cannot afford our books and journals.
I resonate to Dr. Glass’s observed paradox in the importance of referred publications. According to Glass:
- "There’s an irony hiding in this business of “refereed publication.” If I am vaguely interested in a topic that isn’t at the center of my work, then I’m prone to pick up a peer-reviewed journal that rejects 90% of everything sent to it. I simply don’t have the background to plow through tons of stuff and judge it for myself; trust the experts."
However, if someone is going to play in my professional sandbox (i.e., present research regarding CHC theory and measurement), I believe I am best served by being my own filter of all available information. As stated by Glass:
- "But if you are talking about things that come to the core of my own research (let’s say, the re-segregating effects of school choice policies), then please don’t filter out anything for me; I want to see whatever anyone is writing on the subject and I want it right away. I’ll judge it for myself. When we interviewed a sample of “hard scientists” about their reading habits, they said exactly that."
It is my belief that the readers of this blog must serve as their own filters, judges, and reviewers of any data-based results I present. Those knowledgeable in the areas where I present data and/or hypothesis are in the best position to evaluate the integrity of the information presented. In hope that the Blogs “comments” feature will provide an outlet for those who want to question or critique what is presented. In the pursuit of knowledge that (ultimately) may impact the lives of individuals (e.g., individuals referred for comprehensive psycho-educational assessments), I believe in open access and the timely dissemination of results to the widest audience possible. Thus, this blog, which will be judged by the readers and users.
Finally, like Dr. Glass, I still believe in, and continue to pursue, the publication of scholarly results via the mechanism of peer-reviewed journals. There is more than one means by which to contribute to our respective professional domains of knowledge.
Ok….within the next few days I will return to interpretation of the Carroll Analysis of 50-CHC designed tests…starting first with the most interesting Gf factor. And, in the spirit of accountability for the results I present, when done, I will post the correlation matrix analyzed for independent analysis by others.
