What Do We Know about Effectiveness?
Only the abstracts of papers documenting research on the Accelerated Reader undertaken by the University of Dundee are given here. Other evidence is reviewed in the third item listed below. Interested readers should consult the full texts at source. Parallel research on Accelerated Math has been conducted by Professor James Ysseldyke at the University of Minnesota.
Vollands, S.R., Topping, K.J. & Evans, H.M. (1999) Computerized self-assessment of reading comprehension with the Accelerated Reader: Action research. Reading & Writing Quarterly, 15(3), 197-211 (themed issue on Electronic Literacy).
This study was a quasi-experimental action research evaluation of a program for computerized self-assessment of reading comprehension (the Accelerated Reader). It looked at the formative effects on reading achievement and motivation in two schools in severely socio-economically disadvantaged areas. The results suggested that the program, even when less than fully implemented, yielded gains in reading achievement for these at-risk readers that were superior to gains from regular classroom teaching and an alternative intensive method, even with less time devoted to class silent reading practice than in comparison classes. Additionally, the program yielded significant improvement in measured attitudes to reading for girls.
Topping, K.J. & Paul, T.D. (1999) Computer-assisted assessment of practice at reading: A large scale survey using Accelerated Reader data. Reading & Writing Quarterly, 15(3), 213-231 (themed issue on Electronic Literacy).
This study explored the relationship between practice at reading, student reading performance, and organizational features of the school system. Data generated through a program for computer-assisted student self-assessment of reading comprehension (the Accelerated Reader) were gathered as a measure of reading practice for over 659,000 students in grades K-12 in one school year. Students and states performing high and low on reading tests were compared. The data suggested that student reading ability was strongly positively related to amount of in-school reading practice. The time allocated in school to reading practice on self-selected materials declined after grades 5/6. The amount of reading practice was negatively correlated with school size. More reading practice occurred in private than public schools. Implications for action are discussed.
Topping, K. J. (1999) Formative assessment of reading comprehension by computer. Reading OnLine (I.R.A.) [Online]. Available: http://www.readingonline.org/critical/topping/ [November 4]. (hypermedia).
This multimedia review sought to consider the advantages and disadvantages of the Accelerated Reader LIS, and the evidence for its effectiveness. The full text is freely available by clicking the address above. The conclusions are given below:
Of 12 studies of AR that cite substantial outcome data, mostly on norm-referenced test scores, only one failed to find evidence of a positive impact. However, these studies are of very mixed quality, with many failing to control confounding variables or to provide data on implementation integrity; they are consequently unable to be definitive about causal direction. However, more recent studies suggest that, with good quality implementation, the AR program can contribute to teacher effectiveness, in terms of value added in reading and other core curricular areas. In short, it is not whether you have the software but rather what you do with it that makes a difference.
The characteristics of good and poor implementation were outlined, together with the potential advantages and disadvantages of the program. Whatever its advantages, AR is not a substitute for balanced reading instruction. Rather, it is intended as a supplementary and complementary resource - albeit a powerful one under the right circumstances - that can help the teacher deliver the curriculum effectively.
These conclusions have implications for local, state, and national guidelines on literacy instruction that aim to raise teacher effectiveness and standards of achievement. Placing intelligent software in classrooms does not guarantee it will be used intelligently. Information technology is not a replacement for the teaching professional, but a tool with the potential to enhance teacher effectiveness. In the case of AR, as for other Learning Information Systems, appropriate and sufficient high-quality training and support for teachers are needed if implementation integrity is to be sustained at the level necessary to raise student attainment.
Topping, K. J. & Sanders, W. L. (2000). Teacher effectiveness and computer assessment of reading: Relating value added and learning information system data. School Effectiveness and School Improvement, 11(3), 305-337.
The Tennessee Value-Added Assessment System (TVAAS) has for several years used the largest longitudinally merged database of student achievement data in the U.S.A. to generate estimates of school system, school, and teacher effects on indicators of student learning in a number of subjects, including reading comprehension. A Learning Information System (LIS) (not to be confused with an Integrated Learning System) is a formative, curriculum-based, criterion-referenced assessment delivered and scored by computer, which provides detailed feedback in different forms for different stakeholders. The Accelerated Reader (AR) LIS for reading comprehension of real ("trade") books captures the data for each student on the length and difficulty of each book, and the student's percent correct response on the assessment. It aims to help teachers promote and manage effective reading practice. AR data on 62,739 students from grades 2 to 8 in Tennessee schools was merged with the TVAAS teacher effects database, and relationships between these independently obtained measures explored. This illuminated some factors in teacher management of the quality and quantity of student reading practice which could be causative in improving reading comprehension, and the impact of implementation integrity of the LIS.
Topping, K. J. & Fisher, A. M. (2003). Computerised formative assessment of reading comprehension: Field trials in the UK. Journal of Research in Reading, 6(3), 267-279.
Increased curriculum time allocated to reading might not be effective in raising achievement. Teachers need to closely monitor and manage both the quality and quantity of individualised reading of all their pupils for optimal effectiveness. "Learning Information Systems" (LIS) for reading such as the "Accelerated Reader" (AR) enable this through individualised computerised assessment of pupil comprehension of "real books", with feedback to both pupil and teacher. This study explored the impact of AR on reading achievement in 13 schools of different types spread across the UK, the majority socio-economically disadvantaged. Participating pupils were aged 7-14 years. Pre-post norm-referenced gains in reading achievement were measured by group paper reading tests and a computer-based adaptive reading test. The implementation integrity of AR was assessed by direct observation by researchers and through data generated by the programme itself. On both paper and computer-based reading tests, on aggregate pupils in the 13 schools gained in reading at abnormally high and statistically significant rates. Boys tended to show larger gains than girls on the paper test. However, implementation integrity was very variable. In particular, some teachers failed to intervene in response to AR data indicating that pupils were reading ineffectively. AR appears to have potential for raising reading achievement, but only if implemented appropriately.
Topping, K. J., Samuels, J., & Paul, T. (2007). Does practice make perfect? Independent reading quantity, quality and student achievement. Learning and Instruction, 17(3), 253-264.
Does reading practice make perfect? Or is reading achievement related to the quality of practice as well as the quantity? To answer these questions, data on 45,670 students in grades 1-12 who read over 3 million books were analyzed. Measures largely of quantity (engaged reading volume) and purely of quality (success in reading comprehension) showed a positive relationship with achievement gain at all levels of achievement. However, both high quantity and high quality in combination were necessary for high achievement gains, especially for older students. Both were weakly associated with student initial reading achievement, but more strongly associated with the classroom in which the student was enrolled, possibly suggesting the properties of teacher intervention in guiding independent reading were important. Implications for theory-building, research and practice are explored.
Topping, K. J., Samuels, J., & Paul, T. (2007). Independent reading: The relationship of challenge, non-fiction and gender to achievement. British Educational Research Journal, 34(4), 505-524.
To explore whether different balances of fiction/non-fiction reading and challenge might help explain differences in reading achievement between genders, data on 45,670 pupils who independently read over 3 million books were analyzed. Moderate (rather than high or low) levels of challenge were positively associated with achievement gain, but non-fiction read was generally more challenging than fiction. Non-fiction reading was negatively correlated with successful comprehension and reading achievement gain. Overall, boys appeared to read less than girls, but proportionately more non-fiction, but this less carefully - especially in the higher grades - and had lower reading achievement. Differences between classrooms in promoting successful comprehension of non-fiction were evident, suggesting intervention could improve achievement. Implications for research and practice are explored.
Topping, K. J., Samuels, J., & Paul, T. (2007). Computerized assessment of independent reading: Effects of implementation quality on achievement gain. School Effectiveness and School Improvement, 18(2), 191-208.
This study elaborated the "what works?" question by exploring the effects of variability in program implementation quality on achievement. Particularly, the effects on achievement of computerized assessment in reading were investigated, analyzing data on 51,000 students in grades 1-12 who read over three million books. When minimum implementation quality criteria were met, the positive effect of computerized assessment was higher in the earlier grades and for lower achievement students. Implementation quality tended to decline at higher grade levels. With higher implementation quality, reading achievement gains were higher for students of all levels of achievement and across all grades, but especially in the higher grades. Very high gains and effect sizes were evident with very high implementation quality, particularly in grades 1–4. Implications for practice, the interpretation of research, and policy are noted.