SSTUWA - State School Teachers' Union of Western Australia

Charter Schools are not a Good Advertisement for School Autonomy

Charter schools are an experiment in school autonomy in the United States. After 20 years we can say that the experiment has not been a marked success. The weight of research evidence from meta-analyses, literature reviews, national and regional studies shows that charter schools have not delivered better results than other public schools.

1. Charter schools are an experiment in school autonomy

Charter schools are independent public schools in the United States. They are publicly funded but operate free from many of the laws and regulations that govern traditional public schools. In exchange for that freedom, they are bound to the terms of a contract or “charter” that lays out a school’s mission, academic goals, and accountability procedures.

The degree of autonomy for charter schools varies considerably between states. They are not permitted to levy fees but can obtain funding from private sources and many receive funding from philanthropic and other non-profit organizations. The large majority of charter schools are able to hire and dismiss staff, determine staff working conditions and the school schedule, determine their own curriculum and teaching methods, and control their budgets. The most common remaining restriction concerns requirements for teacher certification.

There are an estimated 5,600 charter schools in operation, serving approximately two million students in over 40 states and Washington DC. They constitute a small part of the US public education system, enrolling about three per cent of all students. However, they are a very significant component because turning traditional schools into charter schools and opening new charter schools is a key feature of US education policy. It is an idea that has been taken up in other countries such as Sweden, England and, to a lesser degree, Australia.

The autonomy given to charter schools was seen as a way to provide greater educational choice and innovation within the public school system and increase student achievement. There is much controversy over their success. Some studies show that they do no better than traditional public schools in terms of student achievement. Others show better results and still others show worse results. Other studies point to a range of other effects such as increasing social segregation of students.

Part of the controversy arises from different research methods used to measure the effects of charter schools and the range of factors influencing student results which are taken into account. Isolating the impact of charter schools from other factors involves sophisticated research methods and highly technical statistical modelling. Different methods have their own advantages and disadvantages which may lead to different results which are difficult for a lay-person to interpret and understand.

2. The impact of charter schools on student achievement

The first place to start in assessing the evidence about charter schools are professional ‘meta-analyses’ and literature reviews of studies employing sophisticated statistical techniques to account for a range of factors which influence student achievement. These reviews exclude studies which do not employ rigorous research methodologies, of which there are many.

Despite the fact that charter schools have been operating for over 20 years in some parts of the US, there are still surprisingly few rigorous studies that specifically study the impact of charter schools at the elementary level. There are also few high school studies. The large majority of rigorous studies of the impact of charter schools are at the middle school level. This in itself suggests a limitation in the available evidence.

Meta-analyses

Meta-analyses attempt to synthesise the results of several studies. The latest meta-analysis was published late last year by the US Center for Reinventing Public Education [Betts & Tang 2011; see also Betts & Atkinson 2012]. It included 25 studies of charter school performance and found “compelling evidence that charters under-perform traditional public schools in some locations, grades, and subjects, and out-perform traditional public schools in other locations, grades, and subjects” [p.1].

It concluded that there was no difference between charter and regular public schools in middle school reading and high school reading and mathematics. There were statistically discernible positive impacts of charter schools in elementary school mathematics and reading and in middle school mathematics. However, the effect sizes were very small. The largest effect found would move a student with median test scores — ranking at the 50th percentile — to around the 52nd percentile after one year at the charter school. Other positive effects were even smaller. A number of studies combine elementary and middle schools together and overall they find no significant effect of attending a charter school on reading or mathematics achievement.

The positive results for charter schools largely come from studies involving relatively few students. Three out of ten studies of elementary schools found negative results from charter schools and two of these studies involved large numbers of students. The number of students in these two studies was 1.6 million and 1.7 million, while the average number in the remaining studies was only 22,000. Most of the estimates finding positive results in middle schools are also from studies of relatively few students. The three studies of high schools which found positive results were also the three smallest studies.

The meta-analysis excluded the many studies of individual charter schools belonging to the Knowledge is Power Program (KIPP) schools. KIPP schools comprise only a very small proportion of charter schools. There are only about 100 KIPP schools in the US, but the studies of these schools account for about 75% of all studies of charter middle schools. A separate meta-analysis was done on these studies. It found largely positive and large effects of these schools in reading and mathematics.

KIPP schools have particular characteristics which may account for their results [Baker & Ferris 2011; Miron et. al. 2011]. They operate a much longer school day and school year than traditional public schools. Typically the school day lasts from 7:30am until 5:00pm weekdays and includes mandatory Saturday school every alternate week. They receive substantial additional funding from private sources, especially from large philanthropic organisations. KIPP schools also have much lower proportions of students with disabilities and English language learners than other public schools.

Charter school results by racial group are unimpressive. The impact of charter schools on white students is almost universally negative. The main exception is high school reading achievement, for which attending a charter school is associated with a positive and significant effect size. The impact on black and Hispanic students is mostly insignificant. The exceptions for Hispanic students are a negative effect on reading tests in middle school studies and a positive effect for high school mathematics. The results for low income students, special education students, and English language learners are positive, but small and statistically insignificant.

The meta-analysis arrived at the following conclusion:

The overall tenor of our results is that charter schools are in some cases outperforming traditional public schools in terms of students’ reading and math achievement, and in other cases performing similarly or worse. [p.55]

An earlier meta-analysis which synthesized the evidence across 47 studies came to the same conclusion [Miron et.al. 2008]. Overall, 19 studies had positive findings, 12 studies had mixed findings, and 16 had negative findings. The mean impact rating for charter schools was indistinguishable from zero. The overall conclusion of this meta-analysis was that charter schools perform similarly to traditional public schools [23].

Literature reviews

Literature reviews of charter school studies have also concluded that charter schools do not perform any better than traditional public schools. A review of major studies of charter schools published in the Handbook of Research in Education Finance and Policy concluded:

Research to date provides little evidence that the benefits envisioned in the original conception of charter schools – organizational and educational innovation, improved student achievement and efficiency – have materialized…Convincing evaluation of student achievement effects are now in from five different states. In none of these states have charter schools, on average, had large or unequivocally positive effects on student achievement. [Bifulco & Bulkley 2008: 440].

A review published by the Federal Reserve Bank of Chicago found that the weight of the evidence does not suggest that charter schools are more effective than traditional public schools [Rouse & Barrow 2008]. A recent review concluded that “the vast majority of charter schools get no better and no worse test-based results than comparable regular public schools” [Di Carlo 2011: 9]. Earlier reviews came to the same conclusion [Carnoy et.al. 2005; Hill et.al. 2006].

National studies

The other main source of evidence on charter school outcomes is large sophisticated national studies.

The largest, most rigorous and comprehensive study of student achievement in charter schools in the United States to date found that charter school results were worse than or no better than those of traditional public schools [CREDO 2009]. The study analysed the results of charter schools in 15 states and the District of Columbia and compared them with those of demographically matched students in nearby public schools. It found that the gains in maths results for nearly half of all charter schools (46%) were no different from those in comparable traditional public schools while over one third (37%) of charter schools had significantly worse results. Only 17% of charter schools had significantly higher maths results than students in comparable traditional public schools.

The study also analysed the aggregate impact of charter schools on student performance using a nationally pooled data set covering 70% of all charter school students. On average, the learning growth of charter school students was lower than their traditional public school peers, although the absolute differences were quite small. The gains in reading for charter school students were only slightly below that of traditional public school students while the gains in maths were significantly less for charter school students.

The study concluded:

...this study reveals in unmistakable terms that, in the aggregate, charter students are not faring as well as their TPS counterparts. Further, tremendous variation in academic quality among charters is the norm, not the exception. The problem of quality is the most pressing issue that charter schools and their supporters face. [CREDO 2009: 6]

A large study of middle school charters commissioned by the US Department of Education’s Institute for Education Sciences also found no difference in student achievement between charter schools and traditional public schools [Gleason et.al. 2010]. It compared the outcomes of 2,330 students in 15 states who applied to charter schools and were randomly assigned by lotteries to be admitted or not admitted to the schools. On average, the charter schools were neither more nor less successful than traditional public schools in improving mathematics or reading test scores, attendance, grade promotion, or student conduct within or outside of school.

Similar findings were made in another study of charter schools across eight US states by the RAND Corporation [Zimmer et.al. 2009; see also Zimmer et.al. 2012]. It found that student achievement in charter schools was either lower than or does not differ substantially from those of traditional public schools. It further found that competition from charter schools does not increase student achievement in nearby traditional public schools.

Earlier sophisticated national studies came to the same conclusion. A study commissioned by the US Education Department covering 150 charter schools and 6,764 public non-charter schools found that average results in reading and mathematics in charter schools were lower than those for traditional public schools [Braun et.al. 2006]. Another study using national test results in mathematics found that traditional public schools achieved significantly higher results in grade 4 while charter schools achieved slightly higher, but statistically insignificant, in grade 8 [Lubienski & Lubienski 2006].

Regional studies

While the latest evidence from national studies generally shows that charter schools do not achieve any better results than traditional schools, some recent sophisticated studies of charter schools in Boston and New York City have found gains by charter schools compared to traditional public schools [Abdulkadiroglu et.al. 2009, Hoxby et.al. 2009; CREDO 2010]. However, the gains are over-stated in two of these studies. The Boston study only included high achieving charter schools, which comprised only 7 out of 29 charter schools in the city [Jennings 2009]. The Hoxby New York study contained methodological problems which when corrected resulted in much lower gains [Reardon 2009]. The size of the gains by charter schools in the CREDO study is relatively small.

Numerous studies have now been done on the impact of charter schools on student achievement in many states, cities and school districts across the US. Charter schools in some locations have done better than traditional public schools, in others they have done worse and in others no better.

3. Conclusion

Thus, the general weight of evidence is that charter schools are no more successful than traditional public schools in terms of student achievement. The overview of a recent special issue of the journal Economics of Education Review on the charter school experience concluded:

... the existing literature is inconclusive about the aggregate effect charter schools have on student achievement. Some studies in some locations find charters outperform traditional public schools, some find they are no different than the traditional ones, and some find they perform worse. [Toma & Zimmer 2012: 209]

As another reviewer has said: “There is no test-based evidence for supporting either form of governance solely for its own sake” [Di Carlo 2011: 4]. The charter school experience suggests that school autonomy is not a magic bullet for increasing student achievement.

References

• Abdulkadiroglu, A.; J. Angrist; S. Cohodes; S. Dynarski; J. Fullerton; T. Kane and P. Pathak 2009, Informing the Debate: Comparing Boston’s Charter, Pilot and Traditional Schools The Boston Foundation, Boston, January. Available at: http://www.tbf.org/utilitynavigation/multimedialibrary/newsdetail.aspx?id=9490.
• Baker, B.D. and R. Ferris 2011. Adding Up the Spending: Fiscal Disparities and Philanthropy among New York City Charter Schools. National Education Policy Center, Boulder. Available at: http://nepc.colorado.edu/publication/NYC-charter-disparities.
• Betts, J.R. and E. Tang 2011. The Effect of Charter Schools on Student Achievement: A Meta-Analysis of the Literature. Center for Reinventing Public Education, Seattle. Available at: http://www.crpe.org/cs/crpe/view/projects/1
• Betts, J.R and R.C. Atkinson 2012. Better Research Needed on the Impact of Charter Schools, Science 335, 13 January, 171-72.
• Bifulco, R. and K. Bulkley 2008. Charter Schools. In Ladd, H.F. & Fiske, E.B. (eds.). Handbook of Research on Education Finance and Policy. London: Routledge, 425-446.
• Braun, H.; F. Jenkins and W. Grigg 2006. A Closer Look at Charter Schools Using Hierarchical Linear Modeling. NCES 2006-460, National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education, Washington, DC, August. Available at: http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2006460.
• Carnoy, M.; R. Jacobsen; L. Mishel and R. Rothstein 2005. The Charter School Dust-Up. Economic Policy Institute & Teachers College Press, Washington & New York.
• Center for Research on Education Outcomes (CREDO) 2009. Multiple Choice: Charter School Performance in 16 States. Stanford University, Stanford, CA, June. Available at: http://credo.stanford.edu/ .
• CREDO 2010. Charter School Performance in New York City. Stanford University, Stanford, CA. Available at: http://credo.stanford.edu/ .
• Di Carlo, M. 2011. The Evidence on Charter Schools and Test Scores. The Albert Shanker Institute, December. Available at: http://www.shankerinstitute.org/publications/charterreview/
• Gleason, P.; M. Clark; C. Clark Tuttle and E. Dwoyer 2010. The Evaluation of Charter School Impacts: Final Report. NCEE 2010-4029, National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education Washington, DC. Available at: http://ies.ed.gov/ncee/pubs/20104029/.
• Hill, P.T.; L. Angel, and J. Christensen 2006. Charter School Achievement Studies, Journal of Education Finance and Policy, Winter: 139-150.
• Hoxby, C. M.; S. Murarka and J. Kang 2009. How New York City’s Charter Schools Affect Achievement. New York City Charter Schools Evaluation Project, Cambridge, MA, September. Available at: http://www.nber.org/~schools/charterschoolseval/.
• Jennings, J. 2009. The Boston Pilot/Charter School Study: Some Good News, and Some Cautions, Eduwonkette blog, Education Week, 7 January. Available at: http://blogs.edweek.org/edweek/eduwonkette/2009/01/the_boston_pilotcharter_school.html .
• Lubienski, S.T. and C. Lubienski 2006. School Sector and Academic Achievement: A Multilevel Analysis of NAEP Mathematics Data. American Educational Research Journal 43(4): 651-698.
• Miron, G.; J.L. Urschel, and N. Saxton 2011. What Makes KIPP Work? A Study of Student Characteristics, Attrition, and School Finance, Working Paper. New York, NY: National Center for the Study of Privatization in Education. Available at: http://www.ncspe.org/list-papers.php
• Miron, G.; S. Evergreen and J. Urschel 2008. The Impact of School Choice Reforms on Student Achievement. Education and the Public Interest Center & Education Policy Research Unit, Boulder CO. Available at: http://nepc.colorado.edu/publication/the-impact-school-choice-reforms-student-achievement. Reardon, S.F. 2009. Review of “How New York City’s Charter Schools Affect Achievement.” Education and the Public Interest Center & Education Policy Research Unit, Boulder CO. Available at: http://epicpolicy.org/thinktank/review-How-New-York-City-Charter.
• Rouse, C.E. and L. Barrow 2008. School Vouchers and Student Achievement: Recent Evidence, Remaining Questions. Working Paper 2008-08, Federal Reserve Bank of Chicago. Available at: http://www.chicagofed.org/webpages/publications/working_papers/2008/wp_08.cfm.
• Toma, E. and R. Zimmer 2012. Two Decades of Charter Schools: Expectations, Reality and the Future, Economics of Education Review 31 (2): 209-212.
• Zimmer, R.; B. Gill; K. Booker; S. Lavertu; T.R. Sass and J. Witte 2009. Charter Schools in Eight States: Effects on Achievement, Attainment, Integration, and Competition. RAND Corporation, Santa Monica. Available at: http://www.rand.org/pubs/monographs/MG869/ .
• Zimmer, R.; B. Gill; K. Booker; S. Lavertu and J. Witte 2012. Examining Charter Student Achievement Across Seven States, Economics of Education Review 31 (2): 213-224.