Summary of findings from 'How are California’s Charter Schools Performing?'
June 2005
In a recent analysis of the performance of California’s charter schools, EdSource found that these schools--particularly those using a classroom-based approach--were substantially more likely to meet academic improvement goals in 2004 than were other public schools. Charters did not look as good on other performance measures, however, and important differences among charter schools and between charters and regular public schools must be kept in mind when making comparisons.
When test data is used as the measure of school performance, it can be done several different ways. The most accurate approach is to compare the performance of a particular group of students over time, but that cannot be done for all students in California because of limitations in the current student data system. California’s API system makes it possible, however, to do three other types of comparisons.
- Absolute comparisons ignore differences in student characteristics as California does with its API rankings. These are released each year along with schools’ Base API scores. The state system only makes absolute comparisons among all schools that serve similarly-aged students, i.e., elementary, middle, and high school.
- Similar school comparisons look at schools whose students share similar characteristics. This approach acknowledges that school performance is highly correlated to students’ backgrounds, most notably the education level of their parents. Schools in California receive a “similar schools” ranking that serves this purpose.
- Comparisons over time examine a single school’s improvement in performance. In California this involves a comparison of Base and Growth APIs in a two-year cycle. The state assigns each school a growth target that is a benchmark for its success and reports on whether each school met its target. A school’s API growth target is five percent of the difference between its Base score and the target score of 800. Student subgroups must improve by 80% of the school’s growth target for the school to meet all of its targets. For example, a school with a Base API score of 700 would have a growth target of 5 points, and its subgroups would have targets of 4 points. Schools with scores of 800+ are expected to keep their scores above 800.
It is important to keep in mind certain factors when analyzing charter school performance. First, the universe of charter schools is diverse. Some use a traditional classroom-based approach while others rely more on distance-learning and/or independent study. In addition, some charters are brand new while others have been around for ten or more years. Finally, some charter schools were started from scratch and others were once regular public schools that were converted to charter status.
In addition, the number of charter schools is growing at a very rapid rate so the universe of charter schools in California is quite different from year to year. Lack of data is also a significant obstacle. About one-quarter of charter schools—and more than 40 percent of nonclassroom-based charters—did not have test scores for enough students in the 2003/04 API cycle to be included in comparisons over time. Also, some charters have non-traditional grade structures (such as 2-7 or 9-10) that make comparisons problematic. Finally, direct comparisons between charters and noncharters can be misleading because of differences in student demographics. Generally, charter schools are less likely to serve Hispanic students, English learners, or students from low-income families.
In its analysis, EdSource found several noteworthy patterns of charter school performance. Collectively they present some reasons for optimism and some causes for concern. They also raise a number of interesting questions that should be explored more thoroughly.
Missing test data from a large percentage of the state’s charter schools undermines the ability to assess their academic performance and thus their effectiveness as an alternative learning environment for some students.
For the 2004 Growth Academic Performance Index, about 23% of the charter schools in the state’s API system lacked sufficient test data to even receive API scores. That compares to just 6% of the noncharter schools in the API system. Only 13% of classroom-based charter schools were missing data, but 46% of nonclassroom-based charter schools lacked the necessary data to produce a valid API score.
The California Department of Education lists a variety of reasons why schools do not receive a score, but there were two main reasons that charter schools were missing data. One is that the schools did not test a sufficiently large group of students to have statistical meaning and preserve individual students’ privacy. The other is that the students tested were not sufficiently representative of the school as a whole, often due to a large percentage of parents’ excusing their children from STAR tests, upon which API scores are largely based. Charters are more likely than non-charters to lack API scores because of a large percentage of parent opt-outs.
Among charter schools with data, particularly classroom-based charters, improvement meeting academic growth targets is particularly notable.
The relative performance of charters started to improve in 2003 when API scores began emphasizing the California Standards Tests instead of off-the-shelf norm-referenced tests. In 2004, 64% of classroom-based charters (which constitutes 60% of all charters) met their 2004 API growth targets, outperforming both noncharter schools (48%) and the nonclassroom-based charter schools (44%) for which data is available.
Elementary charters lag behind non-charters on Base API scores, but have recently done better meeting growth targets.
In an absolute comparison between all elementary charters and noncharters, the charter schools on average have slightly lower 2004 Base API scores. This is in spite of the fact that elementary charter schools serve a smaller proportion of English learners and low-income students than their noncharter counterparts. However, charter elementary schools have improved the rate at which they meet their API growth targets. In 2004 the group of elementary charters was the first to outperform noncharters, with 57% of them meeting their targets compared to 46% of noncharters.
The small group of charter middle schools performed quite well.
Just 31 charter middle schools were included in the state’s accountability system in 2003/2004 and that small number tempers any assertions about the academic performance of charters serving this age group. That said, among the 27 of those charter middle schools that had 2004 Growth API scores, 81% met their growth targets (compared to 54% of noncharter middle schools).
California’s middle grades charter schools on average serve fewer disadvantaged students than their noncharter peer schools. While it is therefore not surprising that they ranked relatively high on their 2004 Base API scores, they also compared favorably to noncharter middle schools with similar students based on the state’s similar schools rankings.
Firm conclusions about charter high school achievement are problematic.
Charter high schools increased in number very quickly, going from 34 schools with API growth data in 2003 to 120 in 2004. Most of these new charters are start-up schools that tend to differ substantially from noncharters in the students that they serve and the teachers that they employ.
More than half of charter high schools ranked in Deciles 1–3 (the bottom 30%) based on the 2004 Base API. But comparing them to noncharters serving similar students nets more mixed results, with equal proportions comparing well and poorly. In terms of improvement, missing data is again a challenge. Just 71 high schools had growth data for the 2003/2004 cycle. Of those schools, 58% met their growth targets compared to 49% of regular high schools.
In an EdSource study that matched 97 well-established charters against a large group of comparable noncharters, the charter schools’ test scores generally improved more than those of noncharters.
Based on an analysis of California Standards Tests (CSTs) for grades 3, 7, and 10 in English and grades 3 and 7 in math, charters did slightly better at increasing the percentages of students scoring “proficient and above” and decreasing the percentage scoring “far below basic” from 2002 to 2004. The differences appear slight, but they are statistically significant—meaning outside the normal range of random variation.
In eight out of 10 sets of school-level comparisons in which each charter was matched with a group of noncharters facing similar challenges and at a similar starting point with respect to performance, a majority of charters showed more improvement on the math and English CSTs from 2002 to 2004 than their comparison noncharter schools:
- Charters serving grade 3 students outperformed their comparison noncharter schools in improving their percentages scoring “far below basic” but not those scoring “proficient and above.”
- A majority of charters serving grade 7 students outperformed their comparison schools in both subjects and at both score levels.
- In 10th grade English, a majority of charters showed greater improvement than their matched noncharters at both score levels.
State and federal policymakers are eager to find solutions for the state’s chronically low performing schools. Under the No Child Left Behind Act, turning them into charter schools is one option, but is conversion to charter status an effective solution?
Not necessarily, but it might be under certain circumstances. Just as in the regular public school system, the charter community has examples of both low-performing and exemplary schools. Before state and local policymakers advocate the use of public tax dollars to convert low performing public schools to charter status, they should closely examine the types of charter schools that are succeeding, for what kinds of students, under what conditions.
That said there are real success stories among charter schools. Many of them are providing exciting learning environments. And many are successful serving students who for various reasons have not found a good fit in a regular public school setting, including many low-income children of color. Of particular note is the burgeoning use of charters as a strategy for addressing the stubborn challenges of high school reform, particularly in disadvantaged communities.
In addition—based on state testing data—charter schools have recently started to make impressive gains, improving performance and meeting academic growth targets faster than noncharters in many cases. The charter API growth data for 2004 is definitely promising, but one year does not yet make a trend.
California’s charter school experiment can be a lightning rod for conflicting political views. But students and the state would be better served if charter schools were instead seen as laboratories of school improvement that add to the information researchers, educators, and policymakers can use to make sure that every child has an opportunity for academic success in the state’s public schools.
Order the full report:
Trouble reading this page? Our site is best viewed in IE 5, Safari 1, Firefox 1 or AOL 9 or higher.