Frequently Asked Questions About 'Similar Students, Different Results: Why Do Some Schools Do Better?'

October 2005

Study findings       Press release

What was the purpose of this study?

The study sought to determine why California elementary schools serving similarly-challenged student populations vary in their performance on the state’s Academic Performance Index (API) by as much as 250 points (on a scale of 200 to 1,000). EdSource, along with researchers from three other institutions, surveyed K-5 classroom teachers and principals at 257 schools to determine what practices are more common at high-performing schools than at low-performing schools.

What were the study’s key findings?

The study identified four interdependent practices associated with higher API scores: prioritizing student achievement; implementing a coherent, standards-based curriculum (defined below); using assessment data to improve student achievement and instruction; and ensuring the availability of instructional resources.

These practices tended to occur together, creating a cumulative effect. No single action or category of actions provides a clear advantage to student performance.

Which schools were included in the study?

The study selected elementary schools in the 25th to 35th percentile of the California School Characteristics Index (SCI), a measure of student demographics. Schools in this SCI “band” generally serve a high percentage of low-income students but are not the poorest in the state. The median school(*) in this sample had:

• 40% English learner students;
• 78% of students participating in the free or reduced-price meals program;
• 32% of students with parents who were not high school graduates; and
• An ethnic breakdown that was 68% Hispanic, 14% white, 4% African American, and 3% Asian. The remaining 2% were American Indian, Filipino, Pacific Islander, multi-ethnic, or students who did not indicate an ethnicity.

The 25th-35th SCI “band” includes 547 elementary schools. The researchers contacted 269 of these schools and 257 chose to participate. The survey included results from approximately 5,500 teachers and 257 principals in 145 California school districts. Only traditional public schools were included in the survey. Charter and private schools were not included. (*) The median is found by arranging schools in order of their values on a particular variable and finding the value in the middle of the group. As a result, there is no one school that represents the median on all of these factors.

What makes this study different?

Many studies have examined high-performing schools as a group, in an effort to understand their methods. This study took a different approach. Rather than studying a specific performance zone, the researchers examined all schools within a specific demographic band, and then used a regression analysis to determine which activities more common at high performing schools than at low performing schools were correlated with high API scores. This methodology succeeded in identifying practices associated with high performance that can be replicated at most schools.

What is the Academic Performance Index, and how is it calculated?

An API score is a one-number summary of multiple test scores, with different tests receiving various weights in the index. Schools receive an API score between 200 and 1000. The schools surveyed in this report varied on the 2005 growth API by as much as 250 points despite having relatively similar student profiles. In addition to the school as a whole, schools receive a score for each “numerically significant” subgroup of pupils categorized by ethnicity or poverty.

Can the survey’s conclusions be applied to students in schools outside of the 25th to 35th percentile of the School Characteristics Index?

Because the researchers focused solely on schools within the 25th to 35th SCI band, they cannot say that the results are generalizable to schools outside that band. Educators working in schools with similar student populations may, however, want to consider whether the practices this study found to be correlated with higher API might also work for their schools and students.

How can educators use these results to improve performance at their schools?

The survey results indicate that elementary schools serving low-income students that strongly engage in the four interdependent practices identified are more likely to have high API scores. However, this study was designed to detect correlations between practices and API scores only—it did not measure whether or not these practices had actually or solely caused the higher API scores in these schools. Nonetheless, based on the strength of the correlation detected, educators working to improve performance may wish to focus their efforts on the four interdependent practices identified.

One of the key practices identified is the development of a coherent, standards based curriculum. What is this?

Generally speaking, a coherent curriculum means that all students within a grade cover the same material, and the material students learn in one grade builds on the material they learned in the previous grade. For example, a coherent math curriculum could mean that the number skills students learn in first grade are necessary to master the arithmetic skills they learn in second grade. Standards based means that the curriculum and instructional program are strongly aligned with the state’s academic content standards.

What school improvement practices other than the four identified by the study were covered in the study’s survey?

Questions about involving and supporting parents, building a positive school environment with effective student behavior policies, and supporting teacher collaboration and professional development were included in both the teacher and the principal surveys. They were all found to be positively correlated with API scores. However, these practices did not have nearly the impact of the four key interdependent practices. The surveys may be viewed at www.edsource.org.

What is EdSource?

EdSource was established in 1977 as an independent, impartial, not-for-profit organization whose sole mission is to clarify complex California education issues and to promote thoughtful decisions about public school improvement. EdSource does not advocate or lobby and has developed a solid reputation as a credible and respected source of K–14 education information.

Who was on the research team that conducted the study?

The research team consisted of a group of experts from EdSource, Stanford University, the University of California, Berkeley, and the American Institutes for Research. EdSource team members included Trish Williams (Project Study Director), Mary Perry, Carol Studier, and Noli Brazil. Team members from Stanford University were professors Michael Kirst (Principal Investigator), Edward Haertel (Senior Technical Consultant), and Sean Reardon. University of California, Berkeley, team members included Elisabeth Woodey and Melissa Henne. Jesse Levin and Roger Levine were the team members from the American Institutes for Research. The complete biographies of research team members are available at www.edsource.org.

The full Research Report of Initial Findings is available for free download as a .PDF file at www.edsource.org. Report appendices are also available for download.