RAND Corporation
Implementation and Performance in New American Schools: Three Years into Scale-up,2000
The purpose of this study was to evaluate the implementation and performance trends of 104 New American Schools (NAS), including 16 Expeditionary Learning schools, nationwide. Expeditionary Learning was one of seven designs evaluated and was represented in three of the seven sampled cities, including Cincinnati, Memphis, and San Antonio. All NAS schools assessed in this evaluation initiated the implementation process in either 1995-1996 or 1996-1997, and the report states that it is common for successful school reform designs to take five years or more to lead to meaningful change. In general, NAS designs were located in cities that were predominantly urban, high poverty, and high minority. These factors are important to consider when comparing NAS schools to district and state averages. The report suggests that the overall performance results were mixed due to the wide variety of designs and cities included in the evaluation. In general, however, the cities including Expeditionary Learning schools demonstrated promising results, considering the relatively short period of time the schools had been implementing the design.
Implementation
In order to evaluate progress of designs implementation, each design was rated on a core implementation index based on the following six criteria:
- Parental and community involvement
- Student assessments in accordance with academic standards
- Teacher development of individualized learning programs for students
- Fluid student grouping
- Teacher participation in professional development and common planning
- Explicit performance expectation for students.
Furthermore, RAND pointed out various factors that tended to effect design implementation. Such factors included teachers perceptions of their students, school size, grade level, principal leadership, school resources, design team assistance, and district support.
In general, findings indicated that implementation progress was higher in schools that had been implementing the design for a longer period of time, as would be expected. In some cases, however, implementation did not change after the first year of implementation. The reports suggests that this trend might be due to lack of school support from the designs after second year of implementation, or to the designs becoming such a routine part of the school that implementation began to stabilize. When designs implementation progress was compared among cities, Memphis and Kentucky NAS schools ranked highest on the core implementation index. It is notable that five Expeditionary Learning schools were included in the analysis of Memphis implementation, since Memphis was one of three sampled cities that included the Expeditionary Learning design.
Performance
Performance analysis was based on the academic achievement of NAS schools in Math and Reading in comparison to district and state norms. Of the 163 sampled schools, about half demonstrated gains in both disciplines in comparison to their districts. When the evaluated cities performance levels were compared to one another, it was found that the New American Schools in Memphis and Kentucky were the most successful in Math improvement, whereas the Cincinnati and Washington schools were most successful in Reading improvement. As previously mentioned, the Expeditionary Learning design was included in Memphis, Cincinnati, and San Antonio samples, thus, Expeditionary Learning was represented in cities that excelled in both Math and Reading.
The following is an overview of performance results found in the three sampled cities representing the Expeditionary Learning design. Although Expeditionary Learning schools were included in each of these cities, Expeditionary Learning was only one of several designs that comprised NAS schools in each city.
Memphis
As a standardized test, Memphis has used the Comprehensive Test of Basic Skills, version 4 (CTBS/4) since 1990, but adopted the CBTS/5 Complete Battery Plus (Terra Nova) in 1998. Results from these tests were used to assess Memphis NAS schools performance in this evaluation. Prior to implementation of NAS designs, the elementary schools performance levels had been below both district and national norms. By 1997, however, the NAS schools that had initiated design implementation in 1995 outperformed both the district and national norms in Reading and Math. By 1998 New American Schools that had initiated design implementation after 1995 were also outperforming the district average in these two disciplines. In middle school grades, NAS schools outperformed the district in both Math and Reading by 1997. Thus, regardless of grade level, NAS student performance levels were better than district levels within only a few years after initial design implementation. The evaluation also concludes, " [the] higher poverty schools appeared to derived the greatest benefits from these reforms." (p. 118)
Cincinnati
Cincinnatis main standardized test changed from the California Achievement Test (CAT) to the Stanford-9 in the spring of 1997. Thus, results from both these tests were used to assess the performance of Cincinnati NAS schools. Since these two tests are not comparable, the results from each had to be analyzed independently, which made assessment of these scores fairly complex. In general, the elementary grades of the Cincinnati NAS schools appeared to make "modest gains" in comparison to the district, whereas the middle school grades did not do as well. It is notable that, over the period of time considered, the district as a whole experienced some major decreases in scores in both Math and Reading. Whereas some NAS schools scores paralleled these decreases, others did not. Overall, performance results from both Cincinnati NAS and non-NAS schools were quite mixed. By 1998, however, the majority of NAS schools were making greater progress in comparison to the district as a whole in both Reading and Math.
San Antonio
Schools in Texas administer the Texas Assessment of Academic Skills (TAAS) to students in grades 3-8, as well as to high school students starting in 10th grade. High school students are required to pass the TAAS in order to graduate from high school. The state employs a "percent passing" standard, which refers to the minimum acceptable passing standard. Score reports of the TAAS were used in this study to evaluate NAS designs in San Antonio. According to these score reports, both NAS and district schools were well below the state norms in both Reading and Math before NAS design implementation, and NAS schools tended to be below district averages. However, both NAS and other district schools made large gains in subsequent years, quickly narrowing the gap between their scores and state norms. This occurred at all grade levels, including middle, elementary, and high school grades. In some instances NAS schools gains in TAAS percentage points were greater than those of the district. For, example between 1995 and 1998 NAS schools 10th grade scores improved 30 percentage points in Math, compared to the district gain of only 24 percentage points. Furthermore, across grades and disciplines, NAS schools tended to make noticeably greater gains in comparison to those of the state since initial design implementation.
|