Hawaii Race to the Top Evaluation

The Race to the Top (RTTT) initiative provided an unprecedented amount of federal competitive grant funding to push educational systems toward innovation using coherent and comprehensive reforms. AIR's evaluation of the grant in Hawaii found that, although student achievement in Hawaii has continued to improve during the course of RTTT, those gains did not appear to be the result of the grant, but rather the continuation of a trend of improvement that started earlier.
 

Project Summary

In August 2010, Hawaii became one of 12 winners of the RTTT competition, focusing its reforms on the areas of standards and assessment, data systems, effective teachers and leaders, and improvements in the lowest performing schools. These reforms were intended to work in concert to create better conditions for learning, better instruction for all students, and, ultimately, better opportunities for students in college and careers. Although student achievement in Hawaii has continued to improve during the course of RTTT, our analysis suggests that those gains did not appear to be the result of the grant, but are the continuation of a trend of improvement that started earlier.

In October 2011, the Hawaii Department of Education contracted with AIR to conduct a three-year, mixed-methods evaluation of the state’s RTTT initiative—the federal grant program that provides states with significant funds to reform and improve their education systems.

Through this independent evaluation, AIR has provided Hawaii with ongoing information on RTTT implementation to promote the continuous improvement of the state’s RTTT efforts and a summative evaluation of the initiative’s impact. The evaluation explored (1) the extent to which Hawaii’s RTTT initiative was implemented as planned; (2) the changes in education policy and professional practice that were associated with Hawaii’s RTTT implementation; and (3) the impact of Hawaii’s RTTT initiative on student achievement.
 

Findings

The RTTT grant has brought about deep and far-reaching change in Hawaii—among the most significant accomplishments have been the transition to the Common Core; the development of a new teacher performance system; and efforts to align and streamline activities in communication, planning, support, and monitoring for all schools. Most notably, the state has focused its reform work on six priority strategies in an effort to ensure that all schools work on the same core set of activities. Through RTTT, Hawaii established systems, procedures, and policies to promote a culture of data use across its complex areas and schools. Supporting teachers in using these data to make meaningful changes in classroom instruction remains a challenge. Teachers’ perceptions of RTTT have been mixed—and influenced by the intensive reform activities rolled out in the final year of the grant. 

Although student achievement in Hawaii has continued to improve during the course of RTTT, our analysis suggests that those gains did not appear to be the result of the grant, but are the continuation of a trend of improvement that started earlier. It may be too early to detect the impact of Hawaii’s reform efforts on student outcomes, given statewide implementation of core RTTT activities did not take full effect until the 2013–14 school year.
 

Methods

AIR conducted a rigorous mixed-method evaluation for the state of Hawaii, addressing to what extent the state’s RTTT program was implemented as planned and met its key benchmarks, especially for RTTT resource usage and participation. AIR evaluators analyzed which changes in policy and professional practice were associated with implementation as well as the resulting impact on student performance. Finally, the study team examined the impact of Hawaii’s RTTT initiative on student achievement.

The evaluation relied on an array of data collection methods, including interviews with key stakeholders, surveys of teachers and principals, review of documentation, and case studies. In addition, to measure the effect of Hawaii’s RTTT initiative, AIR used an interrupted time series model to measure changes in student performance. AIR provided timely, actionable information to the state of Hawaii; supported continuous improvement; and enabled RTTT leaders, participants, and stakeholders to judge the value and worth of the initiative and its components.
 

Research Questions

1. To what extent was Hawaii’s RTTT implemented as planned and meeting its key benchmarks, especially for RTTT resource usage and participation?
2. What changes in policy and professional practice were associated with Hawaii’s RTTT implementation?
3. What was the impact of Hawaii’s RTTT on student performance in relation to Hawaii’s RTTT targets?