• Generating the Right Evidence to Inform Policy

Our current projects include analytic and design work on five large-scale randomized trials and other prominent efforts to provide WWC-certified technical assistance to both practitioners and researchers across the nation.

Technical Assistance for the National Evaluation of the Investing in Innovation (i3) Program

Purpose:

Investing in Innovation (i3) is a central U.S. Department of Education-funded competitive grant program that funds school districts and their nonprofit partners to implement research-based educational programs and practices to improve student outcomes. To date, the i3 program has awarded more than $1 billion to 117 grantees to scale-up, validate, or develop evidence-based education programs. Each i3 grantee is required to fund an independent evaluation to document the implementation and outcomes of the projects. The goals of the National Evaluation of i3 are to ensure that the government’s investment in educational interventions through i3 generates high-quality evidence to inform educational policy decisions and to boost the field’s capacity to design and conduct scientifically rigorous education research.

Project Activities:

To meet the goals of i3, Measured Decisions has partnered with Abt Associates to provide technical assistance to the evaluators of the i3-funded programs to ensure that their evaluations are well-designed and well-implemented.

Products:

Measured Decisions will work with Abt Associates to summarize the evaluations’ strength of evidence and findings in order to learn which programs, practices, and approaches hold the most promise for reshaping the nation’s schools.

Technical Assistance for Education Innovation and Research Evaluations

Purpose:

Education Innovation and Research (EIR) is a competitive grant program for which grantees must fund an independent evaluation of their projects’ effectiveness. Measured Decisions, in partnership with Abt Associates, provides technical assistance on the conduct of these evaluations to maximize the strength of their causal conclusions and the quality of their implementation data.

Project Activities:

Measured Decisions reviews the funded applications to assess the grantees’ needs, provide initial feedback on the evaluation plans, and develop the project technical assistance plan. Measured Decisions develops or adopts criteria for EIR evaluation expectations, develops an overall project technical assistance plan, and customizes technical assistance plans and initial evaluator feedback.

Products:

Measured Decisions and Abt Associates will summarize the evaluations’ strength of evidence and results and will document the funded projects’ final results along with the outcomes of the technical assistance delivered.

National Randomized Field Trial of Descubriendo la Lectura

Purpose:

The purpose of this study is to evaluate the efficacy of the Descubriendo la Lectura (DLL) program, which is the Spanish reconstruction of Reading Recovery, an English literacy program. The population of Spanish-speaking English learners (ELs) in United States schools is large and growing, and many of these students choose to enroll in bilingual instructional programs in early grades. As the number of students receiving initial literacy instruction in Spanish increases, the need for a Spanish language early intervention program for students at risk of literacy problems also increases. DLL has been evaluated previously using a randomized controlled trial (RCT) design) and in this study, researchers will attempt to replicate those results across a broader set of school sites. The team will also investigate how differences across students, teachers, and sites explain variability in the program’s observed effects, and examine the successes and challenges in replicating and reproducing the program’s processes and impacts across the diverse sample of teachers and sites.

Project Activities:

The research team will use a RCT to examine the impact of DLL on literacy achievement for two first-grade cohorts of Spanish-speaking ELs who are struggling readers. Researchers will randomly assign students to experimental conditions across 30 schools, and participate in pre- and post-tests of their Spanish and English reading skills using intervention-specific and standardized, norm-referenced measures. The team will collect measures of program implementation, and will examine factors hypothesized to moderate and mediate outcomes. In addition, researchers will compare the second grade literacy achievement of DLL students to a random sample of ELs who did not receive services, to look for any ongoing or later-emerging effects after the intervention is completed.

Products:

Researchers will provide evidence about the efficacy of the DLL reading intervention on improving literacy outcomes and closing achievement gaps for young Spanish-speaking ELs who are struggling readers. The research team will also produce peer-reviewed publications, conference presentations, and engage in web and social media outreach to disseminate their findings.

Promoting School Belongingness and Academic Performance: A Multisite Replication Trial of a Scalable Student Mindset Intervention.

Purpose:

The purpose of this study is to test the efficacy of a brief writing exercise designed to promote school belongingness and academic performance in middle school students. This study is a direct replication of a prior IES efficacy study to understand more fully how, for whom, and under what conditions a brief, inexpensive, school belongingness mindset intervention may impact students’ achievement at the transition to middle school.

Project Activities:

The research team will randomly assign two cohorts of 6th grade students to complete the school-belongingness writing exercise (treatment) or a neutral writing exercise (control) two times early in their 6th grade year. The team will test both short- (end of 6th grade) and long-term (end of 7th and 8th grade) impacts of the intervention on student social-psychological, behavioral, and academic outcomes.

Products:

Researchers will produce evidence of whether the positive impacts of a brief mindset writing intervention to promote school belongingness at the transition to middle school can be replicated in a different geographic setting with a different student population to support generalizability of findings. The researchers will disseminate findings through peer-reviewed publications and other reports shared via institutional websites, e-newsletters, and social media.

Reducing Achievement Gaps at Scale Through a Brief Self-Affirmation Intervention

Purpose:

In this study, researchers will test the effectiveness at scale of a low-cost self-affirmation mindset intervention on the achievement, behavior, and attitudes of 7th grade students testing specific hypotheses about the intervention’s impact on Black and Hispanic students. These minority student groups are susceptible to the threat of conforming to or being judged by negative stereotypes about the general underperformance of their racial/ethnic group (“stereotype threat”). The hypothesis is that the self-affirmation intervention will buffer these students from the harm of stereotype threat by affirming their core values in multiple domains, thus allowing students to increase their well-being and perform better in school. Prior evidence of the efficacy of this intervention has been mixed, depending on the school demographic composition and intervention implementation features.

Project Activities:

The researchers will recruit 48 schools and randomly assign students within the schools to either the intervention or control condition. All students will take a survey at the beginning and end of the school year. The teachers will be trained on how and when to administer the intervention (or control activity) during the year, prior to high-stakes assessments. The researchers will then collect student survey data (socio-emotional constructs) and school records (attendance, grades) to examine the effects of the intervention.

Products:

Researchers will produce evidence of the effectiveness of the self-affirmation mindset intervention when implemented at scale. The researchers will also produce peer-reviewed publications.

Quasi-experimental Evaluation of the Kids Read Now Summer Reading Program

Purpose:

This quasi-experimental study documents the impact of the Kids Read Now summer reading program on literacy achievement. The program includes both school-based and home-based components that together encourage students to remain engaged in reading high-quality books over the summer months. Through Kids Read Now, elementary-school students receive nine free books to promote reading during the summer break and to include in their home libraries. Using administrative data provided by several participating school districts, researchers will match control students to the Kids Read Now students within each participating school based on demographic information and achievement outcomes from the spring prior to students’ participation in the program.

Project Activities:

Kids Read Now will provide participation information, including the number of books requested and received by each participating student and his/her parent, guardian, or caregiver. The reading achievement data come from districtwide assessments and, thus, are of some policy relevance and practical importance to educators and students within the participating sites. We use propensity score matching methods to match Kids Read Now students from grades 1 through 4 to non-participating students from the same schools and grades. This quasi-experimental study provides the first opportunity to assess the impacts of the replicable Kids Read Now program and to provide potentially useful evidence to educators and policymakers concerned with promoting summer literacy learning.

Products:

Researchers will produce quasi-experimental evidence to help assess  whether the Kids Read Now program impacts the reading achievement of the participating students. The team will disseminate findings through peer-reviewed publications and other reports shared via Kids Read Now and social media.

A Multi-site Cluster Randomized Controlled Trial of the Impacts of Bernie’s Book Bank on Elementary Students’ Reading Behaviors, Attitudes, and Achievement

Purpose:

With this experimental study, the researchers will document the impacts of the Bernie’s Book Bank program on students’ reading achievement and their reported reading behaviors and attitudes. Since 2009, Bernie’s Book Bank has established an effective and replicable model for collecting millions of new and gently used surplus books and delivering them directly into the hands of low-income Chicago-area infants, toddlers, and school-aged children who have few or none of their own. This project expands the reach of the program to Milwaukee, Wisconsin and the Milwaukee Public Schools, where each participating student will receive eight free books each year. Using administrative data provided by the school district, researchers will examine the reading outcomes across a sample of 60 participating elementary schools, 30 of which were assigned to receive the program and 30 randomly assigned to a control condition. 

Project Activities:

Bernie’s Book Bank will provide book distribution data, including the number of books received by each participating school. Reading achievement data come from policy-relevant district and state assessments. In addition, a student survey administered by the researchers will assess how the program affected students’ reported book ownership, motivation to read, and engagement in various literacy behaviors. This experimental study will offer the first causal evidence of impact for the Bernie’s Book Bank program.

Products:

The results of the study will help policymakers and research understand the impacts of Bernie’s Book Bank on a range of key literacy outcomes. Reports and peer-reviewed journal articles will document the outcomes of the study. Additional dissemination will include professional conference presentations and brief announcements through various social media outlets.

A School-Level Randomized Controlled Trial of the Impacts of the Reading Is Fundamental (RIF) Books for Ownership Book Distribution Program on Students’ Longitudinal Reading Achievement

Purpose:

Using a school-level randomized design, the researchers will investigate the impacts of RIF’s Books for Ownership (BFO) program on students’ and schools’ reading achievement and reported literacy attitudes and behaviors. The primary component of the BFO program is a school-based book celebration event, during which free books are distributed to children. Schools implementing the program typically implement one or two book celebration events per school year. Most participating schools implement an event toward the end of the school year in an effort to encourage summer reading and prevent the characteristic “summer slide” in students’ reading performance. As BFO participants, schools are encouraged by RIF to distribute at least three books to each student per year. Using administrative data provided by the participating school district, researchers will analyze the reading outcomes across a sample of 60 participating elementary schools, 30 of which were assigned to receive the BFO program and 30 randomly assigned to a control condition. 

Project Activities:

Through RIF, the researchers will access book distribution data, including the number of books received by each participating school. Reading achievement data come directly from district and state assessments, which are used for student and school accountability. Finally, a student survey administered as part of the study will help measure how the program affected students’ reported book ownership, motivation to read, and engagement in a variety of positive literacy behaviors. This will represent the first experimental study of the RIF’s long-established, flagship program.

Products:

With the results, RIF, the policy-making community, and researchers will gain new understandings of the impacts of the BFO program on a range of important reading outcomes. Researchers will generate peer-reviewed journal articles that will disseminate the outcomes of the study. Additional dissemination will include reports, professional conference presentations, and information about the study and its findings as summarized through various social media outlets, such as Twitter.