“Thank you for bringing this program to our classrooms, the kids are so enthusiastic about the red bags (which allow them to take ownership over books when many of them have none at home).” Teacher, Community Day Care, Lawrence
Raising A Reader MA maintains a multi-tiered approach to monitoring and assessing our impact. At the core of our work, we aim to impact child pre-literacy skills. Our success hinges on two key indicators: (1) our ability to ensure that our partners implement the core program model with fidelity, and (2) that families are regularly integrating Dialogic Reading strategies into their routines.
Our program evaluation is designed to answer the following questions:
- Did our partners implement the core program model with at least 90% fidelity
- Do at least 25% of parents participate in a Dialogic Reading workshop or other program to learn high-impact home reading skills?
- Do 75% of families share books three or more times a week after participating in Raising A Reader MA?
If we achieve these outcomes we can use national evaluation data to link our program with improved child outcomes.
Fidelity of Implementation
In order to ensure that our program is implemented with fidelity, we carefully monitor community activities through a customized database to track the following outputs:
- The percentage of staff at Raising A Reader MA implementation sites who participate in Raising A Reader MA training
- The percentage of implementation sites facilitating the red bag book rotation program and related instructional activities with integrity
- The percentage of families who receive multilingual orientation materials about Dialogic Reading and the red bag book rotation program
- The percentage of communities offering Dialogic Reading workshops that are accessible to eligible families
Raising A Reader national maintains a body of research, including 20 independent evaluations, that shows that when these elements are in place, children show significant growth in multiple measures of language and literacy.
Our program staff collect and analyze data including participant demographics, training comprehension, program implementation by site, and constituent satisfaction at least twice a year to ensure programs are on track to achieve implementation and outcome goals. These core elements are aggregated in a partner scorecard, designed to assess each individual program’s, as well as the overall community’s, success.
Parent Behavior Change
Decades of research has shown that when parents read regularly with their children, the children’s pre-literacy skills improve. In order to track this important behavior change, Raising A Reader MA employs parent pre- and post- surveys as well as focus groups and interviews to access both the quantity and quality of their shared reading behaviors.
Annual program evaluation data shows that the percentage of families reading books at home three or more times a week increases by as many as 20 percentage points over a year, up to 75% reporting reading together at home three or more times a week. Baseline data collected at the start of the program shows that before the program, fewer than 55% of families report reading together at home regularly.
Child Outcomes
All of our evaluation elements link together to point towards improved child outcomes. In addition, we aim to directly measure these child pre-literacy skills. Our first quasi-experimental study using measures of vocabulary and letter awareness was designed and led in 2010 by Dr. Nonie Lesaux of the Harvard Graduate School of Education. We are currently partnering with school districts and the MA Department of Early Care and Education to share more of this critical data.
Management Information System
Raising A Reader MA uses a customized program evaluation and performance measurement tracking system built on the Salesforce platform. It fuses staff activity, partner assessment, constituent behavior and survey response data, allowing for real-time staff engagement with key metrics. Used daily, it allows for easier analysis to inform program strategy and data driven decisions.