Background
2017
The First Collaboration between SERP and the District of Columbia Public Schools
SERP first worked with DCPS on a project to scale the Strategic Adolescent Reading Intervention (STARI), a program initially developed in partnership with Boston Public Schools. With an EIR mid-phase grant, SERP had the opportunity to engage four new districts outside of Massachusetts in an implementation and evaluation of STARI. DCPS joined that study.
2018
Field-site partnership formed
DCPS indicated an interest in becoming a SERP Field Site. Beyond participating in the evaluation of a SERP program, field site partnerships involve deeper commitments from both SERP and the district. Field site partners nominate their own problem of practice, which becomes the focus of ongoing, collaborative work to understand the problem in context and iteratively design and test a scalable solution to the problem.
DCPS’ Proposed Problem of Practice:
- Why are schools in the same geographic area with the same demographic characteristics producing such different literacy outcomes for their students, given that the curriculum, guidelines, and resources provided by the district are the same?
- Can we identify practices in the higher performing schools that might be successfully transferred to the lower performing schools?
Read more
DCPS had developed policies and practices over two decades aimed at improving early literacy outcomes.
These included:
- Universal preschool for 3 and 4 year olds.
- Adoption of a common phonics program and a phonemic awareness program in every elementary school.
- Development of a core curriculum district-wide to develop comprehension skills and background knowledge.
- Dedicated weekly teacher release time for teacher collaboration in every school, facilitated by an instructional leader.
- Implementation of IMPACT: a teacher evaluation program based on 5 Essential Practices rooted in research evidence, along with a system of rewards and penalties for those at the ends of the performance distribution.
As these reforms took hold, the district’s NAEP scores rose faster than any urban district in the nation. However, the achievement gap widened. The district noted that in the lowest income areas of the city, schools with the same demographic characteristics were producing widely differing student outcomes. District leaders were looking for an explanation for that variation, as well as the identification of contributors to higher performance that might be spread to lower performing schools.
Partnership Grant:
SERP & DCPS applied for an IES Partnership Grant. Catherine Snow (Harvard University) and David Blazar (University of Maryland) joined the research team. Intensive observations in 6 schools and analyses of district-wide data were proposed, with the goal of identifying opportunities for intervention. The grant was awarded in 2019.
2019-2021
Observations conducted to identify malleable contributors to student literacy outcomes
The observations were conducted in two waves.
2019-2020:
The district selected 6 schools with varying levels of growth in student reading outcomes and asked principals in those schools to allow the SERP team to conduct observations in K-2 classrooms (1 per grade). Two researchers took detailed notes to establish sources of variation to inform observations in year 2. Observations were cut short when the pandemic abruptly disrupted the school year.
2021-2022:
An observation protocol was developed based on year 1 observations, and principals were asked to permit observations in all K-2 classrooms for the entire 120 minute block. Five schools complied with the request and the sixth principal selected one teacher/grade for participation in the observations.
2021-2022
Analyses showed windows of opportunity for development
Observations showed wide variation in the extent to which students were asked to explain their thinking (as opposed to answering a question without explanation). The variation consisted both of teacher prompting for explanations to the whole class, and the opportunities for students to explain their thinking to a partner (turn and talks).
Observations also captured wide variation in productivity during the block of time (intended to be 60 minutes) when students were at literacy workstations while the teacher worked with a needs-based small group. In some classrooms, students moved efficiently to well-planned stations and engaged in focused work, while in others, students lost time during transitions and were unable to sustain focus, often on activities that were not planned as a reinforcement of the skills being taught.
Read more
These observations were consistent with district data indicating that teachers scored lowest on “student ownership” in the five essential practices, followed by "instructional rigor."
The instructional practices involved in both cases are notoriously challenging, especially for new teachers. Some teachers, however, were masterful. In their classrooms, students were asked to explain their thinking frequently, and began to do so without prompting. Students developed a sense of independence and self-regulation that enabled sustained focus during independent work in workstations.
SERP and DCPS used the analysis of data district-wide (including data on teacher evaluations across the five essential practices), as well as data consolidated from observations to date, to apply for a development grant in 2021.
2022
Funding for development work secured
The partnership team, through regular deliberations, fleshed out a plan to capture the practice of the successful teachers on video, and build and evaluate two professional learning courses for teachers: one focused on developing student discourse and one on effective literacy workstations. Each course would also illuminate the approaches taken by teachers to develop a classroom culture to support student independence. After the development phase, the courses would be evaluated in an RCT.
Read more
We proposed using existing DCPS data to measure changes in student outcomes, and the teacher evaluation data already collected by the district to capture changes in perceived teacher effectiveness. The proposed courses would be asynchronous so that they could be assigned to new teachers year after year without veteran teachers needing to sit through the content repeatedly. A coaching guide would be developed for the school-based instructional leaders to support their teachers, along with checklists of things to look for when they visit classrooms.
News of the IES grant award in the spring of 2022 allowed the partnership to move quickly to recruit six teachers for video recording so that Swivl cameras could be placed in their classrooms by the first day of school (helping to capture how culture begins to be established as children enter the classroom on day 1).
2022-2023
Video for both courses collected
In addition to the Swivl videos, teachers’ classrooms were professionally recorded for the entire literacy block at three times during the school year (fall, winter, spring). Interviews with the teachers were also recorded, providing an opportunity for teachers to explain why they did what they did and what considerations they were balancing in making instructional decisions.
Once the project was underway, DCPS realized that it would have been a good idea to include some teachers in bilingual classrooms. Two additional first grade bilingual teachers were added, and recording in their classrooms began in January.
2023-2024
Two professional learning courses and word work materials developed
Workstations: The SERP team combed through classroom video, drafted the
Effective Literacy Workstations
course content, and professionally recorded the instructional components of the course in the 2023-2024 school year. Feedback on the course draft was solicited from:
- a small group of teachers who piloted the draft sessions,
- district literacy leaders who participated in the monthly partnership meetings,
- district leaders who oversee teacher evaluations,
- project advisors Robert Pianta (University of Virginia), Lowry Hemphill (Boston University), Nell Duke (University of Michigan), Rhonda Bondie (Fordham University), and Cathy O’Connor (Boston University). Revisions were made continuously in response to feedback.
Lowry Hemphill suggested that the workstations course would be much more powerful if teachers had differentiated materials to work with, particularly for the Word Work station. From her extensive classroom experience, she was aware of how time-consuming it is for teachers to create these materials themselves, given the need to differentiate for students at multiple levels of mastery. The lift, especially for new teachers, is very heavy. Hemphill and Emily Hayden (SERP) drafted the workstation activities, which were produced by SERP staff (Matt Ellinger and Wafaa Ahmed).
Discourse: We combed through the classroom video a second time, this time looking for examples for
Developing Student Discourse. This course was developed in the second half of 2024 in preparation for implementation in January, 2025. While there was insufficient time for teachers to pilot the course in advance, all other sources of feedback for the workstations course were tapped again.
2024-2026
Randomized control trial underway
DCPS randomized all elementary schools to one of three conditions: Workstations, Discourse, or Control. The workstations course began in the fall of 2024-25 school year, and the discourse course began in the winter.
In the spring of 2025, schools were randomized again. Control schools were randomly assigned to one of the two courses. Schools previously assigned to one of the courses were randomly assigned to either the other course or the control group.
Implementation to date has highlighted a major challenge of RCTs. Because only a subset of schools are intended to participate in each course, and a subset of teachers within each school (teachers in their first 3 years of teaching) were required to take a course, communication between the district central office and the schools has been challenging. District leaders created opportunities for Zoom meetings between the SERP team and principals and coaches, but only a small proportion of each participated (with some reporting that they did not know about the meeting). Visits to schools revealed that many coaches and teachers considered the courses optional. While the district directed schools to participate according to the random assignment and offered professional development points for course completion, compensated time or substitute teachers had not been provided.
The RCT is still underway, with detailed data being collected on the number of sessions each teacher completes and the timing of session completion. Analysis will be conducted in 2027, when district data from 2026 are shared.