Approach

“Our theory of action begins with what we would argue are the prerequisites for the effectiveness of those partnerships.”

Guiding an ambitious and long-term mission

The SERP mission is ambitious and long term. It will take years before success in reaching the ultimate goal of an infrastructure at scale that can support continuous improvement and knowledge accumulation can be judged. However by mapping out the theory of action that will guide the development of the SERP processes and programs of work over time, we can mark short term progress toward the long term goals.

SERP Theory of Action

Prerequisites: (present from day 1)

  1. Commitment of top district leadership to the field site collaboration
  2. Focus on problem of importance to the district
  3. Ability to bring high quality knowledge resources to the table
  4. Ability to effectively coordinate and steer the work to maintain productivity
  5. Ability to “flatten the field” so that all sources of expertise are held in high regard and the culture is one of mutual respect.

⇓ Cause ⇓

Engagement: (years 1-3)

  • Contribution of time/effort by researchers, developers, practitioners
  • District openness regarding process, policy, data
  • District willingness to adopt research designs
  • Involvement of faculty, graduate students and practitioners in research on problems of practice
  • Willingness of other districts to replicate model

⇓ Cause ⇓

Production: (years 2-6)

  • Production of tools that a) incorporate multidisciplinary expertise, b) have high use value to schools
  • Production of new scholarship that is practice relevant and multi-disciplinary
  • Creation of research instruments and protocols that are widely shared and built upon over time
  • Generation of cross-district research; improved understanding of critical features of context that affect outcomes
  • Increased capacity in the research and practice communities to participate in high quality research in school contexts

⇓ Cause ⇓

Outcomes: (years 2-7)

  • Improvement in student achievement
  • Spread of innovation across districts
  • More systematic knowledge accumulation and use

Prerequisites.

In the SERP model, the collaborative partnerships among practitioners, researchers, and developers are the generators and incubators of innovation and improvement. Our theory of action begins with what we would argue are the prerequisites for the effectiveness of those partnerships. The first is the commitment of the district’s leadership at the highest level to the collaborative process. School district administrators have a great many demands on their time and attention, both internally and externally. And large, urban school districts typically have many requests from researchers to host studies for which the researchers have received grants. If SERP is seen as one more such project, the sustained engagement of administrators and teachers in the work, and the integration of that work into the fabric of district life, will not happen. Neither is it likely that the district will be willing to share detailed district data, or support rigorous research designs, both of which are critical to knowledge and tool generation. In order to ensure commitment, therefore, the partnerships will be created only in districts where the top leadership agrees in advance to give priority to the field site work.

The second prerequisite is that the problem of focus be one that is of high importance to the district. SERP and the school district serving as a field site do not have identical goals. While both share the goal of generating improvements in student achievement, it is in the interest of the district to achieve that end with as few new demands on its staff as possible. SERP, on the other hand, seeks to develop knowledge that meets high standards of evidence, and tools for improvement that are transportable to other districts. These goals place greater demands on staff than would improvement efforts for which research and transportability were not concerns. Eliciting district commitment to the SERP process despite the demands it imposes means that the value to the district must be high enough to warrant the costs of participation. Choosing a problem that the district has identified as one of high priority greatly enhances the chance that the district will consider the work of sufficient value to support in a sustained way.

Beyond the choice of topic, SERP will need to demonstrate value through the capacity to bring high level expertise to the table. This third prerequisite has foiled many efforts in the past to improve the quality and relevance of education research. The national laboratories, for example, have a stated mission that reads very much like that of SERP. But they have not been able to attract the nation’s leading researchers to their efforts. SERP may have greater access to this community because of its status as an offspring of the National Academy of Sciences. Bringing together an impressive group will not carry the effort far, however, if the members of the group do not feel that their time is well spent. The fourth prerequisite is therefore the ability of SERP to manage the field site work so that participants feel both engaged and productive.

Closely tied to the goal of managing the productivity of the group is the last prerequisite: creating a culture of mutual respect and genuine regard for the variety of sources of expertise involved in the work. The folklore in education is that research and practice partnerships fail because the cultures of the two groups differ dramatically, and neither is able to listen to, appreciate, and value the expertise and experience of the other. SERP must succeed in “flattening the field.” Each member’s views must be subject to the same scrutiny, with none holding a position of higher ground.

While each prerequisite individually can be addressed straightforwardly, addressing them as a group requires a nuanced balancing act. The requirements of research must be balanced against the demands on practice; the requirement that the group maintain a high level of productivity must be balanced against the demand that each member feel that his or her position has been heard and respected. Whether the prerequisites are in place, then, cannot be captured on a scale in which more is better. More attention to productivity is good only to the extent that it does not alienate group members, for example. And while more respect for the views of others is generally positive, if it means that the ability of the group to move forward is held hostage to unrestricted discussion, then the proper balance has not been struck.

Engagement.

Whether the prerequisites for effective collaboration are in place and are effectively balanced can be captured in the evidence of engagement among the field site partners. While the prerequisites must characterize the SERP work from the very start, engagement is likely to take considerable experience working as a group.[1] The level of engagement is reflected in its most obvious form through commitments of time, effort, and resources by researchers, developers, and practitioners. The quality of engagement, however, is more nuanced and trust dependent. Evidence resides in the willingness of district participants to open the policies, processes, and practices in the district to scrutiny. It also resides in the willingness to share data, and to support research designs that are effortful and require negotiation at both the district and school level.

Engagement on the part of researchers and developers is captured in the extent to which personal responsibility is taken for work products for which they are then held accountable to the group. Again, this requires not only a commitment of time, but a willingness to open one’s thinking and work to the scrutiny of a group whose members view that work through lenses shaped by very different sources of expertise.

Finally, if the collaborative process is well balanced, evidence of engagement should emerge in the form of interest on the part of an expanding group of others in both university and school settings in the work. Success of the model should also spur engagement by generating an interest in replicating the model elsewhere. Replication is critical to the ultimate goal of creating an infrastructure in which work can be replicated across sites.

Production.

According to the SERP theory of action, effective engagement of practitioners, researchers, and developers in problem solving R&D will lead to the creation of several kinds of distinctive products:

  • Instructional and management tools that incorporate multidisciplinary expertise and have high use value to schools;
  • The creation of research instruments and protocols that are widely shared and built upon over time;
  • New scholarship that is practice-relevant and multi-disciplinary;
  • The generation of cross-district research that advances the understanding of critical features of context that affect the response to interventions.

In addition to products, the SERP collaborations, by virtue of their requiring the participation of graduate students, more senior researchers, and practitioners in the work, should expand the capacity in both the research and practice communities to participate in high quality research and entrepreneurship in school contexts.

Each of these products has its own time scale. In the early stages of the work, the development of instructional tools and research instruments can get under way quickly once engagement happens. New scholarship will likely follow later, as results of the work become available. Cross-district research and capacity expansion of a significant magnitude will be longer still in coming, requiring the creation of multiple sites.

Outcomes.

The ultimate, long-run goals of SERP are student achievement gains, the spread of innovation across districts, and the systematic accumulation and use of knowledge in educational practice. To say that student achievement gains are a long-run goal does not mean that progress is not expected in the short-run. While getting partnerships formed, problems defined, and research and development under way is likely to require a minimum of two years in any district, we would expect to begin to see some evidence of increases in student achievement by the end of year two or three. But achievement gains are a long-run goal in the sense that generating immediate gains is of less interest than generating the capacity to engage in continuous improvement year after year.

If immediate gains of maximum magnitude in a single district were the target, a productive approach would be one of identifying a fully designed program likely to deliver the most powerful instructional experience to students in short order. With adoption and the necessary teacher training, results might be expected almost immediately. As a long run strategy, however, this approach is short sighted. The intense focus during the period of reform that brings about the change cannot be sustained over time. As the focus wanes, the culture of the schools and the beliefs of teachers reshape the intervention to create something that often resembles only remotely what was first introduced. And the intensive, full-program design that creates the gains initially prevents further spread of pieces of the innovation that may be of value in their own right.

With a goal of building the capacity for continuous improvement, the components of which can be transported across sites, the SERP strategy is substantially different. Verbs that characterize outside reform efforts, such as adopt or impose are replaced by verbs that characterize partnership work, such as build on, co-design, and adapt. Innovation in the partnership context does not assume a program will change a culture; the assumption instead is that the designers must “culture the change” so that it can become part of routine instructional and/or management practice, and remain so even after the initial push has subsided. And the transportability goal will result in the production of multiple tools to address specific challenges that can be adapted in different district contexts, rather than single programs that come as complete packages. Without question, the short term impact on student test scores in a single district are likely to be less dramatic in the SERP partnership model than in a more intensive and scripted reform model. We would argue, however, that the SERP model is likely to create gains that are more sustainable across time, and more likely to spread across districts. Evidence of such spread, as well as of more deliberate accumulation of knowledge, should grow during years 3-7. These, ultimately, will be the indicators on which SERP’s success can be judged.


[1] Shepard Kellam and Linda Chinnia reported to the SERP-BPS Core Group that the partnership between the Baltimore City Schools and the Center for Education and Prevention required between one and a half and two years of interaction before engagement occurred.