SERP – MSAN Field Site Partnership

The problems of focus in the MSAN work – minority student engagement and algebra achievement—are of high importance to the nation as a whole. Successfully educating students in mathematics is now understood to be critical to the competitiveness of the nation and the life prospects of individuals. There has been a nationwide push to have all students take the more challenging courses that were once reserved for the highest achieving students. And recent evidence suggests that when school districts insist that all students enter a rigorous high school mathematics track, the number of students who pass those courses increases. For example, Gamoran and Hannigan (2000) report that efforts in the New York City school system to enroll all students in college-preparatory mathematics at high school entry resulted in a rise in the percent of all students passing courses in academic math from 42% to 54%. And in the six communities where the College Board’s Equity 2000 program was introduced, even greater success was claimed. On the other hand, an alarmingly high percentage of students put in these more rigorous classes fail. In New York, the failure rate rose from 37% to 42%. The greatest success of Equity 2000 is in Prince George’s County, but even there a quarter of the ninth grade students failed algebra.

The failure rates in high level mathematics are not race-neutral. For example, in Evanston Township High School (ETHS), one of the districts involved in the MSAN work, the great majority (82%) of ninth grade white students were enrolled in honors algebra, or some level of geometry, advanced algebra, or trigonometry. Among the students who were in ninth grade pre-algebra or algebra (typically indicating lack of success in 8th grade algebra) 81% were minorities, though minority students represented only about 50% of the freshman class. These racial differences in enrollment are reflected in student achievement on standardized tests. For instance, ETHS’s current 9th-grade students took the EXPLORE test (part of the ACT sequence of tests) as eighth-graders. Differences by race and ethnicity on the mathematics subtest were large: while the white students’ average score on the mathematics subtest was 19.6 (25 is the highest score), the average score for African-American students was 13.0, and 15.0 was the average score for Latino students.

A declaration that all students will take higher level mathematics in high school is not enough. The critical national goal of preparing our students to be competitive in the labor market of the future clearly cannot be achieved without an investment in our capacity to support success in high level mathematics among the students we are now losing in large numbers. The creation of a SERP field site can broaden and deepen the MSAN work already under way by expanding the investment in research and development in several respects:

1. Broadening the participation of researchers in mathematics and assessment. Innovation in any field is fed by a competition of ideas. In university environments, the competition of ideas is the currency of exchange. But when the ideas of researchers are shaped into programs for school improvement — a step that is critical to create value for practice — the competition of ideas loses out to a competition of programs. SERP’s response to this dilemma is to pull together a group of people who take different approaches to solving the same instructional challenges in order to keep the competition of ideas alive among those designing and using programs. The MSAN work in mathematics is rooted in the nationally recognized work of Uri Treisman, and in the Agile Mind Curriculum developed by Treisman and his colleagues. SERP would bring other mathematics researchers and developers to the table who take different approaches to algebra instruction, but who are confronted by, and are struggling with, many of the same problems. This approach in Boston[1] has proven highly productive. Not only are the needs of participating districts well served by the range of ideas from which to draw, but the work of researchers and developers is advanced by the stimulation of the group interactions.

SERP would also bring people with particular expertise in assessment to the table. In order to make progress on the problem of supporting high achievement in algebra among a broader range of students, teachers must be supported with assessments that pinpoint the critical gaps in students’ knowledge that are likely to undermine their progress. Identifying these critical areas has been a problem of high importance in the MSAN work. Expanding the group to include additional people with deep expertise in algebra assessment will be an important contribution of the design group.

2. Broadening participation of experts on motivation and engagement. While the task of motivating adolescents to engage in academics is formidable, the research literature on how to accomplish this feat is not. The National Research Council report on the engagement of adolescents in schooling, titled Engaging Schools (2004), synthesizes much of what is known, but also confirms that this is an area in which research and development is in dire need of strengthening. The MSAN work under way, with the expertise of Josh Aronson, Uri Treisman, and Catherine Good, is addressing this urgent need. As in the case of algebra, SERP would broaden the expertise on motivation and engagement by inviting others to participate in the work with differing views or angles of approach. Again, the competition of ideas in a problem-solving context will be critical to moving the field forward.

3. Bringing additional research and evaluation capacity to the table. The goal of the SERP field site partnerships is to build the capacity to work deeply with districts in a sustained way. A prerequisite for partnership is sharing student level data. SERP would work with the field site districts to create a data base that will allow students to be tracked over time so that long run program effects can be captured, and questions can be answered regarding “for whom” and “in what contexts” an effect is seen.

SERP would also strengthen the MSAN work by engaging individuals with high level expertise in research design and program evaluation. While addressing the problems of algebra instruction and student engagement requires the participation of people with deep expertise in these subjects, these are rarely the same people who have deep expertise in the analysis of program impact. This combination of expertise is critical to improving the design of programs over time, as well as to generating independent documentation of program effects.

4. Bringing additional expertise on school organization and implementation. A basic reality of all instructional interventions designed to increase student learning is that their implementation typically varies widely from classroom-to-classroom, from school-to-school, and from district-to-district. The possible sources of variability in implementation of instructional innovations are: (1) features of the innovation itself—its complexity, the level of knowledge and skill required by teachers and supervisors, and the gap between existing practice and the new practices required by the innovation; (2) features of the organizational setting in which the innovation is implemented—the capacity of the organization to monitor and supervise instructional practice, the ability of the organization to generate data on practice and learning, and the ability of the organization to create opportunities for teachers and administrators to work jointly on instructional improvement; and (3) external supports — the ability of sponsors of the innovation or district personnel to provide supports to practitioners in schools — professional development, coaching, networking — that is consistent with the practices of the innovation.

The literature on instructional improvement acknowledges the importance of these factors in influencing the degree of implementation of innovations, but it does not discriminate in any systematic way among them. Hence, the literature is not very useful to practitioners, whose main interest is not in observing that interventions vary in their implementation, but how to create conditions in which interventions work more consistently across settings. (See, e.g., Cook, et al. 1999; Datnow, et al. 2000; Elmore 2004; Slavin, et al. 1996; for a view that focuses on curriculum content and usability, see, Cohen and Hill 2002). In all SERP field sites, the school organization and implementation issues will have a critical place on the agenda. SERP will bring expertise on these issues to the table, and will draw on the tool development under way in other SERP districts for use in the MSAN work.

The MSAN districts are attractive as a SERP field site partner for multiple reasons. First, the initiative of the founding districts in forming the network points to an unusual level of commitment to incorporating research into practice. The district superintendents form the MSAN board of directors, symbolizing the support for using research at the highest level. Second, the MSAN philosophy and goals are closely aligned to those of SERP. Laura Cooper, one of MSAN’s founding leaders, was a member of the National Academy’s SERP committee, and she and Sharyn Stumpf (an MSAN participant from Madison) are members of the Design Team in Boston. Third, the MSAN research and development initiative currently under way will jump start the field site work significantly. Fourth, in order to study the influence of district context on the success of interventions—a problem that has reduced the spread of innovation in education to a crawl—SERP will need an infrastructure of field sites that represent a range of district contexts. The inner-ring suburban schools involved in MSAN offer a desirable contrast to the more urban settings of Boston and San Francisco. And finally, while only 3 or 4 of the MSAN school districts can be sites for active research, the network of 25 schools will be both a resource and an audience for the field site work, articulating needs, offering expertise, and making use of the products and knowledge generated by the work.

Plan of Work

A planning grant is sought to lay the groundwork for a SERP-MSAN field site focused on the problems already identified by the MSAN superintendents: the achievement gap in algebra learning, and the engagement of minority students in academic high achievement. The work will involve five separate tasks:

Task 1. Identify the MSAN districts that will constitute the field site. Among the 25 MSAN districts, three or four will be chosen as field site districts. One of these will be Evanston Township, but the others are not yet certain. Candidate schools will be evaluated for participation according to the following criteria: 1) Is the district’s leadership committed to the field site model? Evidence of commitment will include an agreement regarding the membership of the Core Group that will guide the work, and the willingness of these group members to participate actively in monthly meetings once the partnership is under way. Before the district is considered a strong candidate for a partnership, a meeting will be held that includes the members of the prospective core team to confirm their willingness to make the necessary commitments. 2) Is the district open to data sharing for research purposes, and to expanding the collection of data as needed? Willingness to have researchers use data on students, teachers, and schools under conditions that protect student and teacher privacy will be critical to effective research. If a district is unwilling to provide the data, it cannot be an effective partner in research and development. The SERP director will also inquire about the history of research relationships in the district in order to determine whether data access has been an area of high sensitivity for the district. 3) Is the district willing to commit to rigorous research designs? Much can be learned from mining data already in a district’s computer files, or by collecting data that can easily be generated as teachers and schools go about their everyday work. However when a set of practices or programs appears very promising, testing the impact through the use of a rigorous research design becomes critical. An effective partnership anchored in research and development requires that the district be willing to support such research designs. This issue will be discussed openly as a condition for partnership. 4) Is the district able to negotiate its political environment successfully? The operation of a field site can bring stability to a school district because the agenda for systematically examining and improving practice, even though anchored at the highest level of decision-making in the district, stretches through middle management and into individual schools. The change of a single leader therefore has less of a destabilizing effect. But the history of education improvement efforts is littered with promising initiatives that were tossed aside with a change in leadership. To ensure that the partnership effectively takes root in the district, it is helpful to have the support of the critical political players beyond the superintendent. The balance of power will vary from district to district, as will the alignment between the district superintendent, the school board, and the mayor. The ability of the superintendent to elicit the support of key political actors for the field site establishment will be explored. The memorandum of understanding written for SERP and the Boston Public Schools appears in Appendix C. A comparable memorandum will be required as an agreement between SERP and any other school district that enters into a field site partnership.

Task 2: Identify key research team and design team participants. The MSAN work already under way jumpstarts the collaborative process in that the districts’ priority issue has been identified, and several key research participants are already engaged in the work. During the grant period SERP, in collaboration with Dana Center and MSAN colleagues, will conduct the background research to identify potential members of a design group, interview them to ascertain the likelihood that they will be productive group participants, and invite the best candidates to the first two meetings of the design team.

Task 3: Resolve logistical challenges. The SERP-MSAN field site will have multiple district members, which will introduce a set of logistical challenges not present in single district sites. These challenges, such as organizing the regular core group meetings across multiple locations, will need to be resolved during the planning period. It will be particularly important to identify a research home for the work given the multi-district focus. The University of Wisconsin’s Center for Education Research is a candidate site.

Task 4: Convene the Core Group and the Design Team to map out a plan of work. During the grant period, two meetings of the Core Group and the Design Team will be held to chart a direction for the partnership work in the first two years. These meetings will be highly supported, both in the preparation stage –laying the groundwork for productive interactions—and in the follow-up period—using the discussion to chart a course for future work.

Task 5: Write a proposal. Drawing on the discussion with Core Group and Design Team members, and on the SERP experience in Boston, a proposal for the first two years of work in the MSAN field site will be prepared for consideration by the Board of the Goldman Sachs Foundation.

Project Team: The project team will be drawn from three organizations: MSAN, SERP, and the Dana Center.

• MSAN will take responsibility for the initial exploration of interest among its network of schools, and will arrange for meetings with the school leadership. MSAN will collaborate with SERP in working out a structure for consultations of the Core Group. MSAN participants will play an active role in the design group meeting, presenting the work to date with their Data Center partners which will serve as the jumping off point for the field site collaboration. They will also contribute to the proposal preparation.
• SERP will orient interested districts to its mission, and to the structure and operation of the field sites. The SERP leadership will negotiate the commitments with the districts that it requires for participation. In consultation with MSAN and the Dana Center, SERP will take the lead in identifying and recruiting design team members, and in finding a “home base” for the field site research. SERP will convene the design team meeting, and shape the meeting discussion into an agenda for future field site work. SERP will also take primary responsibility for preparation of a two-year proposal for field site research, development, and implementation.
• The Dana Center will work with SERP to ensure that the work in progress is well integrated into the SERP planning. Like MSAN, it will play a critical role in the design team meeting, and will collaborate in preparation of the proposal.