As we approach the third year of the Public Schools Accountability Act (PSAA), it appears that one of its components is in need of a major overhaul. The PSAA requires each site to select an external evaluator who in turn is to create an action plan. In response to concerns from teachers and parents about many of these plans, CTA arranged for a study to determine the overall quality of these plans and the utility of the guidelines under which they have been developed.
The Evaluation Group for the Analysis of Data at the University of Arizona was commissioned by CTA to examine a stratified random sample of action plans for elementary, middle and high schools. Each plan was reviewed by a panel of experts and rated against 20 criteria. These criteria include the extent to which the plan reflects the state of the art in evaluation and action plans; reflects the California Academic Content Standards; and addresses barriers to improvements in student achievement.
"One overarching and disheartening conclusion to be drawn from review of the action plans is that most of them are doomed to fail," write the authors of the study.
Following are some of the findings:
- The action plans convey a sense of responding by tinkering with - or tweaking - the practices of schools.
- None of the action plans explicitly recognize the existence of a model similar to it but successful in its academic and social goals.
- There was a general inadequacy of needs assessment on which the plans were based.
- None of the plans indicate how data would be collected, analyzed, reported or used for program adjustments.
The report makes an invaluable distinction between the problem-focused approach that characterizes the action plans and strategy-based solutions.
"The most striking observation regarding the action plans is that in general, they seem to lack an overall model or strategy for school reform. By an overall model or guiding strategy, we mean an orderly as well as systemic approach to the design of an action plan. The lack of a guiding model is evidenced by a general tendency to identify scattered, narrowly focused techniques rather than a broader systemic approach to changing student performance and achievement."
Two examples illustrate this distinction:
Problem #1: Poor reading performance.
The problem-focused answer: New reading program.
The strategy-based answer: 1. Determine which children are not reading well. 2. Identify specifc reasons for poor reading. 3. Design specific interventions (for example, increase reading at home; obtain temporary tutor).
Problem #2: Low parental involvement.
The problem-focused answer: Create a PTO.
The strategy-based answer: 1. Determine why parents are not involved and develop alternative ways of eliciting involvement. 2. Determine in just what way low parental involvement affects children and the school and devise alternative ways of providing whatever input is required.
Millions of dollars are being spent on a program that, at best, will yield narrow short-lived results. As Robert Lin argued in his landmark report of a year ago, test scores always tend to rise in the first few years of reform, and then the scores plateau. Indeed, test scores have been rising in California and will continue to rise a little longer. However, the systemic and profound changes needed for sustained improvement - for children to be truly educated - are still far off. The findings in this study indicate clearly that the process for developing action plans, indeed the action plans themselves, must be overhauled.
An abridged version of this study can be found on the CTA Web site [www.cta.org]. It contains recommendations for changes in structure and content that will make action plans much more powerful vehicles for improving public schools.
Justo Robles
Robles is manager of CTA's SB 395 Program.
