 |
Download: A Full Report on the Readability of the School Accountability Report Card (SARC) (PDF 752 KB, 22 Pages) |
| Download: A Summary Report on the Readability of the School Accountability Report Card (SARC) (PDF 483 KB, 3 Pages) |
Grading the School Accountability Report Card:
A Summary
Neil Peretz, Andrea Luquetta, Gabriel Baca and Gary Blasi*
Introduction
The School Accountability Report Card is
intended to be an important part of the educational
accountability system in California. A School Accountability Report Card (SARC) is prepared for each school in California to inform parents and community members about how well a school is doing. Each school district is required by state and
federal law to produce a SARC for all the schools in
the district. The State provides a template for what
the SARC should contain.
Over the past few months, faculty and students at the
UCLA School of Law have investigated how well
the School Accountability Report Card serves the
goal of informing parents and community members
about their schools. Our research question has been
whether the SARC can be understood by parents
and community members. The attached report
documents our findings in detail. Put simply: The
SARC is failing.
Methods
The investigation tested the understandability
of the SARC format published by the California
Department of Education. This format is used by
the largest school district in the State and many
others. We assessed the SARC through three
different approaches. First, we analyzed the SARC
with five proven computerized readability analysis
programs. Second, we conducted extensive in depth
focus groups with parents to evaluate their
understanding of the SARC. Finally, we presented
the SARC to well-educated, civic-minded citizens
to assess how well the SARC enabled them to make
factual judgments about schools. By all measures,
the SARC fared poorly.
Readability Measures:
The 17th Grade?
Using commercial readability analysis software,
we analyzed key sections of the SARC** on the five
most established measures of “readability.” These
computer program produce an estimate of the
reading grade level required to understand a piece
of text. We then ran the same tests on a sample of
documents we thought most people would agree are
not particularly easy to understand:
• A VIOXX Patient Information form from Merck
• Proposition 98 itself (which created the SARC)
• A form Lease Agreement for Month-to-Month Tenancy
• The Microsoft Windows XP Software Driver Installation Instructions
• The IRS Instructions for Form 6251 Alternative Minimum Tax
• The IRS Form 1040A Instructions
The median readability test score of the SARC
indicate that the SARC requires 17.2 years of
education to be properly understood, more than
ANY of the comparison documents listed above.
Comprehensibility:
Can this Many Rotarians be Wrong?
It is not enough, or course, to be able to read
a document. One must be able to make accurate
judgments about the information in the document.
For this part of our study, we obtained the help of
45 members of two Rotary Clubs in Ventura County
and Los Angeles County. Rotary Club members are
a cross-section of business and professional people
who tend to be well-educated and actively involved
in their communities. In addition to asking them about their impressions of the SARC, we asked them
some objective questions that could be answered
based on information contained in the SARC. The
question areas and the performance of this group
are set out below:
• Whether students were scoring higher or lower than the national average: About 1/3 were unable to determine the correct answer
• Whether the school was fully staffed and whether teacher credentials were improving: Nearly 2/3 were unable to determine the correct answer
• How many students were taking college prep courses: 80% of these subjects were unable to determine the correct answer
Given their level of education and strong civic
interest, we think these Rotarians should be better
than average at understanding public documents.
It is obviously the documents themselves that are
failing to communicate accurate information.
Do Parents find the SARC Useful and Understandable?
We conducted two focus groups at with parents at
UCLA to get a more nuanced view of how people
understand the SARC. These parents were diverse
and included professors, administrators, students,
and janitors and other campus workers. Participants
provided detailed assessments of the SARC. One
parent of a middle school child described the
experience by saying, “I can understand the words
and the numbers but it’s not making any sense to
me.” Another parent, who is both a law student
and former teacher, said, “I taught for years and I
still don’t understand this.” All the focus group
participants thought the information on the SARC
was important, although few had been aware that
such report cards existed.
For many parents and community members, the
readability of the SARC is not relevant, because
they are not literate in English. Census data
indicates that fully 27% of households in California
are “linguistically isolated,” meaning that there is
no individual in the household able to speak English
very well. For these families, translating the SARC
into accessible English is only a first step.
Recommendations
The School Accountability Report Card has great
potential for informing parents and communities
about how well their schools are performing. That
potential is not being realized because there are
flaws in how it is written and presented. While
these preliminary findings are drawn from the
observations of a fairly small sample, it is clear that
even well-educated, highly motivated people have
great difficulty making sense of the SARC. Given
the importance of the information on the SARC and
the taxpayer resources already invested in collecting
and disseminating this information, we recommend
that:
1. The State should draw upon outside expertise to assess the comprehensibility of the SARC.
2. Based on that detailed study and on recommendations from experts in preparing such documents, the State should design, test, and publish a more comprehensible SARC template for use by school districts.
3. The State should require that the SARC be translated into languages used by significant groups within each school district’s population.
(notes)
* This summary was prepared by Professor Gary Blasi, UCLA
School of Law, and summarizes a detailed report, Grading
the Report Card: A Report on the Readability of the School
Accountability Report Card (SARC), prepared by UCLA law d
students Neil Peretz and Andrea Luquetta and Gabriel Baca,
doctoral candidate in the UCLA Graduate School of Education and
Information Sciences.
** We tested the portions of the SARC parents had identified in surveys as particularly important: (1) Teacher and Staff Credentials, (2) Post-Secondary Education Preparation, and (3) Standardized Test Scores.