Welcome

This week’s CommenTerry is an excerpt from my new report, "School Vouchers: From Friedman to the Finish Line." 

Representatives Jonathan Jordan, Marcus Brandon, Bert Jones, and Paul "Skip" Stam have filed the session’s first voucher bill, House Bill 269: Children with Disabilities Scholarship Grants.  As state lawmakers begin to debate this bill (and possibly others), it is critical that they are familiar with the more than 15 years of research on voucher programs in the United States.

Bulletin Board

  • Attend. A list of upcoming events sponsored by the John Locke Foundation can be found at the bottom of this newsletter, as well as here.  We look forward to seeing you!
  • Donate. If you find this newsletter mildly informative or entertaining, please consider making a tax-deductible contribution to the John Locke Foundation.  The John Locke Foundation is a 501(c)3 not-for-profit organization that does not accept nor seek government funding. It relies solely on the generous support of individuals, corporations, small businesses, and foundations.
  • Share. The North Carolina History Project seeks contributors to the North Carolina History Project Encyclopedia. Please contact Dr. Troy Kickler for additional information.
  • Discuss. I would like to invite all readers to submit brief announcements, personal insights, anecdotes, concerns, and observations about the state of education in North Carolina.  I will publish selected submissions in future editions of the newsletter. Requests for anonymity will be honored. For additional information or to send a submission, email Terry at [email protected].
  • Revisit. We have archived all research newsletters on our website.  Access the archive here.

CommenTerry

While there is disagreement about the regulatory, political, and legal dynamics of voucher programs, there is a general consensus that school choice has a positive academic impact on participating students.

According to the Friedman Foundation for Educational Choice, ten empirical studies of scholarship programs have used random assignment or experimental research design, the "gold standard" in social science research. Nine of those studies, including two that evaluated Charlotte’s Children’s Scholarship Fund, concluded that scholarship recipients had statistically significant increases in performance (See Facts and Stats below). One yielded inconclusive results.  In addition, more than twenty studies identified ways that scholarships delivered "spillover" benefits to traditional public schools. 

As mentioned above, the Children’s Scholarship Fund – Charlotte (CSF-C) has received considerable attention from the educational research community. The CSF-C, a private scholarship program started in 1999, awards scholarships to Mecklenburg County families that qualify for the Federal Free and Reduced Price Lunch Program. Currently, the maximum award is $2,500 per child and the program funds scholarships for approximately 400 children each year. To date, the program has awarded over $7 million in private school scholarships to over 5,000 low-income elementary school students.

Students are selected for the program by a random lottery administered by the Foundation For The Carolinas. A lottery functions as a random assignment mechanism, thereby generating randomly assigned experimental (scholarship recipients) and control (applicants who do not receive scholarships) groups. By comparing the two groups, researchers were able to determine whether there was a treatment effect free from statistical bias. In other words, random assignment permitted researchers to determine whether private schools, and not a variety of external factors, produced academic and behavioral gains that outpaced those of their public school counterparts.

In an August 2000 evaluation of the Charlotte Children’s Scholarship Fund, Jay Greene found that low-income, predominantly African-American, scholarship recipients had combined reading and math scores six percentile points higher than the control group after only one year of schooling. In a follow-up study published in the November 2007 issue of Policy Studies Journal, Joshua Cowen found comparable gains for recipients of scholarships from the CSF-C.

Michael Patrick Wille’s study of two large-scale choice programs, the Milwaukee Parental Choice Program and CSF-C, concluded that choice students in the publicly-run Milwaukee program had greater academic gains than those in the privately-run Charlotte program. Wille warned, however, that data collection issues likely compromised his analysis. Specifically, he was only able to obtain one year of data for Children’s Scholarship Fund students compared to five years of data for the Milwaukee Parental Choice Program.

Qualitative studies of the Charlotte Children’s Scholarship Fund conclude that parental satisfaction is high and most scholarship recipients have graduated from high school. One parent survey found that 96 percent of participating families rated their experiences with CSF-C as "excellent." In addition, 70 percent of families rated their schools of choice "above average" or "outstanding," while only 37 percent awarded their former schools similar ratings. Most importantly, over 90 percent of parents reported that their chosen private schools improved the academic performance, social skills, behavior, self-esteem, and overall engagement of their children.

Those findings mirrored ones reported in a Harvard University study of Children’s Scholarship Fund families nationwide.  In "Children’s Scholarship Fund Charlotte Recipients Tracking Study 2011," Dr. Alex Schuh of FRONTIER 21 Education Solutions examined graduation rates among scholarship recipients. He found that 181 of 186 (97%) CSF-C recipients eligible for graduation in the spring of 2011 had graduated. Unfortunately, the sample was only a fraction of the CSF-C students eligible to graduate in 2011. On the other hand, the rate was consistent with studies of Children’s Scholarship Fund programs in Ohio and Pennsylvania.

Regrettably, the scholarship is meeting less than half of the tuition required to attend a private school in Mecklenburg County. During the 2011-2012 school year, the average scholarship was just under $2,000 per child, and the average out-of-pocket payment for participating families was $2,633. For low-income families, this out-of- pocket payment was not trivial. Participating families had an average gross income of $28,853, so the average family committed nine percent of their gross income to tuition payments.

There are conceptual pitfalls associated with using evaluations of a privately-run scholarship program to speculate about the potential success of a publicly-run voucher initiative. Private programs are subject to different kinds of budgetary, political, and regulatory pressures than their public counterparts typically encounter. On the other hand, the success of Children’s Scholarship Fund should encourage lawmakers to replicate elements of similar private voucher programs. At the same time, they must acknowledge that the resultant public scholarship program is not (and can never be) a direct replication or reflection of private models.

(Note: References are available here.)

Facts and Stats

From Gold Standard Studies: Evaluating School Choice Programs by the Friedman Foundation for Educational Choice:

  • Patrick Wolf, Babette Gutmann, Michael Puma, Brian Kisida, Lou Rizzo, and Nada Eissa, "Evaluation of the D.C. Opportunity Scholarship Program: Final Report," U.S. Department of Education, June 2010.  Washington D.C. — The students offered vouchers graduated from high school at a rate 12 percentage points higher (82 percent) than students in the control group (70 percent), an impact that was statistically significant at the highest level. Students in 3 of 6 subgroups tested showed significant reading gains due to the voucher offer after 4 or more years. Overall, reading and math gains from the program were not statistically significant at the 95 percent confidence level, but overall reading gains were significant at the 90 percent confidence level. Parents remained more satisfied with their child’s school and viewed it as safer if offered a voucher, even though students had similar views of school satisfaction and safety whether in the treatment or control group.
  • Joshua Cowen, "School Choice as a Latent Variable: Estimating ‘Complier Average Causal Effect’ of Vouchers in Charlotte," Policy Studies Journal, November 2007.  Charlotte, NC — After one year, voucher students had reading scores 8 percentile points higher than the control group and math scores 7 points higher.
  • Alan Krueger and Pei Zhu, "Another Look at the New York City School Voucher Experiment,"American Behavioral Scientist, January 2004.  New York, NY — The voucher students had higher scores, but the results did not achieve statistical significance. Subsequent analysis has demonstrated that this occurred because the study used inappropriate research methods that violate the norms of the scientific community; if legitimate methods are used, the positive results for vouchers become significant
  • John Barnard, Constantine Frangakis, Jennifer Hill, and Donald Rubin, "Principal Stratification Approach to Broken Randomized Experiments: A Case Study of School Choice Vouchers in New York City,"Journal of the American Statistical Association, June 2003.  New York, NY — After one year, voucher students had math scores 5 percentile points higher than the control group.
  • William Howell and Paul Peterson, The Education Gap, Brookings Institution, 2002.  Washington D.C. — After two years, black voucher students had combined reading and math scores 9 percentile points higher than the control group.
  • William Howell and Paul Peterson, The Education Gap, Brookings Institution, 2002.  Dayton, OH — After two years, black voucher students had combined reading and math scores 6.5 percentile points higher than the control group.
  • William Howell and Paul Peterson, The Education Gap, Brookings Institution, 2002.  New York, NY — After three years, black voucher students had combined reading and math scores 9 percentile points higher than the control group.
  • Jay Greene, "Vouchers in Charlotte," Education Next, Summer 2001.  Charlotte, NC — After one year, voucher students had combined reading and math scores 6 percentile points higher than the control group.
  • Cecilia Rouse, "Private School Vouchers and Student Achievement," Quarterly Journal of Economics, May 1998. Milwaukee, WI — After four years, voucher students had math scores 8 NCE points higher than the control group. NCE points are similar to percentile points.
  • Jay Greene, Paul Peterson and Jiangtao Du, "School Choice in Milwaukee: A Randomized Experiment," in Learning From School Choice, eds. Paul Peterson and Bryan Hassel, Brookings Institution, 1998.  Milwaukee, WI — After four years, voucher students had reading scores 6 Normal Curve Equivalent (NCE) points higher than the control group, and math scores 11 points higher. NCE points are similar to percentile points.

Education Acronym of the Week

CSF-C — Children’s Scholarship Fund – Charlotte

Quote of the Week

"Impressively, nine out of 10 random-assignment studies, considered the gold standard of scientific research, show statistically significant academic benefits for students using scholarships to attend private school. Additionally, another 20 studies show public schools improve when faced with competition from private schools."

– The Friedman Foundation for Educational Choice, Research Overview

Click here for the Education Update archive.