Smart Start, Gov. Jim Hunt’s early childhood development program, was created in 1993 to help get North Carolina preschoolers ready to learn and thus boost educational performance. It was been a successful program from the standpoint of gaining national attention, but until recently its effectiveness as a school-readiness intervention was impossible to estimate. This has not stopped some politicians from proclaiming Smart Start “successful.” Earlier this year, for example, Hunt attributed apparent gains in North Carolina reading scores on 1998 national tests to the existence of Smart Start, even though the highest grade that students from Smart-Start supported preschools could have reached by 1998 was third grade and the reading tests was of fourth-graders.1

Since early 1998, however, at least four studies have been published of Smart Start’s impact on school readiness that can provide fair-minded observers with critical information about the program’s effectiveness. Although portrayed by news reports and elsewhere as proving Smart Start’s success, the studies, when closely read, suggest a far different conclusion. Indeed, the most recent and most comprehensive study published in September by the Frank Porter Graham Child Development Center at UNC-Chapel Hill found that the vast majority of Smart Start expenditures had no statistically significant effect on participants’ readiness to learn in kindergarten.

A Closer Look

For the new study, researchers picked six counties where there were active Smart Start partnerships and recruited an experimental group of 214 children who had attended a Smart Start-supported child care center as well as a control group of 294 children who had attended child care centers not involved in Smart Start. The researchers then used four measures of readiness to learn: the Kindergarten Teacher Checklist (KTC), the Peabody Picture Vocabulary Test, the Social Skills Rating Scale, and the Social Skills Rating Scale for Problem Behaviors.2 There were a number of methodological concerns here, and the authors properly cited them. The largest concern was selection bias; children were not randomly assigned to centers with and without Smart Start involvement, thus raising the question of whether subsequent differences were due to Smart Start or to the skills and motivation of parents making these choices.

Nevertheless, taken as face value, this study deals a serious blow to the program’s perceived effectiveness. The researchers found no statistically significant differences between children who had attended Smart Start-participating centers and those who didn’t. The researchers did find a statistically significant, but very small, improvement in one of the four measures for a subgroup of Smart Start participants who attended centers where Smart Start had made expenditures in direct quality improvement. The study also found that, on three of the four measures, the percentage of very low performers decreased for the subgroup vs. the control group. Smart Start’s administrators and media reports on the study played up these last results, ignoring the main finding. That was a mistake. As the authors note, 75 percent of Smart Start expenditures fund activities not directly related to child care quality improvement.3 That means that three-fourths of Smart Start funds have been spent in ways that have been shown not to improve readiness to learn, according to the study.

Furthermore, even for the subgroup of Smart Start children receiving the most attention, gains that were statistically significant were simultaneously so small in magnitude that, based on past experience with preschool intervention programs such as Head Start, they are unlikely to last beyond first or second grade. For example, in the one measurement (the KTC) where the average score for the Smart Start subgroup was truly higher, the gain was only 0.2 points on a 1-to-5 scale (4.4 for the Smart Start Subgroup vs. 4.2 for the control). This is a 5 percentage point gain. As previous studies of Head Start and other preschool programs demonstrate, only very large differences in readiness to learn in kindergarten are likely to persist throughout a child’s academic career. Most short-term gains from Head Start, for example, disappear by second grade as the performance of students from similarly disadvantaged backgrounds tend to converge regardless of whether they attended Head Start.4 Absent any evidence to the contrary and no studies of Smart Start to date have tracked students into their first or second grades it would be reasonable to conclude that, based on past research, the few Smart Start recipients who have actually increased their readiness to learn will lose this edge over their peers as they progress through school.

Previous research points to similar conclusions. In 1998, the Graham Center conducted a study of Smart Start in Orange County. A flawed study, since it lacked a true control goup, it still found no statistically significant impact on non-poor children and a modest gain for poor ones.5 A more valid study was conducted in 1998 in Mecklenburg. It found no statistically significant impact on those who spent a single year in a Smart Start center. It did find a measurable gain for kids who stayed in such a center for three years, but once again, the gain was relatively small 2 percentage points on one measure and 7 percentage points on another.6 Finally, in early 1999 the Graham Center released a study of 200 Smart Start-supported centers that showed gains in child care quality butno impact on kindergarten readiness.7

Conclusions for Policy Makers

These four studies of Smart Start outcomes, taken together, provide a rationale for reforming the program as follows:

  1. Direct Smart Start expenditures should target at-risk children. The 1999 study suggests that, if Smart Start has a measurable impact, it is largely confined to those children who are significantly at-risk for problem behaviors or poor preparation. The 1998 Orange County study further identifies poor children as benefiting, albeit modestly, from Smart Start while non-poor children did not. These findings suggest that redirecting scarce resources to poor children, perhaps through the expansion of existing means-tested day-care vouchers, would be a far better policy than Smart Start’s current blunderbuss approach, which lacks focus and accountability for results.

  2. Direct relief for families is better than indirect subsidies through Smart Start. As stated earlier, the vast majority of Smart Start expenditures for items such as new playground equipment at day care centers and health screenings do not appear to have a measurable impact on readiness to learn. If, as the study authors suggest, the goal of Smart Start should be expanded to include general improvements in child well-being rather than just readiness to learn, the most efficient policy tool to achieve that would be to convert most Smart Start dollars into an expanded Smart Start Tax Credit for families with preschool children, as the Locke Foundation has previously recommended. In its 1999-2001 alternative budget, for example, Locke proposed a $250 refundable tax credit for each preschool child. Converting existing Smart Start dollars to this purpose would provide significant tax relief to many North Carolina families while still allowing more than $100 million in current funding to be used to increase day care vouchers for low-income children.

John Hood, President

 

Notes

  1. Hunt quoted in “Reading Focus Delivers Results,” News Release, N.C. Dept. of Public Instruction, March 4, 1999, p. 1.
  2. Kelly Maxwell, Donna Bryant, and Shari Miller-Johnson, “A Six-County Study of the Effects of Smart Start Child Care on Kindergarten Entry Skills,” Frank Porter Graham Child Development Center, UNC-Chapel Hill, September 1999, pp. 6-9.
  3. Ibid., pp 1, 10-13.
  4. Ruth Hubbell McKey, et. al., “The Impact of Head Start on Children, Families, and Communities: Head Start Synthesis Project,” Executive Summary, U.S. Department of Health and Human Services, June 1985.
  5. Maxwell, et. al., “The Effects of Smart Start Child Care on Kindergarten Entry Skills,” FPG Child Development Center, UNC Chapel Hill, June 1998.
  6. Bruce Yelton and Amy Whitcher, “Child Care Experiences and Kindergarten Achievement: Assessing the Impact of Level of Care and Program Improvement Efforts,” Charlotte Mecklenburg Schools, May 1998.
  7. Stephanie Hawco, “UNC Study Suggests Smart Start May Not Be Making the Grade,” WRAL-Online, February 23, 1999.