ICYMI: The Florida Charter Study is Baloney

This post has been modified to included more specific information regarding Dr. Jay Greene’s testimony during the Citizens for Strong School v BOE trial.

Full disclosure: my son attends a charter school that currently satisfies his specific needs, so I understand why some parents choose charter schools for their children. As a former board member, I also understand the limitations of charter schools. More importantly, perhaps, as a physician and scientist, I am skeptical of research that finds results beneficial to those funding the study.

During the recent Citizens for Strong Schools vs Board of Education trial, defense witness Dr. Jay Greene presented the results of a new Florida charter school study, which was a follow up to an original study by Booker, et al.  Dr. Greene claimed this new study showed that charter school students in Florida have better adult outcomes; earning higher salaries and completing more years of higher education. (You can watch his testimony regarding this at 1:15:00 here.)

Dr. Greene is the endowed Chair of the Department of Education Reform at the University of Arkansas. I don’t believe he disclosed that his position, and indeed his entire department, was endowed by a $10 million gift from the Walton Family, a family outspoken in its support of charter schools at the expense of traditional public schools, vouchers and in education “reform” in general. (You can read more about Dr. Greene’s association with the Walton Family here and here. You can decide whether his perspective is that of an unbiased researcher or not.)

During his testimony, Dr. Greene explained the Booker, et al, study looked at charter middle schoolers and compared outcomes based on whether they attended a traditional public high school or a charter high school (this group he called “lucky” to have a charter high school nearby). He called the study a “particularly good study because it is a comparison of apples to apples.” He suggested that its experimental design approximated the quality of a placebo controlled medical trial. He suggested that every child who attended a charter school in middle school would choose to attend a charter school high school, if they were lucky enough to live near one. Thus, in Dr. Greene’s opinion, comparisons of groups of charter school 8^th graders who chose to attend a charter rather than a traditional public school would be equivalent to a randomized controlled trial. He spent more than 10 minutes explaining what a high quality study this was and how it confirmed his belief that “charter schools tend to contribute to a high quality, efficient education system.”.

Dr. Greene’s mention of this new data  was of interest to me, as I had previously heard that most research had not been able to demonstrate significant benefits from charter school attendance.  The research I had seen appeared to confirm that, like all schools (public and private), some charter schools are good, some are particularly bad and most would be considered average.

For example, in 2009, charter schools in eight states, including Florida, were studied (read report here) “there is little evidence that charter schools, on average, are producing test-score impacts that are substantially better or worse than those of conventional public schools.” This, surprisingly, is the conclusion of the first Booker et al study that Dr. Greene discussed in his testimony.  The authors did see some small differences in some non-standardized test based outcomes and sought to pursue those ideas in the followup study.

In 2013, Stanford University’s Center for Research on Education Outcomes (CREDO) published the National Charter School Study, which looked at charter schools in 26 states and concluded that kids in most charter schools are doing worse or no better than students in traditional public schools. The study did show charters had made some improvements in the four years since the previous CREDO study. When looking specifically at Florida, the CREDO study showed students in traditional public schools, on average, read at a higher level than those in charter schools and did just as well in math. (Read more here).

Here at Accountabaloney, we have argued that test scores may not be the best measurement of a quality education (read more here and here), so I was intrigued by the report of this new charter school study that apparently showed education outcome measurements beyond mere test scores. Measuring quality education by a metric other than test scores would be a significant step in the right direction. I wanted to learn more about this study, so I turned to Google..

On April 11, 2016, the Hechinger Report reported that a new study in the Journal of Policy Analysis and Management (JPAM) showed “something startling: the charter students might not have produced higher test scores when they were in school, but years later, when they were in their mid-twenties, the charter school students earned more money, and were more likely to have attended at least two years of college (although still only half of them did so).

The JPAM study only looked at students who attended charter high schools and the authors suggested that, since the studied high schools were all small, the size of the high school (rather than whether it was charter or traditional) might be responsible for the positive results. The Hechinger Reort asked the question:

“Still, what’s causing kids to stay in college and later earn more money is a mystery. The authors wonder if these charter schools are doing more than providing better college counseling, and might be “particularly good at promoting skills such as grit, persistence, self-control, and conscientiousness.”

But there’s no data to prove that.”

A closer look at the study design suggests that there may be problems with the study. These are well documented by education blogger, Mercedes Schneider, in her April 10, 2016 blog, titled “Charter School Grads Earn More? Read On…”, which we have reproduced here in its entirety: