Do Charter Schools matter?
Dec 1st, 2009 by David Anderson
A new Stanford study will cast doubt in some minds about the effectiveness of Charter Schools. The study had novel methodology and was very complicated. It compared individual charter school students with group averages of traditonal public school students. It did not compare the individual student’s gain or loss from the year before, but compared the individual student to the average of a like group of students from a previous school. It made no value per dollar provision. It made no evaluation of school safety or any other criteria beyond test scores. It made no comparisons between the level of education before charter schools in those states and afterward. The study has invalid methodolgy upon which to make any conclusions about charter school effectiveness. None the less, I think the study had useful information which can help policy makers and frankly charter school leaders.
The study’s authors stated their is a lot of strength shown in charter schools. 1/6 were better than tradtional public schools and most were equal. The problem is that 1/3 underperformed.
My first problem with the study is that the study’s methodology for measuring charter school student scores versus traditional public schools is not consistent. It is best explained by this memo abstract.
A recent study of charter schools’ effect on student achievement has been published by
CREDO (2009). It contains a serious statistical mistake that causes a negative bias in its
estimate of how charter schools affect achievement. This paper explains that mistake.
Essentially, the achievement of charter school students is measured with much more error
than the achievement of the controls, which are not individual students but are group
averages of students in the traditional public schools. By using the achievement data as both
the dependent variable and (lagged) an independent variable, the CREDO study forces the
estimated effect of charter schools to be more negative than it actually is. This paper notes
that the CREDO study violates four rules for the empirically sound use of matching methods
to evaluate charter schools’ effects. The main conclusion is that the CREDO study is not
reliable, most obviously because the statistical mistake means that its estimates of the
charter school effect are substantially biased downwards from the truth.
The second problem that I have with the study is its advocacy of closing charter schools which do not perform to the same level as the average traditional public school. My first inclination would be to say let’s cut the funding level of the traditional public schools to same level that charter schools have then shut down all of the schools traditional and charter that don’t make it. No, I am not serious. I have no problem with allowing failing schools to close. I do have a problem with making one measure the only criteria for determination. The market should determine if the school closes. If it under-performs expectations, it will have to make changes. I am all for an academic review process. I had that experience with a charter school that my children were attending (one still does). While they did well, scores overall declined. I was privileged to become part of a new board. I saw first hand that charters can change very quickly. Within two years the test scores were almost triple the percentage passing. The advantage of charters is that they can change. If people still want to attend, that should be the first option not closure. If you want charters to experiment, then you can not run to close them the first time something does not work. That will not serve the teachers, parents, or students in the long run.
It also does not highlight the fact that a percentage of the students are in charters because they were not doing well in traditional schools. The study showed distinctions between the first year charter students and those who stayed in the schools. That did not make the press release. I hope that policy makers actually read the study.
The study has other elements which give hope to choice advocates. The charter schools that did well were in states which allowed more of them. In states with caps, the charter schools underperformed. The more restrictions placed on the schools, the more likely they underperformed. This makes sense if you do the same thing with less money, you will likely get poor results. I would hasten to add that when one allots for the mistake which skews negatively charter schools, under perform may be an overstatement..
The real question this study begs to answer is do charter schools matter? I think it is safe to say that the mere existence of Charter schools is not a magic bullet. The question is are they part of the solution for making education better. The answer is yes, if. Yes, if there are number of charter schools in the area. Caps do not provide enough competition. The states which have caps are the ones which lag. This states which have a free wheeling authorizing system also have issues with low performing schools. It appears that having a few authorities who can enforce accountability. More equality in funding also assists in the process. The dollars should follow the children.
In some areas, charter schools not only equal or outperform with less money, but they help pressure other schools to perform. I wish more attention to that factor was in the study. Someone has to overlay the states with successful charters and states with increase in scores over the last 10 or more years. The Center for Education Reform has 16 years of studies which show the general success of charter schools with more traditional statistical methodology.
I do think we have to value test scores as the number one factor in measuring schools, but I reject that it should be the only factor. First, I believe the scores are statistically compromised in most states because they do not measure a starting and ending point. Many charter schools measure both with Iowa or other tests, but the states no longer recognize that. Another fact is that unlike the fear that all of the best students will leave public schools, the reality is that charter schools attract both the high performers and the low performers. If your students are doing well where they are, you are less likely to change schools than someone who is not being served. The study did control for first year charter students. When they are eliminated, the results for charters look markedly better.
I believe other factors such as school safety,and discipline (such as a bully free environment) also matter. That is why I believe that the choice of parents should matter as much more than bureaucrats looking at a single measure abstract which does not even measure the achievement of individual children during the school year. I call for our public officials to carefully glean useful lessons from this study, but not make policy based upon one study which did not even look at our state. The critics will distort this study and gloss over any difficulties in measurements which call into question its conclusions, but those in responsible positions must delve deeper than press releases and executive summaries.
I have read numerous educational studies and sadly any detailed look at the internals and methodology tend to reveal a bias in favor of what the researcher wants to find and this tends to be a liberal bias.
It is sort of like Climate Gate. The researchers are more than willing to set the game rules so only their side wins and this tends to be for a liberal outcome.
It would be nice to have 5 or 6 straight studies instead of dualing studies based upon biased criteria. I think that not looking at whether or not the individual student improves or lags is a fundamental flaw in our entire system of accountability when you change schools. I have hopes that the Markell administration will move Delaware in a different direction with MAPS.
I support charter schools, but not the way you do.
Charter schools are valuable as prototypes and incubators to develop new practices that can be transferred to the traditional schools. In order to do this, charter schools are necessarily a brain drain and a cash drain on the traditional schools. That is why charter schools should exist, but no more than a few of them.
So it is incorrect to see the charter school system as a replacement or a parallel system that competes with traditional schools. Charter schools are adjuncts and not replacements.
And when we evaluate charter schools, since they are incubators we should evaluate them by the practices they have developed and successfully transferred to the traditional schools.
You just contradicted every fact and study we discussed. They are not a brain drain and they do not work well when their are few of them. They tend to serve the students who need help, not those who are already well placed. You see that in this state. 85% of the charter schools are about serving the kids better not being an elite school aimed at attracting the top 5%. There is nothing wrong with that because the top 5% is underserved, but the reality is very few charter schools across the nation are designed that way. More are designed to help the special needs children.
Charter Schools fill a niche that traditional schools cannot. Their biggest successes have been with underserved minorities. If certain practices can be adapted for regular public schools, as noman suggests, then I think it is great. Certainly, public school administrators should watch these developments and adapt when necessary.
However there are a whole host of children where the targeted specialized charter schools can serve that traditional schools cannot. While there are ROTC programs at 99% of Delaware’s Public High Schools, I would not compare an ROTC class to the experience of the Delaware Military Academy. Just how much of the DMA experience can be transferred to regular public schools I question. What I do know is that DMA will give a specialized, niche experience that no “comprehensive” high school can.
“There are lies, damned lies, and statistics”…W. Rogers.
Any ‘study’ incubated at Stanford is probably tainted. Since, according to leftist academic orthodoxy, there are no longer any objective standards, why would anyone accept a social analysis born in academia? They can’t even hold to standards in the sciences, as seen in the released e-mails relating to climatic change. The academic left are frauds, interested only in maintaining their own failed status quo, and thus, their own jobs.