11/03/2011

11-03-11 - The difficulty of school reform

I'm so opposed to top-down metric based "reform" that I figured I should talk about what I think is a better alternative.

First of all there is no doubt that American public schools are sick. There are lots of good teachers and good classes, but also lots of bad. In my opinion, we don't need massive structural reform, we need a way to get rid of the bad teachers.

(almost always a cluster of bad teachers goes with a bad principal and often a bad superindendent too; they tend to be teachers with seniority who just don't care much anymore, and they all just want to maintain the status quo)

I do believe that charters are not the answer. There's nothing wrong with private schools, but they should be private. I don't believe that federal money should go to private institutions, almost ever, because it leads to corruption, and it also just sucks funding out of the public school system. The charters almost always wind up being a way to discriminate about entrants in some way (even just by desire to go to them), and are also often just a way to sneak around the teacher's union. Anyhoo.

I think the answer is motivating teachers and rewarding good teachers, and also being able to fire bad teachers. If teachers are motivated to succeed, and principals are motivated to hire good teachers and fire bad ones, you have a more free labor market and things will improve.

But how do you do that? This is where the trouble comes in.

I believe standardized test performance is a terrible way to do measure success. Most simple metrics like this would be similarly bad.

Judgement by a panel of peers doesn't work, because the teachers get into collusion and just say everyone is great. Perhaps this could be improved by making teachers grade each other, and forcing the grade to be on a curve so there are gauranteed to be winners and losers. But this would just degenerate into a game of "Survivor" where the old guard makes alliances to vote for each other and so on.

I believe the best answer is to let parents grade the teachers. Schools are one of the few areas where I think local government is actually better than top-down federal government, because it's one of the few areas where the local people actually pay attention to what's happening and get involved. (on the other hand, I think local school funding is probably unconstitutional and needs to be abolished; it creates great inequality to this day, despite many court rulings trying to redistribute funding (such as the "robin hood" ruling in Texas))

One idea is to let parents apply for what school they want their kids in and what specific teacher they want. Kids are then assigned by lottery, but you count the number of applications each teacher gets and that's their score. It's basically measuring demand as if teaching was a good. Teachers with high scores get raises and teachers with low scores get fired.

Now you obviously have to control for things like teachers just giving all A's, so people apply because it's the "easy" teacher. One solution might to force all classes to be graded on a bell. That would actually balance out the social stratification of classes because the grade-grubber kids might want to avoid the most prestigious classes (since they would be full of smart kids and very hard to do well on with a bell curve).

That's all sort of okay I think, but there's a big problem, which is that it biases strongly against areas where the parents don't give a shit. And those are the most problematic areas.

24 comments:

Aaron said...

With parents you gotta really worry about small community/witch-hunt effects (esp in all the tiny backwater districts). You definitely need to have redundancy to catch that sort of thing happening: "hmm, all these parents said "Bob the Atheist Science Teacher" was horrible, but his test scores are good, and the kids reviews of him are reasonable, his peers score him well.

I wonder how much science there is on how corporations attract top talent. I really just don't see a huge amount of 'firing of poor performers/old employees that just can't cut it anymore' going on in industry as a method of getting there. It seems like rigorous interview process + on-the-job training is the route that is more commonly taken to greatness.

Justin Paver said...

You'd like to think that most parents are sensible, well informed, not prejudiced and willing to participate, but you'd be wrong. Many parents have their own wildly divergent theories on the role of a teacher, including babysitter, moral and religious mentor, and chief discipline administrator, rather than the real role they should be playing: educator. I'd find it hard to believe that parents with these theories, which are unreasonable yet highly prevalent, would be able to divorce these notions of role from their evaluation of a teacher's performance.

So, is seems to me your best answer is yet another metric-based reform, terrible at worst and significantly flawed at best. Implementing your suggestion would turn the parent-teacher relationship into a popularity contest, and it would force teachers to coddle parents even more than they already have to. It'd turn teachers that want to keep their jobs into politicial creatures vying for public attention - I don't think I need to say why that is bad.

Parents already have too much say in the functioning of schools by virtue of the pressure they can apply upon school districts, which are largely political offices. Even in non-backwater districts, parents can *today* get teachers fired by making baseless and non-meritorious complaints to the school districts, and teachers have been left no recourse to defend themselves. Teachers have been fired over intimating such things as autistic signs or behavior problems to parents, even though it would be best for the children to get early treatment. Most parents refuse to believe their children are nought other than perfect angels.

Once fired by a district, one would be hard pressed to find another district that will hire a teacher. Career-enders like this happen to *good* teachers all the time, while other good teachers burn out due to the high-burden that is placed upon them - the suggestion that "teachers with low scores get fired" is yet another burden/worry to place on teachers.

In a field where in California the credential process is insane (yes it's already like a rigorous interview process and on-the-job training!), have you thought out why any sane person would go into this profession? In my opinion, like game development, passion for the profession/industry should not be the excuse that people use to mistreat, overwork and underpay the person in that profession.

Here's an article that might provide some more perspective, but from the other side:
http://www.cnn.com/2011/09/06/living/teachers-want-to-tell-parents/index.html

cbloom said...

"It'd turn teachers that want to keep their jobs into politicial creatures vying for public attention - I don't think I need to say why that is bad."

Basically what you're saying is that both democracy and capitalism are bad. That if voters/consumers are allowed to choose what they get for themselves, they will inevitably be short-sighted morons and choose horrible things. You are implying that some benevolent overlords should choose what is best for people and force it on them.

There is certainly merit to your point.

billyzelsnack said...

I don't see the difficulty of assigning a grade to a teacher. Each teacher has a set number of kids and each kid is frequently tested. If too many of the kids are not progressing as expected then something is wrong and further investigation is needed. If a teacher regularly excels at progressing kids then that teacher should be awarded.

Actually I do see the difficulty. You could never implement such a thing because the teacher unions would never buy it as that ruins their lowest common denominator business model.

Justin Paver said...

Heh, can never tell when you're being sarcastic - but I guess that's what makes you so clever, and me a communist who's espousing forceful benevolent overlords apparently ;)

But seriously, that's not quite what I said. The whole thrust was that I don't embrace the notion that parents as a whole can magically make better decisions, and more fairly than those with a deeper understanding of the issues. The noise to signal ratio in such a parent driven ranking would come from the parents and their diverse biases and beliefs which, as has been evidenced in many cases, are often incongruous with reality.

I don't think I'm too far off the mark in suggesting there should be less politics in teaching, not more.

penwan said...

"I think the answer is motivating teachers and rewarding good teachers, and also being able to fire bad teachers."

I agree with you. I'd say there are three pillars: attracting smart/talented teachers, rewarding good teachers, and removing poor teachers.

For the latter two you need some way to evaluate teachers. I think you'd want some combination of the following in a teacher evaluation: peer review, administrative review (principal/superintendent/school board) and parental feedback. In districts where (lack of) parental involvement is a problem you just end up weighting the other reviews higher in the evaluation. And if a teacher in such a district does manage to promote parental involvement and receives positive feedback they'll be rewarded.

But I actually think the #1 problem is attracting smart/talented teachers. We don't compensate high performing teachers well enough to attract people to the field. They can make much more money in private industry, law, or medicine, and so that is where potentially great teachers end up.

"I don't see the difficulty of assigning a grade to a teacher"

That's silly. I don't think we have particularly good measurements of student performance. Standardized tests measure how well students have been prepared to take standardized tests on a particular subject, they do not measure a student's mastery over that subject.

cbloom said...

"The whole thrust was that I don't embrace the notion that parents as a whole can magically make better decisions, and more fairly than those with a deeper understanding of the issues. "

Fair enough, I agree. I don't think the parent-vote method works at all, I was just trying to work out a thought experiment about how you could rate teachers in a sort of "market driven" approach, where consumers get to set the price and value of the good.

There are two major problems with any market driven approach to education :

1. The parents (or voters) are not the consumers of education - the kids are. And one of the goals of education should be that even kids with horrible parents can get good educations.

2. All market theories assume rational actors. That is, the basic theorem that open markets maximize utility only works with rational self-interested consumers. And the consumers are not rational. In this particular case, "buyers" of education would not actually buy in such a way as to maximize their education return.

"I don't think I'm too far off the mark in suggesting there should be less politics in teaching, not more."

Yeah. I wonder how a tenure system would work at lower level schools.

cbloom said...

" I think you'd want some combination of the following in a teacher evaluation: peer review, administrative review "

The problem is that peer review and administrative review just don't work at the moment. Basically neither of those reviewers have any motivation to be highly critical, in fact they have strong motivation not to be.

In the sickest schools that I'm aware of, the worst teachers tend to be good buddies with the administration. I've seen suggestions of teachers from other schools to come over and do the reviews, or have reviews by a panel of education experts, but I'm pretty skeptical about those too. The "education experts" tend to be narrow-minded people who have some buzz word of the moment on a check list (like "this teacher failed to use the 3R system - reinforce, remind, reward").

cbloom said...

"But I actually think the #1 problem is attracting smart/talented teachers. We don't compensate high performing teachers well enough to attract people to the field. They can make much more money in private industry, law, or medicine, and so that is where potentially great teachers end up."

I disagree with this. I know it's a popular view. There's never a shortage of people who want to be teachers. Despite how horrible it is, there are lots of good altruistic people who want to work with kids and don't mind the low pay. I personally thought about it a lot when I was in college/grad school, and the thing that turned me away was not money. I think the things that turn good people away from wanting to be teachers are - excessively onerous credential requirements in some states; outsiders telling you how to run your class; forced lesson plans and drills to improve test scores; kids that are major discipline problems that you can't control; jackass principals and politics and such. It's not the money. We want freedom to run our own classroom, we want summers off, we want small classes. Stuff like that.

And in any case, attracting great teachers won't help much if they don't have freedom.

Most bright young teachers go through the same cycle -

1. I got hired I'm gonna be a great teacher!
2. Whoah things are fucked up
3. I'm gonna make things better!
4. Whoah why is everyone fighting my attempts to improve the school
5. Whoah why am I being forced to teach in this stupid way
6. OMG the board just forced us to use the worst text book ever
7.a. This sucks I quit
7.b. Okay, I give up hope, run the drills and do what they want

billyzelsnack said...

penwan. Mastery is the goal. That's what regular testing should target. If you don't master a concept you don't move on to the next. There is nothing hand wavy about it.

cbloom said...

billy's just trying to keep my blog "fair and balanced" ;)

@billy - would programmers be better if they were paid by # of lines of code submitted?

penwan said...

Yeah, I take your points w.r.t. teacher evaluation. I really feel that the teachers and administrators are the most qualified to evaluate their peers. But how to divorce the evaluation from the human factor, e.g. politics and the "buddy system"... I really don't have any good ideas. The only motivation that comes to mind is repercussions for not rewarding good teachers and disciplining poor teachers. But what repercussions? They'd have to be external and then you are looking at some measurement of how the school performs and all of the hardships that go along with that..

"And in any case, attracting great teachers won't help much if they don't have freedom."

You might be right, maybe higher pay would not be a large enough factor. I still think if we value education as a society we should be willing to pay (well) for it.

Also, many times when I heard people talk about school reform they talk about removing bad teachers as if a plague of bad teachers is the largest problem. I'm not really convinced of that. But you did mention several factors that prevent teachers from having the freedom to succeed. So maybe we need to focus on how our schools should encourage an environment where teachers have the freedom to be successful.

Sadly, that runs counter to the top-down standardization/assessment track that is driving change in our schools today.

billyzelsnack said...

Your above post talks about the popular view regarding teacher shortages. I think there is a popular view regarding grading teachers.

My oldest kid is in 1st grade. The concepts he is taught in school are trivial. The progression of the concepts is insanely slow paced. I do not understand why it is so difficult to grade a teacher given these two aspects.

cbloom said...

"I still think if we value education as a society we should be willing to pay (well) for it."

Sure, I agree. One problem in the last 10 years is that education spending has gone way up, but very little of that has gone to teacher pay. More and more of the budget goes to administration, testing experts, etc.

One of the ideas of "reformers" is to go back to larger class sizes, which would allow teacher pay to go higher. If you have $100k to spend for 50 students, you could have two $50k teachers in 25 student classes or one highly paid teacher. The reformers imagine that this highly paid teacher will be so superior that the students will do better despite the larger class, and I think that's very wrong.

"Also, many times when I heard people talk about school reform they talk about removing bad teachers as if a plague of bad teachers is the largest problem. I'm not really convinced of that."

True true. People usually talk about school reform as if it's one issue when really there are many separate issues.

My idea with "market pricing" classes is not really intended as a way to fire bad teachers. It's intended as a way to give teachers freedom to do different things in classes, and have some way of rating whether that was successful. Without a rating system, you can't tell if what they're doing is okay, and that makes you afraid to allow any experiments, and you wind up forcing uniformity.

cbloom said...

"My oldest kid is in 1st grade. The concepts he is taught in school are trivial. The progression of the concepts is insanely slow paced. I do not understand why it is so difficult to grade a teacher given these two aspects. "

Dude, 1st grade is supposed to be about play and socialization. The proper way to judge a 1st grade teacher is whether the kids have developed a love for school or a fear/hate relationship with school.

But slow/uniform progress is actually one of the things that has come out of standardized testing metrics.

One of the "big results" of numerical education research is that the average score goes way up if all kids are taught at the same speed, and all levels of kids are mixed together. Letting kids who are ready for new topics move ahead at a different pace is not good for test scores.

So gifted programs are getting reduced; high schools with various levels of classes are getting the
levels merged, etc.

billyzelsnack said...

I didn't mention higher grade levels because I had assumed you would consider all concepts as trivial in a K-12 education. At what point is the non-trivial stuff introduced?

castano said...

You should take a look at "rate my professors". It does not include school teachers and it's not heavily populated yet, but most students that I know use it extensively to pick their classes:

http://www.ratemyprofessors.com/SearchProfs.jsp

jfb said...

What you really get with your 'grade-grubber' "solution" is that kids who want to go into a more difficult class will be discouraged into doing so, because moving up in difficulty you tend to struggle initially. Great for reducing social mobility, if that is your goal? :P

jfb said...

Sites like Rate My Professor are only helpful for people who want an easy ride. My favorite professors in college were consistently rated badly on that site because they presented difficult material and were not interested in slowing down.

"Sink or swim" tends to be rated badly, while "bore everyone to death so the kid who doesn't read the text can pass" tends to be rated well.

jfb said...

...but on the other hand, there are people who prefer that, so perhaps it's fine. I wouldn't want to see it tied to anything though..

cbloom said...

You are annoying. Try to be more amusing and/or insightful. You need to either have a sense of humor or say something actually useful or interesting.

jfb said...

Sorry Charles, I forgot I was only supposed to agree with you. I made no attempt to be funny -- the only effect of curving to solve your supposed "grade grubber" problem _would be_ to keep people from trying harder classes than their usual. On top of that it introduces artificial competition that has nothing to do with the subject.

Ginzo said...

I think Charles that you're underestimating the significance of money in education. You're right that many teachers burn out because of administrative pressures and other bullshit that goes along with it, and that they do not really mind their mediocre salary.

But the job's a lot easier in a well-funded environment. Money yields smaller classroom sizes. Classroom size is the most crucial, and underrated, part of the war involving teachers and administrators. With a class of 15 as opposed to 30 students, teachers can do so, so, SO much more for their students. That's not possible without funding - and it's got nothing to do with individual salaries. It'd mean hiring more teachers, so instead of my former high school having 8 history (social studies really) teachers for 950 students, it'd have maybe 11. That's a huge difference. That's over 30 students you don't have to grade, worry about, listen to, whatever - basically a whole classroom size in today's climate.

I cannot really think of a way to quantify teaching responsibly and credibly. Attempts to do so seem misguided and pointless and invariably lead to the sort of credential-chasing dystopia you correctly postulate as the root of teaching's many evils. We just may need to accept that education is a profession that assessment cannot police, so we'll have to try and hire administrators we trust to reward and punish their teachers accordingly.

cbloom said...

"Money yields smaller classroom sizes. Classroom size is the most crucial, and underrated, part of the war involving teachers and administrators."

Yes; I think I wrote this somewhere; personally I'm a huge fan of small classes. Unfortunately some of the metric-based studies have shown that class size is not very important.

"I cannot really think of a way to quantify teaching responsibly and credibly."

I basically agree. Unfortunately, the idea of metric-based reform has become very popular politically, so it seems that it will be pursued for the near future. Because of that it behooves us to find metrics that actually correlate to good teaching. (I'm afraid some of the currently used metrics may even counter-correlate).

old rants