Education is like Basketball

How's that for a title, eh? Allow me to explain this simile.

In the NBA, basketball teams are constantly trying to get the most out of their players. After all, winning sells tickets. If you don't win, you have nothing. So coaches tinker with their game plans, what plays to run, what defensive systems to employ, who to focus on, who to ignore, etc. They also tinker with which players on their own team should be playing and when. Obviously, the star of the team needs to play the most, but no player should be on the court the whole 48 minutes, so then begins the question of who should play when they're taking a break. And what players effectively complement that star? The value of "team chemistry" is debated perennially, but when a game can be decided by a single basket, in just a few seconds, that "team chemistry" matters as much as anything.

On top of that, team managers are watching every player, and compiling statistics on all the players in the league, in hopes of trading a player here or there for another team's player. The best trades lead to an improvement in both teams. How? Because we're talking about people here, and everyone has something different to offer.

In this extremely tense atmosphere, people have come up with dozens of numerical methods of measuring players' abilities and contributions on the court. One of the more interesting ones is referred to as "plus/minus". It shows a simple number that's either positive or negative: if while a player was playing, his team scored more than the other team, it's a positive number; otherwise it's negative. In theory, a player with a plus/minus of zero has no effect on how well his team performs.

So you would want to get rid of any player with a negative plus/minus, and only stock your team with players with positive plus/minus, right? Well, it's much much more complicated than that. It's quite possible to do in theory. However, plus/minus depends on a lot of factors. First of all, if you have a player who's only on the court with the best players on his team, he might have a much better plus/minus than if he was only on the court with the worst players on his team. Also, some positions in the game of basketball have been shown to have a greater potential effect on the performance of a team. The center position, held by the tallest player often has the greatest effect defensively, while the point guard, held by the shortest and (usually) smartest, has the greatest effect offensively. On top of that, if a great player is somehow playing more often against bad players, he might appear to be more effective, when in reality he just doesn't have as much of a challenge.

The point of that explanation is to show that a seemingly simple and elegant measurement is actually full of weaknesses when it comes time to use it in a practical environment. It's the same case with education.

People right now are trying all sorts of things to come up with a way of measuring teacher effectiveness. However, the problems are legion. A teacher with a gifted class of students from a high-income neighborhood will often seem to perform higher regardless of the teacher's ability. A teacher with a class of challenged students will obviously perform worse. How do you account for that numerically? Furthermore, a student's parents have been shown to have an equal if not greater effect on their child's performance in school. And what about the students' peers? A child in a class of gifted students will have an easier time learning regardless of whether he/she is gifted him/herself.

These effects are compounded over time. Imagine grading teachers based on how their students do after leaving their class. Well, what if those students go on to have several extremely effective teachers, or end up in classes with more gifted students. Or even more confounding, what if after a difficult year their parent signs them up for tutoring?

One might argue that like plus/minus in basketball, many of these scenarios can be balanced out by a large selection of data. While a player's plus/minus can vary wildly from game to game, it tends to average out to a more accurate value over the course of 82 games a year. This is true, yes. However, there are few teachers that get a large enough range of students in terms of ability to balance things out in that way. In other words, a school in a low income neighborhood is not going to have enough gifted students coming in its doors to give enough data on how their teachers could perform with better students. And all of this is assuming the tests themselves are accurate portrayals of student achievement. They're not.

The point of all this is to illustrate just how amazingly frustratingly complex the world of education is. To think you can reduce it all to numbers is naive. Yet the usefulness of reducing it all to numbers is irresistible.