I grade with a colloidal silver pen, of course
By Thoreau
A study of how liberal and conservative professors grade (.pdf) has gotten some attention. The soundbite version is that two economists studied grading patterns of professors at a large, elite university for several years, and found that Democrats tend to give fewer grades at the high and low ends of the scale, while Republicans give a wider distribution of grades. Also, black students apparently get slightly lower grades in classes taught by Republican professors.
Not having read the study in detail, I have no real comment to offer on whether the potentially explosive conclusions are supported by the data, whether the methods are sound, or whether this validates my existing world-view. Instead, I’d rather focus on something else: After reading the soundbite version, many intertubez commenters followed internet tradition and responded with “This social science study can’t possibly be valid, because the authors didn’t control for this variable that would explain everything.” In this case, the variable in question is discipline: There are more Republicans in business schools and economics departments than in teh librul artz, so obviously we’re just seeing disciplinary differences in grading, right? That’s the REAL variable that those smarty-pants social scientists didn’t even bother to account for, and I know this because I read the soundbite version.
Well, in violation of internet tradition I decided to actually Read The Fucking Article. I didn’t read it in detail, but I was able to ascertain two things:
1) The study was limited to the College of Arts and Sciences at an elite university. (This is stated on page 4 of the .pdf linked above.) Right-wing professors in the Business School and soft-grading left-wing professors in the College of Education weren’t included. Yes, there might still be significant variation by discipline within the College of Arts and Sciences, but some of the easy strawman arguments are not applicable here.
2) Although I haven’t examined it in detail, the authors did not ignore differences between disciplines. These points are made on the bottom of page 6 and top of page 7, with reference to tables. I haven’t examined it closely enough to say whether they did enough work or had a large enough sample to really disentangle disciplinary effects from other variables, but I can say that (contrary to those who read only the soundbite version) the issue was not ignored. If you wish to find fault with their claims, you’ll have to actually read and analyze what they said (a librul artz skill) rather than just assert that some variable was ignored and therefore the whole thing is invalid.
Again, I’m not endorsing anything in that study, because I haven’t read it in detail, but some of the obvious stuff is actually addressed (however thoroughly or sloppily) despite the dismissals by intertubez commenters.
Also, I have my own theory which must be right because those smarty-pants social scientists didn’t address it: There is literature showing that students respond differently to feedback depending on whether the professor grades with a red pen or a green pen. Did they consider the possibility that partisan preference dictates ink preference? I recently switched to green pens, just for the hell of it, but I plan to switch to colloidal silver pens when I run out of green ink. (I would just throw my green pens in the trash, but that would not be fiscally prudent.)
Comment by Glaivester —
May 21, 2011 @ 11:23 am
One interesting question (which the study may or may not address, I’m not familiar with the jargon, I don’t think it is addressed but I may be misinterpreting something) is whether the poorer performance of minorities with the GOP professor is due to the larger variance in grades as a whole – that is, seeing as blacks and Latinos perform worse than whites with both Dem and GOP professors, and seeing as the GOP tendency is to to have a wider range of grades, with only those near the top doing better with GOP professors, perhaps any minority whose grades are generally lower than that of (non-Latino) whites will have that difference exaggerrated when working with a GOP professor.
The most obvious way to determine whether the effect is explicitly racial/ethnic would be to take the median black aand median Latino student and see what percentile they would be in with the same grades if they were white. (Actually, it would be even better to take a few samples from several different percentiles and compare them). (For example, would the median black student be in the 30th percentile in both GOP or Dem professors’ classes, or would he be in the 40th in the Dem and the 20th in the GOP professors’ classes)
The most interesting line in the study for me is “What this means is that, controlling for SAT scores, Black students tend to under-perform relative to White students in the university we study.” As SAT scores are to some extent proxies for IQ, the phrase “controlling for SAT scores” implies that this effect is independent of IQ, presumably passing over the usual Charles Murray v. Stephen Jay Gould debates.
Comment by Eli Rabett —
May 21, 2011 @ 2:39 pm
Colloidal nanosilver, of course.
Comment by Eli Rabett —
May 21, 2011 @ 2:39 pm
Sorry, that should have been colloidal nonosilver.
Comment by GMP —
May 21, 2011 @ 7:55 pm
I think it is imperative to discuss the relative merit of green vs purple vs silver pens for grading, since red is out as it apparently makes students feel bad (screams “failure”) and students must never ever feel bad no matter how crappily they do on the test. I am personally partial to purple — is it a simple color preference or is it really my hidden political agenda (keep everyone guessing if I am more red or more blue?)
Comment by BigHank53 —
May 25, 2011 @ 9:28 pm
Buy a fountain pen and use any damn color ink you please. Also, the payoff period on even a $400 fountain pen is much shorter than you think–those rollerballs and gel ink pens add up in a hurry.