Statistical differences and tenure

This interesting piece in the Guardian explains an all-too-common error in social science research.  I’ll let them explain:

Let’s say you’re working on nerve cells, measuring their firing frequency. When you drop a chemical on them, they seem to fire more slowly. You’ve got some normal mice and some mutant mice. You want to see if their cells are differently affected by the chemical. So you measure the firing rate before and after applying the chemical, first in the mutant mice, then in the normal mice.

When you drop the chemical on the mutant mice nerve cells, their firing rate drops, by 30%, say. With the number of mice you have this difference is statistically significant, and so unlikely to be due to chance. That’s a useful finding, which you can maybe publish. When you drop the chemical on the normal mice nerve cells, there is a bit of a drop, but not as much – let’s say 15%, which doesn’t reach statistical significance.

But here’s the catch. You can say there is a statistically significant effect for your chemical reducing the firing rate in the mutant cells. And you can say there is no such statistically significant effect in the normal cells. But you can’t say mutant and normal cells respond to the chemical differently: to say that, you would have to do a third statistical test, specifically comparing the “difference in differences”, the difference between the chemical-induced change in firing rate for the normal cells against the chemical-induced change in the mutant cells…

Nieuwenhuis looked at 513 papers published in five prestigious neuroscience journals over two years. In half the 157 studies where this error could have been made, it was. They broadened their search to 120 cellular and molecular articles in Nature Neuroscience, during 2009 and 2010: they found 25 studies committing this fallacy, and not one single paper analysed differences in effect sizes correctly.

Guilty.  I’ve actual know what to do to compare the difference in the differences.  And sometimes I even do it.  But, if you can get away with arguing for “significant” results in a case like the example here, well, heck, why wouldn’t you.  The article concludes:

But the darkest thought of all is this: analysing a “difference in differences” properly is much less likely to give you a statistically significant result, and so it’s much less likely to produce the kind of positive finding you need to look good on your CV, get claps at conferences, and feel good in your belly. Seriously: I hope this is all just incompetence.

Nice hope, but, it’s pretty clear that the answer is scholars generally like good-looking CV’s, claps, and a happy belly.  And most of all tenure.  So long as a discipline accepts lower standards for concluding “significant” results, everybody’s going to do it.  Now, a good question is, how do you go about changing this?

Advertisements

Capital gains and losses

Great piece in the Post a couple of days ago about the Capital Gains tax rate.  Hits on several key points.

First, despite always hearing about how important a low rate is to spurring investments, there’s not actually a lot of empirical evidence for this.  Instead, we just get the discredited by the financial collapse, Alan Greenspan, insisting it is so:

The theory justifying low capital gains taxes has many philosophical fathers but none as influential as Alan Greenspan, the former Federal Reserve chairman who was treated as an economic seer for decades.

Greenspan said capital gains taxes made people reluctant to move out of one investment and into other, more-promising ones.

In 1997 congressional testimony, Greenspan said the “major impact” of the capital gains tax, “as best I can judge, is to impede entrepreneurial activity and capital formation.”

“The appropriate capital gains tax rate was zero,” he added…

These changes drove down the overall tax rate paid by the wealthy. In 1996, before the capital gains cut under Clinton, millionaires paid an effective rate of 30.8 percent. By 2007, it was 22.1 percent

“Lower capital gains [taxes] are a mixed bag even if you’re just looking at efficiency,” said Leonard Burman, a professor at Syracuse University and former head of tax analysis at the Treasury Department. “It might encourage more risk-taking, but it also creates huge opportunities for tax shelters aimed at converting ordinary income to capital gains. People would make investments only because of the tax benefits.”

Moreover, he notes, given the recent financial crisis, it’s not clear that an absence of risk-taking is what’s ailing the economy…

Secondly, and sadly, the support for this needlessly low rate, is more bipartisan than it should be:

Yet as Congress debated the fate of the Bush tax cuts, a group of 47 Democrats wrote a letter to then-House Speaker Nancy Pelosi (D-Calif.) opposing any hike in the tax on capital gains or dividends…

And last, there’s a reason its bipartisan.  Everybody loves political donations from rich people.

“Wall Street loves the preferential capital gains rate. All of America’s 20- or 30 million wealthy small investors love capital gains rates,” Sullivan said. “It’s just a tremendously popular item with political contributors. It’s something that directly impacts every wealthy household in America.”

End result, is that rich Americans often end up paying much lower effective tax rates than middle class Americans.  Not suprising that we should have seen such a bifurcation and concentration of wealth in recent years.  These charts tell quite a story:

A look at capital gains taxes.

As an economist explains, there’s some good reasons to have capital gains rates lower than the top income tax rate, but there’s nothing to justify the 20% differential, except for the fact that rich people really like it.  Our society has become vastly less equal and has seen far too much of its wealth concentrated in a tiny percentage of individuals.  This is not an accident, but at least in part a result of purposeful policy choices.

%d bloggers like this: