Are criminal risk assessments racist?

Good idea– using data and well-validated models to predict potential for future criminal behavior and responding accordingly with criminal justice procedures.  Bad idea– using data and poorly-validated, racially-biased models to do the same.  I bet which one of these we’ve been using.

Amazingly, those who looked at these models found that the error rates were pretty similar for Black and white defendants.  Alas, they did not look at the direction of the error.  This chart pretty much says it all:

Prediction Fails Differently for Black Defendants

Labeled Higher Risk, But Didn’t Re-Offend 23.5% 44.9%
Labeled Lower Risk, Yet Did Re-Offend 47.7% 28.0%

Overall, Northpointe’s assessment tool correctly predicts recidivism 61 percent of the time. But blacks are almost twice as likely as whites to be labeled a higher risk but not actually re-offend. It makes the opposite mistake among whites: They are much more likely than blacks to be labeled lower risk but go on to commit other crimes. (Source: ProPublica analysis of data from Broward County, Fla.)

So, yeah, roughly equivalent number of mistakes, but the mistakes systematically benefit whites and harm Blacks.  Lovely.  So, now that we know there’s a problem, let’s see what we manage to do about it.

Great article from Pro Publica.  Short version via NPR.


About Steve Greene
Professor of Political Science at NC State

2 Responses to Are criminal risk assessments racist?

  1. ohwilleke says:

    Two thoughts:

    1. It isn’t terribly hard to fix instruments like that when the validation shows that they are skewed. This shows why transparency and regular revalidation is needed when these tools are used, so that these issues do get fixed.

    2. The amount of bias in the non-instrument based human alternative decision making is typically much higher.

    • Steve Greene says:

      Good points– especially #2. But given that it’s really not so hard to fix this, then they should have never been so widely fielded without paying attention to these issues. And why they need to be way more transparent.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: