Just Mercy

I recently finished the book Just Mercy: A Story of Justice and Redemption by Bryan Stevenson. It details Stevenson's work in founding and running the Equal Justice Initiative, an organization fighting racial bias in the judicial system, children sentenced to life in prison (or as Stevenson calls it, death in prison), women being convicted of murder after miscarriages, and mass incarceration in general.

Stevenson describes a number of gut wrenching cases of injustice, primarily the story of a man name Walter McMillian. He was an obviously innocent man who was sentenced to death principally because a county sheriff wanted a conviction for a murder case.

A major theme of the book is that the judicial system is racially biased. Granted the limited number of cases presented in the book are just anecdote, but the stats on convictions clearly show that minorities are incarcerated at disproportionately higher rates than whites. This wouldn't necessarily indicate racial bias if minorities committed crimes at commensurately higher rates. Studies attempting to account for this find that 15-20% of incarceration disparities cannot be accounted for by crime rate disparities.

Combing this book with my recent experience in a jury selection pool, I am left feeling that our judicial system is often more theater than an unbiased application of justice. Not that determining guilt and ascribing punishment is an easy business, but I can't help wondering if there might be a more logical, consistent approach (just today I saw this ruling where the concept of burden-of-proof seems to have been utterly disregarded).

My proposed alternative:
  1. Have an algorithm into which all evidence can be input that will then calculate an estimated probability of guilt.
  2. Another algorithm will consider all aspects of the alleged crime and produce a sentence length (not factoring in the likelihood of guilt, just assuming the defendant committed the crime).
  3. If the probability is below some threshold (say 35%, or maybe just good old 50%), the defendant is declared not guilty.
  4. If the probability is above the threshold, the defendant can choose to 1) serve a sentence equal to the length found by the second algorithm multiplied by the probability found by the first, or 2) take a gamble with a device that declares him guilty or innocent, where the chances of going free are again based on the probability found by the first algorithm. If the defendant is declared guilty, he must now serve the full sentence.
Just throwing out a crazy idea. Another solution would be for people to stop being racially biased.

Comments

Ryan said…
I always wonder how these studies measure racism, because each criminal case is so fact dependent. There have been a number of high-profile cases (Trayvon Martin, Michael Brown) where we were told the shooting was the result of racism, but in my opinion, the evidence showed the shootings were justified. Then we've had some other incidents, like the Castile shooting, where the shooting wasn't justified, yet I'm not sure it was racially motivated. I tend to think this is an area where it's just not easy to draw conclusions. How do you study people's motivations, when presumably they are not admitting that they are motivated by racial bias?

It's funny that you suggest an algorithm as a neutral approach. I listened to an EconTalk podcast about a federal sentencing algorithm the guest criticised as racist. Apparently, the algorithm calculates recidivism pretty well but the defendant's neighborhood heavily factors into the algorithm. The highest recidivism occurs in black neighborhoods, so blacks get harsher sentences. The guest's argument was that the algorithm is racist because, even though recidivism is high in those neighborhoods, and even though the use of that information makes the algorithm work, the algorithm basically judges the individual based on his race rather than as an individual.







Ryan said…
This is the podcast I referenced:

http://www.econtalk.org/cathy-oneil-on-weapons-of-math-destruction/
Brett said…
I don't think anyone is going through on a case-by-case basis and deciding whether or not racism played a part (or if they are, I agree that's not a great idea). The stats that point to racism are taking convictions in the aggregate. I didn't look too deeply into their methods, but I recall one study compared numbers of arrests to numbers of convictions. They found that minorities who were arrested were much more likely to end up being convicted. I'm sure there's some correlation between income-level as well, but controlling for income, race still came out as a significant determinant of likelihood of conviction. I don't see the need to judge the motive in any individual case for this type of analysis.

Regarding an algorithmic approach, I would definitely agree that an algorithm could lead to racist results. It would actually be pretty difficult to not make it racist...but it's probably easier than having a judge/jury that are not racist (and by that I mean having implicit racial biases, not being overtly racist).
Ryan said…
I think the fact that each case is not looked at individually is the flaw. To use an example from another context, Supreme Court observers will sometimes say things like Justice Alito is pro-business because he rules with big business 70% of the time. But that kind of analysis completely ignores that each case before the court is a complicated legal problem, which rises and falls on its own merits regardless of who the parties are. In other words, correlation is not causation.

So to use your example, arrests for blacks could result in more convictions, but that could be for a whole host of reasons that are not racially motivated. Like, maybe Blacks' cases are more likely to involve violence. Maybe there are more black repeat offenders, such that the arrest is more likely to be prosecuted. Maybe Blacks are less likely to take plea deals, which avoid convictions. Maybe, blacks are treated more harshly, but by other blacks who are in positions of power but who are not racially motivated. I'm not saying the disparate treatment isn't racially motivated, I'm just saying I'm very sceptical its possible to control for all the other possible variable and tease out that racial bias is the difference.

Brett said…
Let’s say Alito sides with business 70% of the time and Kagan only 15%. I agree that there could be mitigating factors at work and that this doesn't necessarily mean that they are pro/anti-business. Maybe there were other aspects of the cases that were influencing the decisions. Maybe most of the cases involving businesses also involve tort law and Alito and Kagan have philosophical difference on how tort law should be applied (there has to be something to account for the judges coming to different conclusions right?). If that were the case, then their disparate rulings would be spuriously ascribed to being pro/anti-business when the actual cause is their interpretations of tort law.

But that’s what multiple regression is for. You could then look at tort cases that didn’t involve businesses and see if the trend holds. If it did, that would strengthen the argument that Alito and Kagan differ in how they believe tort law should be applied and weaken the evidence for pro/anti-business biases.

Or are you saying that the judges have no biases whatsoever, and that any case that makes it to the supreme court is going to be a close call, so it’s no surprise that different judges come down on different sides of a case? Thus any apparent trends in their rulings are false interpretations of what are actually random patterns? I could buy an explanation like that for maybe a dozen cases, but as the sample size grows and the trends in decisions persist, the evidence for an underlying factor causing the trend is strengthened.

In the case of race factoring into convictions, the sample size is very large and the difference in conviction rates is also sizable, so there is very likely some reason for the difference. Multiple regression is employed and shows that the difference persists even when controlling for income, education level, type of offense, and every other factor for which there is data. Sure there’s a chance that the actual reason for the different conviction rates is some other factor for which researchers didn’t have data and hence they couldn’t control for it, but how is that more probable than juries having implicit racial bias?
Ryan said…
I get that social scientists try to control for other variables. But at some point, a problem is so complex you cannot control for all the relevant variables and I suspect this is one of those cases. And obviously, the fact that the statisticians controlled for the available variables doesn't mean they controlled for the right variables. There is a reason lawyers spend hours on their cases sorting through facts and that's because each case has its own facts. I'm skeptical of statistical attempts to boil down complicated fact patterns into a few variables.

I'm skeptical of this kind of thing in other contexts, too. We're told breastfeeding leads to better outcomes for kids later in life. I don't believe our statistical tools are powerful enough to tease that out. We're told all the time certain foods cause cancer or prevent cancer (and some do both!). I've heard of older studies that say Tiger Woods causes the field of golfers in a golf tournament to play 1/4 stroke worse. I don't believe for a second we can control for all the other relevant factors necessary to tease out whether Tiger causes other golfers to golf worse. See also, basically every macroeconomic study.

Brett said…
Better late than never...

The authors of these conviction studies are careful in their language. They don't say that 15-20% of racial disparities in conviction rates are caused by racism, they say that their models can explain 80-85% of racial disparities in convictions by factors such as differing rates of committing crimes, poverty, etc.

Granted we are left to guess at the remaining causes, but in the absence of plausible alternatives, it's not surprising that many would take this as evidence of racial bias in the criminal justice system.

"There is a reason lawyers spend hours on their cases sorting through facts and that's because each case has its own facts."

To me this is akin to measuring one single data point very accurately. Sure you want each data point to be as reliable as possible, but large collections of data can show definitive trends even when there is a fair amount of uncertainty in the individual data points.

"I'm skeptical of statistical attempts to boil down complicated fact patterns into a few variables."

Are you suggesting that regression analysis, or maybe all of statistics, is nothing more than smoke and mirrors?

Popular posts from this blog

2019 Stats - Solar

Two Speeches

Books (2022 and 2023)