My partner Robert and I, we talk about prisons. We talk a lot about prisoners. We talk about grief and redemption and “the system”. We talk about sentencing and we talk about executions. We talk about these things more than any couple I know. And we talk about these things from two perspectives: the Chaplain and the Analyst.
Recently, I’ve been troubled by the lack of standards for sharing data. The way data is shared almost ensures that there will be errors in interpretation. From the various US Government datasets to Reinhart & Rogoff, the format of the data delivered almost certainly requires us analysts to do horrible things to make the data usable. These things include cutting and pasting and reformatting. And often repeating this activity across multiples tabs in an Excel workbook or across multiple CSV files. It’s a CSV file, just put it all in one. These horrible things would be entirely unnecessary if data were shaped and shared properly. I have some definite opinions on this and will offer my recommendations in a future post. That is a story for another day.
While I was working on that “story for another day” Robert and I were listening to the Planet Money podcast “How Much Should We Trust Economics?” This episode explores how Amherst graduate student Thomas Herndon’s replication of Reinhart & Rogoff’s (R&R) paper “Growth in a Time of Debt” revealed serious errors in data handling and ultimately a different conclusion. R&R’s paper has been has been widely cited (over 500 times) by scholars, used to set international economic policy and was a foundation point in Paul Ryan’s budget proposal. Through replicating R&R’s paper, Herndon identified critical discrepancies in the data: some of data was missing due to a coding error, some data was simply left out, and other data had been summarized in a non-standard way. Herndon argues that after “fixing” R&R’s data errors it’s clear that their conclusions were wrong. And if their conclusions were wrong, then how many struggling countries with high debt to GDP ratio failed to receive much need aid? How many lives were affected?
So, how else are people adversely affected by misused data and faulty conclusions? The Planet Money team reached out to Economist Justin Wolfers of the University of Michigan to understand how frequently these types of errors occur in studies. And it’s astonishing. And here’s where we get back to talking about prisons. Wolfers argues that’s he’s “replicated every single paper in the death penalty literature” and there are errors in some 90% of them. While Wolfers agrees that some of these “errors” are methodological issues debated about in the community in many other cases “people just screwed up the math”. He explains that one such error changed the author’s conclusion from “each execution deters 18 homicides a year” to “each execution causes another 18 homicides a year”. It causes another 18 homicides a year. Each execution.
My partner Robert asked if I would do a little data sleuthing with him and look at what information is available on prisons and the death penalty. He wanted to know if I could visualize with data what he sees and experiences in his work with prisons. Some years ago I took a workshop with Stephen Few, arguably one of the foremost thinkers on design and data visualization best practices, and at the end of the workshop he implored us to go out into the world and to “use our powers for good”. I hope my work with Robert and our ongoing dialog about prisons, about the people who live and work there, about the people who die there, about the lives affected by those in prison helps to lead me down that path. Our conversation begins with this:
- Between 1977 and 2012 2,637 people have been executed in State prisons in the US
- That’s an average of 73 people executed per year
- Texas has executed people at a rate nearly 5x higher than the next two states leading in executions, Virgina and Oklahoma
- Of our 50 states, only 34 have executed a State prisoner during this time, and only 15 states average more than one execution a year.