How to Google Shit to Win Internet Debates
This week I got into a fun Twitter spat, ostensibly about whether liberal prosecutors are soft on crime. This is objectively not notable beyond the personal entertainment value, but I keep encountering the dynamics illustrated in that conversation with regular frequency. So what I would like to catalog here is a brief description of a look behind the scenes and what my thought process looks like.
The instigating tweet was this pronouncement by law professor Andy Grewal, in response to the latest mass shooting.
We’re both alumni of the same law school, but Grewal’s scholarship and publication history is miles ahead of mine. Given his role as an academic and his scholarship history, my initial assumption is that his opinion is supported by solid evidence. In his tweet, he claims that a disturbing contributor to gun violence is how lax prosecutors are on gun crime. His statement is relatively unambiguous and, crucially, testable.
But what he claimed immediately caught my attention because it contradicted my Lived Experience™. My day job is a public defender, in a very liberal city. And in part because of my *ahem* background, I’m known around town as the gun guy who is specifically sought out to handle firearm cases. I have no shortage of work.
To be perfectly clear, my caseload is not at all representative. I’m only one lawyer, out of hundreds, and despite only working on public defense cases, my caseload will never be high enough to be considered a representative sample of the system as a whole. I cannot (and do not) impute a systemic trend about a county of millions based on a few dozen examples.
But the dissonance between Grewal’s pronouncement and my daily experience did get my attention, and made me wonder about the factual accuracy. This isn’t my first rodeo on this front. Previously, I’ve repeatedly encountered the canard that rioters involved in the 2020 protests/riots were getting lax treatment by law enforcement. That too caught my attention, not only because it goes against the institutional incentives of prosecutors, but also because I was personally aware of multiple cases of rioters getting prosecuted hard for their actions. I’ve written about the difficulty in finding a good answer to that question. Court record systems are ancient and often intentionally opaque and cumbersome (which I suspect is to avoid “Florida Man” phenomenon which likely stems from the state’s unusually transparent public record laws). Synthesizing data of that scale is a monumentous task, and one which I don’t begrudge anyone from avoiding.
Presumably, the lack of strong evidence would necessarily preclude a strong conclusion, but this was not at all the case with the above canard. In the absence of systemic data, folks have instead glommed onto to anecdotal evidence of some riot-related charges getting dismissed for some defendants. Were cases getting dismissed more than baseline? Who cares! This issue came up also when discussing the apparent shoplifting craze caused by San Francisco allegedly choosing not to prosecute crime anymore. There is a torrential deluge of videos (and corresponding appetite) of shoplifters brazenly filling up backpacks, shopping carts, minivans, etc of stolen goods and walking out with no apparent consequences. Applied Divinity Studies had an absolutely phenomenal examination of the apparent Bay Area shoplifting craze using actual data and concluding there was no such thing. Despite ADS achieving a level of analysis I thought was impossible, the response to their conclusion was largely annoyed disbelief. How can their conclusion possibly be true in the face of videos of so many Walgreens heists? The derision towards ADS’s (unpopular) conclusion on this Motte thread is illustrative of this type of response.
This is also exactly what Grewal was doing when I asked what evidence he relies upon to reach his conclusion.
I find this way of thinking extremely bizarre. “What about all these news stories?” should generally not be a sufficient response to “Here’s some data”. Nothing about what I’m saying should at all be controversial: bring strong evidence to strong conclusions!
What I’ve described so far has largely been right-coded, so playing the role of the optometrist for a second, let’s look at it through a different lens. It’s no secret that the killing of black men by police occupies an enormous chunk of the discourse field, particularly by those on the left. You would hope that increased interest would translate into correspondingly increased information but ho-ho-ho let me quickly disavow you of that fantasy. When people were asked “How many unarmed Black men were killed by police in 2019?” a eye-popping 22% of “very liberal” respondents said 10,000 or more (the actual answer is 27).
It’s not surprising why their estimate would be so drastically divorced from reality. There is a torrential deluge of videos (and corresponding appetite) of cops shooting black men, and humans are painfully vulnerable to falling prey to the availability heuristic trap. Our pathetic lizard brains can’t grapple with scale very well, and very often we casually impute “greater frequency in reality” based solely on “greater frequency in my thoughts”. The same thing happens when you ask people to estimate demographics, with apparently the average American estimating that Muslims make up 27% of the U.S. population (1% is the reality).
So our lizard brains suck, but we can at least try to guard against these vulnerabilities with a mix of analytic rigor and inquisitive curiosity. But ho-ho-ho let me again quickly disavow you of that fantasy, because the lure of confirmation bias is irresistible.
I think I get what’s happening. With the 2020 BLM protests, we witnessed an undeniable tide swell in the cultural zeitgeist that was overwhelmingly one-sided. June was a relentless onslaught of performative black squares by every corporate marketing department (thank you again Chipotle and Call of Duty for informing us of your position on the matter) and comedy podcasts skipping episodes to put out somber “quick notes” that sounded like they’re recording from a funeral. Regardless of my full-throated support for BLM’s policy goals, I fully acknowledge how the bandwagon was full of opportunistic participants lacking any sincerity.
Faced with such a blatant spectacle of a naked emperor no one seemed eager to acknowledge, it was therefore reasonable for anyone on the other side of this culture war divide to feel they were colossally outclassed and outnumbered. There was then a torrential deluge of videos (and corresponding appetite) of left-wingers burning down “Biden’s America”, and no corresponding deluge of videos of them being prosecuted. So the embattled fear spreads, and risks conjuring up grievance scenarios (e.g. even the prosecutors are aiding our enemies) which may or may not actually exist. This appears to be related to the notion that the “defund the police” movement was actually successful, and those who asked for it are reaping what they’ve sown.
Are liberal prosecutors soft on crime? Maybe, show actual evidence. Is that softness contributing to an epidemic of lawlessness in liberal cities? Maybe, show actual evidence. It’s dispiriting to see such a distinct lack of curiosity in answering this question truthful. Yet we know how arguments are soldiers.
Let’s turn back to Grewal’s claim about soft prosecutors, where he name drops Chicago’s prosecutor specifically. I don’t practice in Chicago, so I have no idea what the prosecutorial culture is like there. If I had any priors about the scene, it would be somewhat similar to my assumptions about New York City — a very liberal city with stringently-enforced gun control laws. I’ve never even heard of Kim Foxx before this, but I just googled “chicago prosecutor” and then “cook county prosecutor” and then “cook county prosecutor stats”. Turns out, Foxx’s office has publicly available and easily digestible data on how its office deals with felony cases referred to it by law enforcement. Let’s see if you can identify any notable findings in this graph about 2021:
Something certainly sticks out! Apparently a whopping 60% of ALL of criminal prosecutions by her office are just for weapon charges. It completely dwarfs the second biggest category, DUIs, by more than 8x. Is this indicative of a soft prosecutor that is lax on weapons charges? Well that depends compared to what? But knowing nothing at all about what the “typical” prosecutorial caseload looks like, my impression from the findings above would be that Foxx’s office is completely obsessed with prosecuting weapon crimes.
But this doesn’t need to be the end of the inquiry. Perhaps the thesis is that Foxx is lax on prosecuting all crime, but especially weapon crimes. Or perhaps Cook County has an unusually high level of weapon crimes, and so a 60% focus on weapon crimes is still too low. Those are all fair objections! And it helps in highlighting what a further investigation would look like.
None of this is meant to serve as epistemic quicksand. The amount of work required is certainly not trivial, but this is not a hopelessly unanswerable question. Quite a few prosecutor offices post data publicly already, and most of the rest of the information could be easily requested and analyzed (by say, for example, a law school professor) into some compelling findings.
Also, I’m mindful that I’m critiquing a tweet that was either intended as or was just an off-the-cuff hot take. Nevertheless, if someone’s goal is truth-seeking, then more (and potentially better) information should be warmly embraced. But if someone’s goal is just to confirm the theories they want to be true, then my intervention is appropriately seen as an annoying gadfly.