ARLINGTON, Va. — The last eight years have been quite a ride for Nate Silver.
Before he ever got into politics, Silver built player projection models for Baseball Prospectus. They still exist under the name he bequeathed them, PECOTA, an inside joke aimed at only the most down-the-rabbit-hole of baseball nerds.
But when he took his statistical models and applied them to politics, his projection system crushed the analysts in the 2008 presidential election. His site was bought by The New York Times, where he further cemented his reputation by nailing all 50 state projections in the 2012 presidential election.
Silver’s site, FiveThirtyEight.com (or 538) has since moved under the ESPN umbrella, where it has expanded substantially both in size and scope, diving into the world of data journalism. But as election season heats up, many turn to Silver to find the signal in the noisy world of punditry to help them make sense of it all.
Silver was in town Tuesday for a conversation with Tyler Cowen at George Mason University’s Mercatus Center. But before he took part, he sat down with WTOP to talk about the difference between the Times and ESPN, “horse race” politics, the trouble with polls, why NFL coaches keep kicking and punting, and why PECOTA hates the Kansas City Royals so much.
Noah Frank, WTOP: What’s been the biggest difference about being under the ESPN umbrella as opposed to when your site was owned by The New York Times?
Nate Silver: I mean, there are a few fairly obvious things, and one probably is the scale of the operation. We’re 25 or 30 people now — at the Times we were effectively two or three full-time equivalents with the ability to get help from other people.
You spend a lot more time doing editing and management, but the flip side of that is you also have people who can do things on their own. I mean, that’s really fulfilling, when I don’t know something is going on until the end, and then I’m like, this is really amazing work. That’s more fulfilling in some ways than when I have to be, like, very micromanagy and hands on.
But they’re different climates, too. I think ESPN is somewhat more entrepreneurial than the Times. I think also 538 has a much kind of more ambitious mission where we want to be a focal point to cover all sorts of topics and not just bring in the nerds at the end. So that’s different. And the culture is different.
I’ve made the comparison before where ESPN is more like a big state school where people are pretty good, but you’re always aware of the scale. But it’s unpretentious and people reward success.
Where The New York Times is an Ivy League school. You have lots of fascinating conversations, almost everything they do they’re very good, but there’s a bit of, I don’t know, a certain type of elitism, I suppose (laughing). A weird type of, not close-mindedness, but there’s a lot of confidence at the Times that the Times way is the right way to do things.
You’ve recently tweeted about 538’s steady growth. How much energy do you invest in analyzing your own data, and what do you think has helped you be successful in creating sustained traffic where other sites may have failed?
It is different. I think I practically never looked at any traffic data at all when I was at the Times. I think traffic is a useful measure, up to the point where you’re trying to manipulate your journalism to win traffic. We’d rather have a story that 200,000 people respond to than 20,000 or 2,000.
At the same time, I believe in the long run, differentiating yourself with being a site people go to and know there’s a certain baseline level of quality, they know they’re going to get something different. We hope it’s right, too, but at the very least, the perspective you get on the election at 538, or on the NBA, is different than you’re seeing at other places.
We’ve had fairly steady growth and I think the feeling right now in a lot of media circles is you must be either really tiny or really big. And we’re kind of a weird outlier — it’s weird to be an outlier when you’re medium, right? But there are not that many medium-sized sites. If we’re kind of medium on the verge of large, then economically that’s more helpful for us than medium on the verge of small. We think we kind of are medium on the verge of large now. But still, we want to stand out in an environment where there are dozens of media outlets competing for sort of the same turf.
You talked about covering a broad range of topics. I looked through the archives, and there have been only 10 articles on FiveThirtyEight with the “music” tag. Do you think it’s possible to quantify musical trends and styles, and what are your thoughts on algorithms like Pandora’s and Spotify’s Discover Weekly?
So, I’d love to do more with music. More of the choices than you would think are the result of our main culture writer is a big movies guy and not as big a music guy. So things that people think are highly deliberate choices are not necessarily. But we would like to do more with music. We did a big thing on Drake and Nicki Minaj recently. And those stories do well, but yeah — all the areas where people are like, “I wish you covered this more,” I probably agree, for the most part.
Someone recently asked Matt Taibbi this same question, but Ralph Nader said back in 2004 that if Americans brought the same discerning eye to the political process that they do with sports — understood the metrics and statistics and dynamics — we’d be much better off as a country. Is that essentially what you’re trying to do with 538? Is there a way to make politics exciting without cheapening it?
I mean, certainly some of the original conceit of 538, back when it was kind of just me back in 2008, was that I had written about baseball for several years during the early Moneyball era, if you were to call it that, and was frustrated that you had very little of that in politics. It was all about gaffes and momentum, where “momentum” usually means what was happening a week ago and not what was happening next week, which is kind of what it should mean potentially.
The difference, though, was that in politics you always had people who were close to the campaigns who are very analytical. People know [about] Obama and his analytics team, or Ted Cruz this year. But Karl Rove and George Bush were pretty good at that stuff, too, in 2000 and 2004.
So, it’s more really the way politics is covered by the media. And for better or for worse, this is the part that gets us in trouble sometimes, but there’s a fairly explicit critique of “horse race” coverage at 538. It’s not that we don’t care about the horse race, that we don’t care about the polls, but people are kind of taking data and looking it at ways that are not very revelatory and can be pretty superficial.
Your model has buffers built in to accommodate for and correct bias in polls. Have you seen any reaction from the more biased polls in the way they ask questions as a result of your ratings of them? If so, how have you adjusted your own model to compensate for those changes?
I was going to say you don’t want polls to try to be more accurate, but look — there’s an issue called herding, which means some pollsters seem very concerned about being close to the average, close to what Real Clear Politics says, or 538, or pollster.com. That’s really dangerous. It turns out that statistically, what happens when every poll tries to be close to the average, the average itself becomes less accurate. You have less independent information. It’s as though there’s really just kind of one poll and everyone gloms onto that. And so, trying to avoid piling on a pollster when they show an outlier result.
Statistically, there’s way more variance in polling than you would think. If the race is really at, say, Clinton +5 in a state and you have 100 polls that say that, you’re going to have polls that say Bernie up 7 or Clinton up 21. And the impulse is, oh, the pollster must have done something wrong, but a good pollster will actually publish results that it expects are an outlier. Maybe it will go back and then survey the same state again. But yeah — frankly I worry that sites like 538 have a slightly negative effect on the culture of polling.
But I’ve also kind of evolved myself a little bit, where in the earliest incarnations of 538 it was a little bit, “we don’t care how you do your poll as long as you have accurate results.” My view on that has changed a little, in part because some pollsters who were very good in 2008, 2010, the first couple years of the site, have been very bad since.
And I’ve also found that, to the extent you can quantify methodology, there are a couple of basic things you can do. For example, is a poll a member of major polling organizations that require basic levels of disclosure and transparency? That turns out to be predictive. That a poll can have a bad year, but if it applies a good methodology, it will probably end up being pretty good in the end, and vice versa, too.
I’ve written about it this year, and 538’s own Benjamin Morris has written about it as well, but the NFL is still rejecting basic probabilistic math every Sunday in regards to punting and 2-point conversions. I spoke to ESPN’s Brian Burke about why this is, but I’m curious to know whether you think we’ll see sweeping change and acknowledgment of this at some point and, if so, what will that look like. Will there literally be a staff member charged with advising the head coach of the percentage play given the situation?
I mean, you need to change the culture, and I think that a lot of cultural change is gradual, rather than something that happens overnight, necessarily. It is kind of perplexing how much the NFL is willing to give away in terms of expected value.
But I have a friend who advises an NFL team — and I can’t mention which team — and they were trying to give their coach a contract extension. And in the negotiations, [they] said “Oh, by the way, we want you to apply better strategy on fourth down.” And he said, “Give me my extension first, and then I’ll consider it.” Because he’s worried that even if he knows it’s right, or could be led to believe that it’s right, you’re swimming upstream against a lot of people who think it’s wrong — a lot of conventional wisdom. And — I don’t know, I think people kind of underrate the importance of culture, and how hard it can be to overcome.
A lot of what happened in baseball is, you had people like Bill James, who back in the 1980s were laying the groundwork for this stuff. And so it’s kind of this generational change, where you might have a general manager who grew up working for an analytics site and is at least fluent in that vocabulary. In that sense, if you saw change at the high school and college level — although, even there, college coaches are probably a lot better about fourth down strategy than NFL coaches are.
Do you enjoy the irony of PECOTA’s hatred of the Royals (the primary team of former big leaguer Bill Pecota, after whom the system was named) consistently backfiring, and is this the year the system finally gets it right (projected 76 wins) and KC comes crashing back to earth?
If you’re not counting for how bullpens are used, and that’s a big part of it, I think there are things to roster flexibility that can matter a lot — you saw how hard the Royals worked to exploit in the postseason. You know, I don’t know. I handed over the reins on PECOTA a long time ago. I would think when you have a pretty big miss a couple years in a row you then want to do due diligence on those questions at least, and say, “Are there things we’re not thinking about?”
One problem, too, with making preseason predictions is that you consider a roster as being static and not how it might evolve over time. So a team with flexibility, and depth, and good management, there might be a way to measure that, potentially.
At the same time, we’ve looked at over 12 years of data, and over 12 years of data, PECOTA has done a fair bit better than Vegas at team over/under lines. It gave something back last year, I think — it was a pretty bad year for projection systems. But over time, to beat handicappers is a pretty impressive and high bar to clear.
Politics seems like such an all-consuming topic for 18 months before every presidential election. Do you find yourself wishing you had time to focus on other areas of analysis?
Yeah, for sure. I’ve become less of a baseball fan, because every other year the October baseball peak coincides with either a midterm or a general election peak, whereas with the NBA, you can watch playoffs in the spring, when I usually have a little bit more time to relax. It is tricky with politics becoming so all-consuming and how early it started this year. There was so much interest in July, even, about Donald Trump, and increasingly about Bernie Sanders, that it’s pretty exhausting. It’s good if you’re running a website.
I had beers with my colleagues the other day, and I’m like, “I can’t believe they’re making us work on Saturday.” There was another debate. And they’re like, “Well, Nate, you’re the boss.” And I’m like, “F, yeah, I am, right.” But if you’re trying to cover politics for a living, then it’s amazing in one way. And this has become the most fascinating election we’ve seen in a long time. But it is a little bit burnout-inducing, too, where every day there is some type of news.
