The Spreadsheet Psychic
In this year's Democratic primary, for example, the polls were all over the place. Before the Iowa caucuses on January 3, one poll had Clinton winning by nine, one had Clinton by two, and one had Obama by one. Obama won by seven. In the New Hampshire primary, five days later, one poll had Obama by thirteen and most others had him winning by eight or nine. Clinton won by three. Primaries are notoriously difficult to poll, because unlike in a general election, turnout is very unpredictable and people are much more likely to switch their choice at the last minute. As the primaries went on, however, Silver, who had been writing an anonymous diary for the liberal Website Daily Kos, made an observation about this year's voters: While the polls were wobbling wildly state-to-state, the demographic groups supporting each candidate, and especially Clinton and Obama, were remarkably static. He wasn't the only one who noticed this, of course—it was a major narrative theme of the campaign. One pundit summed it up by saying that Clinton had "the beer track"—blue-collar whites, Latinos, and seniors—while Barack had African-Americans and "the wine track": young voters and educated whites.
Every other pundit, though, was doing what they've always done, i.e., following the polls. Silver decided to ignore the polls. Instead, he used this observation about demographics to create a model that took voting patterns from previous primaries and applied them to upcoming contests. No phone calls, no sample sizes, no guesswork. His crucial assumption, of course, was that each demographic group would vote in the same way, in the same percentages, as they had in other states in the past.
Like many of the so-called Moneyball breakthroughs in baseball, this was both a fairly intuitive conclusion and a radical break from conventional thinking. (In Moneyball, for example, the idea that players who get on base most often are the most valuable—which now seems kind of obvious—was a major breakthrough in strategy.) After all, political pundits love to talk about states as voting blocs—New Hampshire's leaning this way, North Carolinians care about this, etc.—as though residency is the single most important factor in someone's vote. Silver's model more or less ignored residency. But his hunch about demographics proved correct: It's how he called the Indiana and North Carolina results so accurately when the polls got them so wrong.
The Spreadsheet Psychic
Nate Silver is a number-crunching prodigy who went from correctly forecasting baseball games to correctly forecasting presidential primaries—and perhaps the election itself. Here's how he built a better crystal ball.
By Adam Sternbergh
In a month when the Dow had its worst single-day plunge in over twenty years, when Lehman imploded, AIG faltered, and WaMu failed, when the word crisis became an everyday staple in newspaper headlines and the presidential race pulled close, then pulled apart, when the Chicago Cubs kicked off a playoff quest to win their first championship in 100 years (then got swept out in three straight games) and, for good measure, some scientists in an underground lab near the Swiss Alps fired up a Large Hadron Collider that some serious observers warned might create a black hole that would swallow up the Earth, it was comforting to sit down and have lunch in midtown with a man who can see the future. It's not that Nate Silver is psychic, or even that he's right all the time. He's just proved very good, especially of late, at looking at what's already happened and using that information to predict what will happen next.
Silver, who's 30, thin, and lives in Chicago, had been flown to New York at the invitation of a hedge fund to give a talk. "They just said, 'Why don't you come in, talk about your models,' " he said with a shrug. "I'll probably just take a lot of questions." Silver doesn't know all that much about high finance; these days, he's spending most of his energy on his political Website, FiveThirtyEight (the total number of Electoral College votes), where he uses data analysis to track and interpret political polls and project the outcome of November's election. The site earned some national recognition back in May, during the Democratic primaries, when almost every other commentator was celebrating Hillary Clinton's resurgent momentum. Reading the polls, most pundits predicted she'd win Indiana by five points and noted she'd narrowed the gap with Obama in North Carolina to just eight.
Silver, who was writing anonymously as "Poblano" and receiving about 800 visits a day, disagreed with this consensus. He'd broken the numbers down demographically and come up with a much less encouraging outcome for Clinton: a two-point squeaker in Indiana, and a seventeen-point drubbing in North Carolina. On the night of the primaries, Clinton took Indiana by one and lost North Carolina by fifteen. The national pundits were doubly shocked: one, because the results were so divergent from the polls, and two, because some guy named after a chili pepper had predicted the outcome better than anyone else.
Silver's site now gets about 600,000 visits daily.
posted by Whitney Tilson at 1:36 PM
0 Comments:
Post a Comment
<< Home