A couple years ago I got conned into working on a political campaign. This wasn't my first time volunteering on a campaign, but it was the first time that I worked with someone who I didn't know personally from before the campaign. We worked our butts off, put in all kinds of time that I'll never get back. Sadly, though, my record of having never won a campaign still stands.
About halfway through the summer I got curious about some things and I decided to try to model the outcome of the election. I set up my spreadsheet with columns for each of the precincts that would be reporting. Then I dug through the Secretary of State's website to get information about the minimum and maximum number of voters in each precinct and how much each one favors one party over the other. My approach was very similar to the process that Nate Silver has used to make his now-famous forecast that there is a 70% chance of Barak Obama winning a second term. And my results back then were very similar to what Silver is reporting today. My candidate was probably going to lose.
The reason I'm thinking about this today is because of an article I read about betting on the outcome of our models. The article can be found here: http://marginalrevolution.com/marginalrevolution/2012/11/a-bet-is-a-tax-on-bullshit.html
The article has a profound thought that I had to copy up to Twitter about a bet being a tax on bullshit paid by the bullshitter to people with genuine knowledge. But right now as I'm writing this I'm more taken in by this quote: "A blind trust bet creates incentives for Silver to be disinterested in the outcome but very interested in the accuracy of the forecast." The idea is that person making the bet should be disinterested in the outcome in order to ensure that passion doesn't start to make the odds seem different. It's like our passion about the outcome forms a fog that makes it difficult to see the mistakes we're making or the biases we're introducing.
After I saw that my candidate was about to lose something inside of me knew that I had wasted a lot of time, and that if I didn't quit the campaign I was going to waste a lot more time. I also knew that it would be very unprofessional to quit the campaign so I was pretty much stuck wasting my time. And I was also a little disappointed because I didn't WANT to lose. And then something happened; I added another parameter to the model.
You see, that year the conventional wisdom was that people from our political party were going to do much better than usual. So I added a variable which was how favorable our party turnout was going to be. Then I correlated that variable with our party turnout in each of the precincts. So if my random variable came up that we were going to have a good or very good year (which was more than half of the time) it would adjust the parameters for each precinct to give us a more favorable outcome. I ran my simulation again, and things got better. But we were still most likely going to lose.
Then I added another variable, this time to reflect how hard we had worked in a precinct. Surely if we were working hard in a precinct we would get a bump there too.
On and on this went, until I was exhausted. And at some point I realized that my projections had turned to pure shit. I had skin in the game; I was not a disinterested better, and I needed to find a way to justify my bet. So I kept skewing the model to make it more likely that I hadn't wasted a bunch of time.
I never shared my projections with the candidate, and when election day came we got kicked about as hard as my original forecast had suggested. In a sense I could have been right if I had just stopped with my first model. And since I never went on to publish or promote the monster that my forecast mutated into, I was never wrong. Instead I was almost right and almost wrong. But I think this story illustrates the value of getting parameters and projections from dispassionate people that don't have any personal interest in the outcome.
1 comment:
Is a linear fit the best here?
Looks a bit a sif there's a rather strong linear fit between 8000 and 16000, and with a steeper slope.
If that is correct, the outliers would be worth examining: why does state 25 do so relatively well on so small spending, and why does Alaska and 'V' does less well with larger spending?
Sometimes the outside or outside 2 values are dropped before a fit is made on the theory that statistical outliers are just that.
Post a Comment