Friday, November 5, 2010

Robocalls are exciting for nerds

What an exciting election! Err, at least for those of us watching these things closely. I think more people were excited about the World Series than all the gubernatorial races combined.

To me, the most interesting thing about the elections wasn't any particular race; it was the use of thoughtful measurement to help run more effective races. I'm writing this blog to explore how this measurement works, and how you can use it to run political campaigns more effectively. What if, instead of guessing whether you would get more votes from buying robocalls or from buying billboards, there was a way to measure the effectiveness of one against the other? Or if you could target your fundraising more effectively, so that you spent less money on direct mail appeals and saw great donations? Or if there were a way to figure out who your strongest supporters were and activate them to do more for your campaign or cause?

Here's an example of this kind of measurement that I helped design. I'm pretty excited about it. Many studies have examined whether reminding people to vote (getting out the vote, or GOTV) via a robocall increases voter turnout, and the answer is conclusively 'no'. But there's also research that shows that, if you call someone or ask in person to pledge that they will vote, they are more likely to actually cast their ballot than if you just encourage them or remind them without asking for a commitment. My old boss at Equality California (EQCA), Amy Mello, was buying robocalls from Switchboard to encourage EQCA's members to vote for pro-LGBT candidates. Amy is familiar with the research that shows robocall's ineffectiveness at GOTV, so she knew that the calls were more about generating support for particular candidates than increasing voter turnout. But Switchboard suggested that Amy buy a slightly more expensive call that allowed people who received it to press a button on their phone keypad as a pledge to vote. The recorded message said something like "Press 1 if you pledge to vote by mail, or press 2 if you pledge to vote on election day." Amy had the brilliant idea of testing whether the more expensive "pledge" robocall had a GOTV effect.

Here's how we designed the experiment. 41,000 households were in the target list to receive the call. We randomly chose 3,600 phone numbers that would not be called; this was our "control" group. The remaining phone numbers were randomly split into two categories: one received a standard robocall ("standard") and one received the "pledge" robocall. By comparing what percentage of people voted in the "standard" and "pledge" groups to the "control" group, we could see if the calls had any effect, and whether the "pledge" calls were more effective than the "standard" calls.

So what was the result? Well, this post is just a teaser. We don't have the individual-level voting history in our database yet, so we can't find out. Keep on reading and I'll let you know as soon as I have an answer!