21 Comments
User's avatar
Robin Epstein's avatar

Clearly, we need to be taken into these weeds! Up until now I was blissfully unaware that some people - some powerful people - argue that destroying OFA was a good idea. Good grief. I thought it was just that folks didn’t know about it, and didn’t understand how it led to us not competing in and thus losing so very many states. As for your question about why he and his ilk are powerful, is his daddy rich? Was he college roommates with a senator’s son? Did he date a Cuomo or a Kennedy? I’m sorry. I know that’s obnoxious. But seriously. Why????

Doran Schrantz's avatar

Thank you for writing this.

Justin Ruben's avatar

Micah - appreciate the overall point here about OFA - super grateful to you for surfacing it. One tiny point - i'm not a statistician but the 'intent to treat' approach you talk about that this paper uses is 100% the standard approach for measuring voter turnout experiments, and it doesn't seem like a ridiculous way to measure the effectiveness of a persuasion experiment either (though it adds noise)... i'm not sure that's the grave flaw you make it out to be. seems like the larger point is that this paper suggests there's weak evidence that OFA's persuasion program backlashed with one group of voters in one state - which is not remotely surprising if true (as you point out, this happens, and if anything it attests to the trickiness of strangers doing voter persuasion) and says little to nothing about the potential impact of an independent OFA, as an organizing and campaign entity, on the world... If we decided to shut down every group (or campaign) that ran a GOTV or persuasion campaign that had the opposite effect to what they intended with a group of voters -- there would be literally none left.

Justin Ruben's avatar

sorry - i missed the medication discussion above - don't mean to rehash. (i'm a nerd, who my family tells me has a strong pedantic streak :-) so i feel compelled to point out that if you had two groups of 1000 people and you gave one of them medication but you knew only a fraction took it and didn't know which ones took it, and the other was a control --- and the group with some folks who had the medication got sicker on average - that would be suggestive evidence that the medication wasn't helpful, at least as prescribed). in the OFA case, the effect they found in this study was not very signficant which is why they are very soft in their language. meaning, could very well just be noise but also an indicator that it wasn't likely winning over tons of votes. again -your larger points stands.

Robin Epstein's avatar

My comment at the top was snarky and I’m glad the discussion got more substantive. But the importance of what you mentioned, Micah, about the “other” potential long-term benefits of canvassing can’t be overstated. I don’t know what OFA would have done in the off years and we’ll never know. But its absence, I think, led not only to us losing elections but also to our ideas disappearing from the discourse. The right never stops. They totally organize between elections and their people feel part of pushing for an agenda that reaches way beyond one election. We’ve not done that at sufficient scale. Nor in a way that connects the various orgs. But that’s changing! Also, to answer the commenter who asked how Zohran trained canvassers, it was about listening and about issues that mattered. And I wouldn’t be surprised if even before the election he and his team saw it continuing beyond Election Day. And, indeed, Our Time, comprised of Zohran volunteers, is canvassing this weekend for universal childcare!

Dan Ancona's avatar

[standing ovation.gif] - all of this Micah!

Jonathan Stray's avatar

I'm not convinced your objections to the study are warranted. The "intent to treat" effect (ITT) seems to me to be exactly the number you want to measure -- campaigns can't make people answer their phone or doorbell, they can only make volunteers call. So ITT is what you are going to get for your money, not the effect of an individual conversation multiplied by the number of volunteers. Those failed attempts to contact are not even necessarily neutral, as they could plausibly annoy people.

Also, Shor cites internal 2012 evidence. Now of course that's just his word, but if true it's an independent line of evidence that confirms this study.

More fundamentally, you're reading Shor's single tweet as a dismissal of all grassroots organizing. I'm not sure he's saying that. Even if OFA really was elecorally net negative, it's not clear how well that is going to generalize to all organizing.

And your argument is just as easily flipped around: what is the evidence that OFA was net *positive*? What is the evidence that any organizing has been net positive electorally, and that it will generalize to future campaigns?

I get that you have strong priors on the value of organizing, but I would argue that we dismiss quantitative work at our peril. And using Shor as a stand-in for distaste of what polling in general is telling us feels a little too close to shooting the messenger.

Micah L. Sifry's avatar

The study's authors say that the treatments (a canvass, phone call or mail) caused their treatment group to like Obama less. But they can't know that. This is like saying -- we were studying whether a medicine helps cure some condition. So we sent 1000 patients a prescription for it. We're not even sure how many got the prescription, let alone how many took the pills as required, but we're estimating that some did. But we can see that since then, these 1000 patients didn't get better and some got worse. So we know this pill doesn't help cure the condition.

Well, maybe?

As for the argument about OFA, with all due respect, if this isn't your field of study, that 2008 campaign is considered the gold standard (until Mamdani 2025) for serious field organizing. See https://www.amazon.com/Groundbreakers-Million-Volunteers-Transformed-Campaigning/dp/0199394601 For Shor, who right now sits at the pinnacle of the Democratic strategist ecosystem, to dismiss OFA 2008 as unimportant or worse, negative in its effects, is a big deal. This isn't just an argument over a tweet.

As for evidence about "any organizing being net positive electorally" -- I'll agree with you that we tend to exaggerate the effects of any tactic. The effects of ads decay fairly quickly and most studies show most other tactics (canvassing, phone, text) have minimal impact in big national contests. Deep canvassing and relational organizing are somewhat more impactful, but in a presidential contest voter attention is already relatively high and hard to shift. Party ID is a very strong predictor of voter choice, by comparison.

At the end of this post, I pointed to some fairly new work studying the impact of independent power organizations that focus far more on long-term relationship building, community organizing and structured leadership development. There are signs that power metrics like "number of leaders developed" and "number of super-leaders" correlate not just to greater voter turnout among the people these groups contact, but also greater likelihood of local/state policy wins. In other words, that a different approach to organizing that is more structured and relational builds actual power the leads to changes in people's lives.

Jonathan Stray's avatar

Just to get this out of the way. I'll happily concede winning elections isn't my primary field. There's doubtless background knowledge and especially context around this discussion that I don't have. I think I came off stronger than I meant to initially.

But evaluating statistical evidence for persuasive effects is my field, and I do think intention-to-treat (ITT) is the right thing to measure. The alternative, "treatment effect on the treated" aka "average treatment effect on the treated" or ATT is a sort of ideal case result. You don't ever get that in reality. In this case, you don't ever get 100% response when you canvas/call. In the case of medicine, you don't ever get 100% patient adherence to a protocol. The other serious problem is that the people who were more likely to answer the door/take the drug might be different in some important way (selection bias) so it's very easy to overestimate what would have happened if there was more compliance. If want to know what you'll actually get when you "prescribe a treatment" then ITT is what you want. This is why ITT and not ATT is the standard for FDA approval for drugs. See https://pmc.ncbi.nlm.nih.gov/articles/PMC2921714/

So now we are in a situation where the 2008 OFA ITT effect is estimated to be zero or maybe negative, and Shor says there's other evidence that the 2012 OFA effect is negative. Assuming Shor is accurately representing the 2012 study, in the face of multiple lines of evidence, I believe one is forced to say that for this style of persuasion to work electorally you would need to do something different.

Of course, as you note, there may be other, non-electoral effects that are positive. Or maybe building these relationships will have positive effects in future elections.

But is there another quantitative estimate of OFA effects on election results? If not, I don't see why we should assume that it was net positive. This hardly rules out organizing, but it does suggest that we need to take a second look at assumptions about the "gold standard."

Thank you for your time engaging seriously. I realize that to you I am just some guy, so I appreciate it. I would actually love to connect with you further, I think we have similar goals but divergent methods, which sounds like an opportunity for learning.

Bob Fertik's avatar

There is a vast body of research into the effects of all forms of voter contact including advertising (across all media) and voter contact (across all channels). Much of that research is gathered by the Analyst Institute (https://analystinstitute.org/) which is open to everyone who "work full-time in, or fund,the civic engagement community."

I am no expert but I have never read a conclusion that ALL canvassing backfires, which seems to be the conclusion of this 2008 study quoted by Jonathan Robinson.

If Robinson's point is that only OFA canvassing backfired for some unstated reason, that reason would need to be explained.

In any event, this alleged backfiring is irrelevant to the question of whether OFA should have been sustained as an independent mobilizing force or shunted off to the DNC where it atrophied.

Gaspard concluded OFA should have been sustained. Isn't that enough to know?

Bob Fertik's avatar

As an anecdote, I was present at a large meeting in 2012 where Jim Messina announced OFA would be revived after its 4 year hibernation to help re-elect Obama.

So clearly Messina was not persuaded by the 2008 study that OFA canvassing backfired.

As a result, OFA ran a large-scale canvass operation in 2012.

OFA 2012 did not have the same passion as OFA 2008 because Democrats lost badly in 2010 after the grueling ordeal of passing Obamacare, where the top progressive priority (a public option) was killed by Joe Lieberman.

Nevertheless, OFA 2012 managed to get Obama re-elected.

Jonathan Stray's avatar

Thanks for the Analyst Institute link, that's a useful compendium. And I'm not at all saying that all canvassing backfires! I'm saying that on balance, we should believe OFA backfired. So what could we do different and better?

I actually do think organizing is useful, I was trying to make the rhetorical point that the evidence for that is unlikely to be stronger than the evidence that OFA didn't persuade voters. If you think that's incorrect... lets see the evidence and talk about it!

Gaspard's statements are not good evidence in my book. It's one guy's opinion. Other people had other opinions. We'd have to look at some sort of outcomes here.

The other point that is getting buried in all of this back and forth is that, of course, building a political organization potentially has benefits beyond the electoral.

What I'm actually advocating for is ruthlessly integrating what we can know about what actually wins -- whether quant or qual or whatever -- and not getting this confused with what we would like to win or what feels good to do. The easiest person to fool is ourselves.

Micah L. Sifry's avatar

This reminds me of Mike Bloomberg's slogan -- when in doubt, bring data. Yes, sure. But Patrick Gaspard isn't just "one guy" with an opinion. I don't know your field of expertise, Jonathan, but I doubt that people in it would say that about, oh, I don't know, Steve Jobs. I suppose we could build a model where we ask 10,000 political professionals to rate the expertise of each other and than weight the ones with greater rankings so we listen to the right people more. And on balance, OFA didn't backfire until AFTER it helped elect Obama, and the reasons it backfired were political, not data-driven.

Bob Fertik's avatar

Candidates and their supporters have been persuading voters through speeches and canvassing since the first election in ancient Greece.

When they won, no one ever asked them to prove - mathematically! - which of their techniques were effective and which were not.

That kind of "quant" thinking only emerged recently, driven by the 2008 publication of "Get Out the Vote: How to Increase Voter Turnout" by Donald Green and Alan Gerber, which they update every few years

https://www.amazon.com/Get-Out-Vote-Increase-Turnout/dp/081572568X#

When every campaign technique is put under the quant microscope, it turns out that canvassing (door knocking) is the second most effective method of increasing turnout, compared to phonebanks, textbanks, mailers, and paid advertising, etc.

The most effective method is relational organizing - voters encouraging their family and friends to vote.

These two methods get combined in the form of "precinct captains" who get to know their neighbors year-round and leverage that relationship to convince them to vote at election time.

For anyone who has ever knocked on doors, this is not rocket science and it doesn't take regression analysis to prove.

Bob Fertik's avatar

Notwithstanding this long and proven experience, the 2024 campaign discovered something new and different.

When canvassers for Kamala Harris knocked on doors, they discovered voters already had very strong opinions about her that could not be changed in the course of a single conversation.

That was because of social media in general, and the meteoric rise of conservative influencers in particular, led by Joe Rogan.

Those influencers did not arise organically - they were heavily funded by conservative groups and heavily favored by social media algorithms.

In the wake of 2024, progressive donors decided they needed to start investing in progressive influencers to try to level the playing field, led by media companies like Pod Save America, Meidas Touch, Chorus, and Courier Newsroom.

This strategy appears to be working, as Donald Trump's polls are collapsing and Democrats are overperforming in every single election.

Steve Schear's avatar

Micah's analysis seems correct to me. Consultants who are "experts" in polling and tv want to preserve their dominance despite their obvious failures to win elections in recent years. So they push out BS studies to deter investment in effective field operations. I rarely agree 100% with anything I read, but I agree 100% with Micah's analysis. It is brilliant.

gary krane's avatar

I love Micah Sifry's work! If we somehow bring democracy back to the federal government it will be significantly in part due to Micah's work being read by strategists, tacticians and as many working and middle class and even angry upperclass democracy loving people as possible. So please spread the word! Meanwhile, for those keen on putting the craft and science of mass nonviolent resistance on steroids, check out the "Feasibility [of a national nonviolent civic uprising and social economic strike] Manifesto" at tinyurl.com/NCUnow

What Is To Be Done's avatar

Micah -- so right again! I posted on this as well a few weeks ago -- making the connection to Mandami's efforts and expanding it what on the Right is the power they get from the MegaChruches. (Apologies for the shameless plug and Substack-bombing" -- I would be so honored if you followed What is to be Done https://substack.com/@xrastone)

"Then, as soon as his hand left the Bible on January 20, 2009, Obama turned his back on the people who got him there. He filled his administration with Beltway insiders—the very ones I’d feared Hillary would hire. Strategically, they decided the fate of the administration would be sealed by what they could get through Congress. Their primary constituency became 435 House members and 100 Senators.

Members of Congress were terrified of Obama’s volunteer army. Would it spawn primary challengers pushing the party left, as Sanders’ campaign later did? Or become a cudgel Obama could swing to enforce discipline, as Trump wields MAGA?

Rather than use the army to move Congress, they executed a bureaucratic Night of the Long Knives on it—shifting it to the DNC headed by Governor, now Senator Tim Kaine. The DNC, a creature of the party establishment and state parties, felt threatened by local organizing it didn’t control. Obama for America became Organizing for America and withered.

And because the DNC is the DNC, it tried to turn volunteers into donors. The fastest way to kill volunteering is to tell volunteers that work is fine, but money is better—especially if that message arrives in an endless stream of emails, mailers, and texts. Millions still get solicited because they signed up in 2008. Hell, they’re still using Obama, Carville, and Begala to sign the appeals.

Who recognized the power of a grassroots army to move Congress? The Koch brothers. They funded the astroturf Tea Party, and when its activists flooded town halls, the Obama volunteer army that could have drowned them out was gone—because there was no longer an Obama volunteer army."

Jen Just's avatar

Fascinating and important Micah. As soon as OFA was folded into the DNC - and in '09 focussed on issues rather than candidates - it lost the steam that powered it and no amount of field organ izing was going to make up for the loss of autonomy volunteers felt when "mybo" (Mybarackobama.com) was scuttled. Mamdani gets it, and I'm hopeful about what might happen next.

The Coop Scoop's avatar

Excellent. But here's one way of seeing the brighter side of Schor's rise:at least he has replaced George Lakoff as Head dem Guru.

Mary Russell's avatar

Safety in the tried and true.