A Response to Deceptive Claims About Draft Academic Paper on Ranked Choice Voting and Turnout

There is good reason to believe that ranked choice voting (RCV) helps racial minorities achieve fairer levels of representation. In 2013, for example, four cities held contested RCV elections, and all four saw gains in representation of racial minorities, including the election of two Latinos, two Hmong Americans and a Somali American (all for the first time in those cities). People of color hold 16 of San Francisco’s 18 offices elected by RCV, up from only nine before RCV’s introduction in 2004. Denise Munro Robb’s scholarly studyfound that RCV in San Francisco led to “increasing minority representation and greater participation rates at the ballot box,” particularly in contrast with the low turnout runoff elections previously held for the Board of Supervisors. Such outcomes are consistent with a long history of RCV elections working well for racial minorities in cities such as New York and Cincinnati, as reviewed in a 2008 analysis by the New America Foundation and FairVote.

However, public officials and others in cities that have debated RCV have once again been spammed with email from Terry Reilly, a San Jose based opponent of ranked choice voting with a track record of deceptive attacks on RCV.[i] Reilly's latest irresponsible and false claim sent to his national list is that a new study by an academic at San Francisco State University “shows conclusively that IRV/RCV decreases voter turnout in historically disenfranchised voting groups.”

Reilly is referring to an unpublished and recently withdrawn draft paper by San Francisco State University professor Jason McDaniel. McDaniel had posted the paper to SSRN, a website for social science researchers to post preliminary studies for feedback from other scholars prior to more rigorous peer review or publication. McDaniel’s paper was flawed, as we explain in a detailed critique (available upon request by emailing me at dspencer [at] fairvote [dot] org). He relied on much too limited data to possibly reach firm conclusions as to the effect of ranked choice voting on turnout generally and racial minority turnout in particular, especially since it contradicted the bulk of evidence suggesting that ranked choice voting works well for racial minority populations. He only looked at five elections for a single office in a single city, just one of which was a meaningfully contested ranked choice voting election.

As a result of our critique and Reilly’s publicizing of the paper, McDaniel withdrew the draft from SSRN. He wrote to FairVote that this version of the paper was not intended for promotion to the public and said that any interpretation of his analysis should be treated “with caution."

We do hope that McDaniel engages in extensive revisions. He applied sophisticated statistical tools to his limited data set and found a string of correlations, but they are best explained by factors other than the introduction of ranked choice voting. For example, McDaniel completely ignored the fact that voter turnout in mayoral elections has sharply declined in many big cities in recent elections, including far more steeply in Los Angeles; in fact, San Francisco’s 2011 election had the highest turnout of any mayoral election in the nation’s 22 largest cities, as of a FairVote review in 2012.

The limitations of McDaniel’s data become plain upon full review of the correlations he found. McDaniel’s most touted claim was that RCV hurts turnout among black voters, but his analysis also found statistically significant results purporting to show that it hurt turnout among white voters and actually helped turnout among black voters over the age of 65. The paper found no significant effect for Latino or Asian American voters (in 2011, Asian American turnout in San Francisco soared to an all-time high and Latino turnout was its highest in decades).

Furthermore, while McDaniel suggests that RCV leads to more overvotes, he does not address the fact that there were more than five times as many ballots invalidated as overvotes in the non-RCV June 2012 Senate primary in San Francisco and Oakland than in the comparably contested RCV mayoral elections in those cities. Furthermore, there have been notably fewer undervotes in Board of Supervisors elections since the introduction of RCV, meaning that more voters who go to the polls to vote for president or governor are also voting in city elections and are not put off by the RCV ballot. Additionally, turnout has been significantly higher and more representative in RCV elections for the Board of Supervisors and non-mayoral offices as compared to previous runoffs for those offices. The 2008 study by FairVote and the New America Foundation found that racial minority voters have had no trouble ranking candidates. Meanwhile, mounting evidence suggests that RCV enhances the civility and positivity of campaigns, and that voters appreciate that difference.

Although no system is perfect and San Francisco’s implementation of ranked choice voting can be improved, RCV in fact has many benefits for participation. Inflammatory claims about RCV, especially when originating from agitators like Terry Reilly, must be treated with great skepticism. More academic research would be helpful to fully determine the effects of RCV on voter turnout in the U.S., but mischaracterization of a single preliminary draft paper should not be taken seriously.

---

[i] Reilly’s unprincipled opposition to RCV dates back to at least 2007. He has engaged in a campaign of multi-media attacks on ranked choice voting around the country. Several posts at IRVFactcheck.com are in response to his attacks, and a FairVote intern made a video in 2011 going through one of Reilly’s videos in a point-by-point example of how he manipulates facts. Terry Reilly is also cofounder of Friends of the San Jose Rose Garden.

Alleged Racial and Ethnic Disparities in Oakland

Who: Alexander Holtzman

What: Stanford University honors senior thesis

IRV FactCheck Rating:  NOT CREDIBLE

Ranked Choice Voting (RCV) opponent Terry Reilly has recently been distributing and promoting by email and news article a 2012 undergraduate thesis by Alex Holtzman, titled “The Unanticipated Inequalities of Electoral Reform: Racial and Ethnic Disparities in Voting Behavior under Oakland's Ranked Choice Voting Program ”. 

As a result of his paper, Holtzman was one of twenty-eight students awarded Stanford University's Firestone Medal for Excellence in Undergraduate Research in 2012. Terry Reilly has highlighted the paper in support of RCV repeal efforts in Oakland and San Francisco (both repeal efforts failed) and to discourage the use of RCV in other parts of the country.

You can probably look just about anywhere and find voting behavior differences along some set of demographic characteristics, regardless of which election method is being used.  However racial and ethnic differences still deserve special vigilance.  But racial and ethnic disparities are generally known to still exist with traditional election methods.  So what should be of particular interest in evaluating RCV implementations is not just whether flaws exist, but how any flaws have changed, what are the causes of those changes, and what are good ways to mitigate the flaws and continue the improvements.

Much of Holtzman's quantitative analysis purports to compare Oakland, California voter turnout data for the mayoral contests in 2006 and 2010. The 2006 contest used a traditional June/November runoff system and was won in June; there was no November runoff. The 2010 contest was decided with one RCV election in November.  In both years, local elections were consolidated with state-wide elections in June and November.

Overall Holtzman finds mixed support for his hypothesis that an alleged increase in complexity associated with voting under RCV causes a decline in voter participation, especially among traditional racial and ethnic minorities. The effects he finds tend to be small and sometimes don't support his hypothesis. But a bigger problem is that his quantitative analysis appears to be seriously flawed from the start.

Holtzman does not actually use any turnout data from the June 2006 mayoral contest. Instead he uses November 2006 turnout data, when there was no mayoral contest, and compares it to November 2010 turnout. He also uses overall turnout data for each election rather than contest-specific data. As a result, he finds an overall increase in turnout of only 1 percentage point. There was actually an increase of 14.1 percentage points between the two mayoral contests (46.0% for June 2006, 60.09% for November 2010, a 30.6% relative increase).

A quantitative analysis that starts with grossly erroneous data and understates the overall effect by more than an order of magnitude (1% vs. 14.1%) just can not provide a credible analysis of the components of that effect. This problem with the choice of data spoils both the within-city analysis of Oakland alone and also the difference-within-difference model that additionally introduces turnout data from Long Beach, California.

The principle claim that RCV proponents make about increased turnout, compared to a traditional two-election runoff system, is not that RCV necessarily brings more voters out to vote, but that it allows the entire contest to be decided in a single election that can be scheduled when the most voters are likely to be voting anyway. RCV helps avoid the one lower-turnout election. In Oakland, the June primary was the low-turnout election. In San Francisco, which used a November/December runoff system before upgrading to RCV, the December runoff was most often the low-turnout election. By using the wrong data, Holtzman's paper fails to investigate this claim.

The 2006 versus 2010 mayoral comparison for Oakland has been written about previously, both overall and by city council district, showing significant improvements in voter participation.  Perhaps the best overview of how RCV has helped increase voter participation is a poster produced by the Ella Baker Center for Human Rights and Oakland Rising.

Holtzman also does a ballot usage analysis that only looks at the Oakland 2010 mayoral data. However his analytical framework is flawed and too often he extends his conclusions beyond what his data shows. Marking a choice for only one candidate does not mean that the ballot will be exhausted or that the voter is disenfranchised or confused about RCV. Holtzman does not investigate actual vote exhaustion rates. Without comparisons to Oakland's experience with its traditional two-election runoff system, the results Holtzman does find beg the question of whether the RCV experience was an improvement. RCV often has lower rates of exhausted votes than comparable contests using a traditional two-election runoff system. The advantage tends to grow as the number of competitive candidates increases and is evident in Oakland.

Because of these and other serious flaws with the paper, most of the core results should not be relied upon and likely are significantly wrong. I contacted both Holtzman and his thesis adviser, Clayton Nall, to share my concerns about the paper. Both declined to discuss the substance of those concerns.

So Holtzman's undergraduate thesis gets an overall IRV FactCheck rating of NOT CREDIBLE.

Some of the ancillary results in the thesis may have some validity, but without a demonstrated comparison to traditional runoffs and a consideration of compensating effects, neither those findings nor the thesis as a whole support Terry Reilly's contention that RCV is a failed reform.