Improving the Security, Transparency and Efficiency of California's 1% Manual Tally Procedures

UC Berkeley School of Information (2008-07-01) Joseph Lorenzo Hall

This Page: https://copswiki.org/Common/M1503
Media Link: https://www.usenix.org/legacy/event/evt08/tech/full_papers/hall/hall_html/jhall_evt08_html.html%20type%3d%22html%22
More Info: Election Audits, Election Integrity, Snapshot Protocol, Election Audit Lawsuit, Snapshot Protocol

Improving the Security, Transparency and Efficiency of California's 1% Manual Tally Procedures

Joseph Lorenzo Hall, UC Berkeley School of Information¹

Abstract:

California jurisdictions have an extensive history of conducting post-election manual tallies of ballot records; they have performed this type audit since the 1960s when the use of lever and punchcard voting technologies became common statewide. We report findings from studying manual tally procedures in a handful of California counties. Through a combination of iterative procedure development and observation of manual tally activities, we designed new procedures that better promote security, transparency and efficiency. We have since generalized these procedures for use in any California county.²

1 Introduction

Election auditing works to ensure agreement between what the voter sees, what the voting system records and what is counted by back-end tabulation systems. A key concept in voting system security and auditability is that of software independence whereby an ``undetected error or fault in the voting system software is not capable of causing an undetectable change in election results.''[ 2] Practically, this is achieved by testing the voting system before an election and checking the election results after the election.

In terms of testing, more than 40 states require voting systems be federally certified by independent laboratories before they can be used in live elections. These laboratories perform a series of tests and audits of the software, equipment and documentation to ensure that a given voting system conforms to the federal standards. Some states have found this process to be lacking and have decided to employ their own experts to further evaluate systems.[ 3, 4, 5]

In terms of post-election auditing, the dominant form of auditing currently in use involves comparing hand-counts of paper audit records with the electronically-recorded and tabulated results.[ 6] A small minority of states, only 12, allow voting systems that do not produce a paper record.³ However, only a handful of States have provisions that require routine hand-counts of these records to audit the electronic results. Even amongst the States that do this type of post-election auditing, they typically mandate a flat-percentage audit of 1%-20% of precincts, machines or election districts, randomly chosen. The state-of-the-art in post-election auditing, in practice, involves tuning the size of the audit to a desired level of confidence (or statistical significance) while taking into account the size of the units being audited and the margins of contests on the ballot.[ 9, 10] This type of ``tuned'' audit can help to ensure that jurisdictions do not needlessly waste time and effort. In the context of this article, audits based on statistical confidence will simply mean more or less hand counting.

2 Background and Motivation

2.1 The Manual Tally Process in California

California has been performing post-election manual tallies for over 43 years.⁴ At that time in the early 1960s, punchcard voting equipment, after lever machines, had become the next major voting technology. The type of punchcard that was best suited for large jurisdictions, like Los Angeles County, maximized the number of ballot positions per ballot; these punchcards had no candidate or choice names printed on the face of the punchcard. The new ``automatic manual recount'' process would serve as a check on the punchcard tabulation machinery in order to ensure that the ballot position number on the paper punchcard corresponded to the correct candidate in the tabulation program.<sup>5</sup> In fact, while the emphasis in current law is that the manual tally is not a recount (vote totals do not change in the tally but do in a recount), the original provision called for a ``public manual recount'' of ballots in 1% of precincts or no fewer than 6 precincts.⁶

California law specifies very little directly about the conduct of the manual tally. The legal definition of the ``one percent manual tally'' is:

[The] ``One percent manual tally'' is the public process of manually tallying votes in 1 percent of the precincts, selected at random by the elections official, and in one precinct for each race not included in the randomly selected precincts. This procedure is conducted during the official canvass to verify the accuracy of the automated count.⁷

For electronic voting systems, the law considers the voter-verified paper audit trail (VVPAT) the record of the vote for tallying purposes and the VVPAT governs if there is a discrepancy between the electronic and paper records.⁸ Finally, Sec. 15360 of the California's Election code specifies a number of high-level requirements for the tally:

• It is conducted over all races in 1% of precincts (or at least one precinct), randomly chosen.
• For races not chosen in the 1% selection, election officials must choose, not necessarily randomly, 1 additional precinct for that race and are required to tally only that race.
• It must include vote-by-mail (VBM) and early voting ballots.
• The elections official must use either a random number generator or a selection method specified in regulations by the Secretary of State.
• The tally is a public process and election officials must provide a minimum five-day public notice.
• The elections official must issue a report as part of its certification of the official canvass that identifies any discrepancies found and that includes a description of how they were resolved.

In addition to these legal requirements, the California Secretary of State, under her authority as chief election official of California, has imposed additional requirements including escalating the size of the audit and increasing scrutiny of certain types of voting systems. These additional requirements are a result of the California Secretary of State's Top-To-Bottom Review (TTBR) of Voting Systems and the Post-Election Audit Standards Working Group.[ 3, 12, 13]

In practice, the hand-counting method used by counties in California seems very similar. The typical tally team uses four people consisting of two talliers, one caller and one witness:

• The caller speaks aloud the choice on the ballot for the race being tallied (e.g., ``Yes...Yes...Yes...'' or ``Lincoln...Lincoln...Lincoln...'').
• The witness observes each ballot to ensure that the spoken vote corresponded to what was on the ballot and also collates ballots in cross-stacks of ten ballots.
• Each tallier records the tally by crossing out numbers on a tally sheet to keep track of the vote tally.⁹

Talliers announce the tally at each multiple of ten (``10'', ``20'', etc.) so that they can roll-back the tally if the two talliers get out of sync.¹⁰

2.2 Motivation

This type of post-election audit--the manual tally of paper records against results in the election database--has become increasingly important now that many states have adopted requirements that voting systems produce independent paper records. Such records do not serve their role as an audit trail if the records are not routinely examined as part of an audit.¹¹

In California, there have been problems in the past due to underspecification in existing legislation. While a catalog of such deficiencies due to regulatory underspecification is beyond the scope of this work, we provide a few illustrative examples. For example, only recently did the manual tally law explicitly specify that VBM ballots be included. With many counties in California now reporting 40-50% of ballots cast as VBM ballots, this had meant that a large fraction of cast ballots went unaudited. Also, the law only recently specified a method for random selection of precincts to be manually tallied.¹² Jurisdictions have in the past used, and continue to use, opaque methods of generating pseudorandom numbers--such as via software provided by the voting system vendor for this purpose--instead of publicly verifiable methods like those described by Cordero et. al.¹³

In addition to the underspecification alluded to above, there are other serious constraints related to timing and resources imposed upon jurisdictions that affect their manual tally. The most significant constraint is due to the time period in which the manual tally must be completed. California law specifies that the canvass, which includes the manual tally, must be complete 28 calendar days after the election.¹⁴ For smaller jurisdictions that do not have to count many ballots, the timing of the manual tally can occur promptly after election day. For very large counties, such as Los Angeles County, the manual tally process can take weeks. This is a serious constraint because, as we explain later, the integrity of the audit critically depends on all initial counting being complete before the tally begins; otherwise, the numbers being audited will change during the audit. Performing a high-quality audit also depends on resource constraints such as a jurisdiction's budget and available staff and space.

As our goal was initially to increase the security and transparency of the manual tally process, we recognized that our efforts would exacerbate these tensions. Our new procedures needed to be optimized, in a sense, to take advantage of possible efficiencies where election officials could save time and resources.

3 Methodology

In order to develop improved post-election audit procedures that could be used by California counties, we chose a non-standard and somewhat exploratory methodology. We go in more detail below, but essentially we did the following:

• We examined the manual tally procedures for a few California jurisdictions interested in improving their procedures.
• We developed an initial set of improved procedures through a series of iterative meetings with one such jurisdiction, San Mateo County.
• This jurisdiction then incorporated our improvements into their own procedures and used them in actual elections.
• We observed our collaboratively-generated procedures in action and noted deviations, improvisations and new issues that came to light only during an actual tally. We also observed the manual tally in a few other counties with which we have had only limited contact, Alameda County and Marin County, and worked closely with a fourth county, Yolo County.¹⁵
• Finally, we generalized the improved set of procedures into one set of procedures that can be used by any California county.[ 1]

We have been fortunate enough to collaborate with a number of election officials to improve their audit processes. Over the past year, we have worked closely in a larger multidisciplinary team¹⁶ with election officials in California's Alameda County, Marin County, San Mateo County and Yolo County.

These counties were interested, to different extents, in incorporating academic input about security and auditability into their post-election process. Our interaction with some of these counties was more indirect in some cases, when the county was interested in academic input on specific elements of their manual tally procedures.¹⁷

We decided to collaboratively redesign of the post-election audit procedures for one county, San Mateo. San Mateo County was willing to work closely with us to re-evaluate their procedures, was a fairly large county and was located close to our home institution. Our multidisciplinary team met with the election officials and staff of San Mateo over the Fall of 2006 and Spring of 2007 and this author observed the random selection and manual tally processes over a number of election cycles and developed the written procedures.

Our aim was to have a rich, iterative interaction where researchers learned about the issues involved in running elections and election officials learned about security, transparency and auditability while we both worked to align our two sets of expertise into concrete election procedures. As our collaboration progressed, our research team was able to highlight what we felt were the best practices in election auditing, from a scientific and policy perspective. In late Fall 2006, we drafted a first set of post-election audit procedures specific to San Mateo County that we modified as a group. These audit procedures were used as a model by San Mateo in the post-election audits of their 2006 and 2007 elections.[ 18, 19]

4 Findings

We were able to make a number of improvements to the procedures in terms of security, transparency and efficiency.

4.1 Security

The manual tally serves multiple security-relevant roles: as an audit process, a deterrent process and a tamper-evidence process. Given the heightened attention to security in elections in recent years, election officials are eager to learn about threats, possible attacks and mechanisms to neutralize exploitable opportunities that could allow subverting the elections process. We recommended a number of security-based improvements to existing procedures and subsequently observed other security-critical behavior that could be improved.

Timing of random selection and tally: Since the purpose of the tally is to compare two sets of independent vote counts, the random selection and manual tally must take place after the count has been completed. That is, a given ballot type cannot be audited until all the ballots of those type are counted.¹⁸ From a security perspective, attackers should not be able to predict which precincts will be audited while they still have an opportunity to influence vote totals. There is evidence that many jurisdictions perform their random selection very soon after election day, before they could possibly have completed counting VBM and provisional ballots.¹⁹ Unfortunately, for very large jurisdictions like Los Angeles, it is not possible to wait until counting is completed before commencing the selection and tally; the tally process would take longer than the time permitted for the canvass. One possible solution for very large jurisdictions is to treat each ballot type as a sampling stratum in a stratified sampling regime.²⁰

Timing of retrieval of tally materials: Related to this last point is the preservation of the chain of custody for the audit materials once the selection is finished. As soon as the audited precincts have been chosen, the ballot materials to be audited become particularly sensitive. A window of opportunity exists here during which attackers who have tampered with the electronic count could avoid detection by manipulating the audit trail. It is important to minimize the amount of time between the selection and tally and particularly to protect sensitive ballot materials used in the audit.

The feasibility of collecting audit materials quickly depends on the size of the county. Notably, it was difficult for San Mateo, a relatively large county, to initially comply with this recommendation as different ballot materials needed for the tally were stored in physically separated warehouses across the county. They have since been able to store all materials on site for the selection and tally at their main warehouse. In contrast, Marin, a smaller county, stores such materials on-site. The time between the selection and tally for both of these counties is now very brief, about one hour. However, in Alameda, another large county about twice the size of San Mateo, three days elapsed between their selection and tally, considerably widening the window of opportunity for tampering with audit trail materials.

Seal verification: Election officials have to pay increasing attention to tamper-evident security seals due to studies that have shown physical access to voting systems can be an important prerequisite for exploiting serious vulnerabilities. Of course, the importance of maintaining the integrity of this seal-based custody chain does not disappear after election day. We observed inconsistent attention to seal verification during the manual tally. While one jurisdiction placed appropriate weight on seal verification, others performed more casual verification to assess that the seal was not broken (but without verifying the serial number on the seal). Attention to seal verification and seal integrity is a critical step for detecting tampering.

Blind counting: To eliminate the possibility of conscious or unconscious influence on the tally by the tally team, the team should operate under blind counting rules. That is, the tally team will conduct the tally without knowing the ballot totals for the tallied precinct. When the tally is complete for a particular ballot type in one precinct, a supervisor compares their totals to those from the election management system (EMS) database. If the totals do not reconcile, the tally team must count the ballots again to make sure there was not a counting mistake.²¹

While observing, we noted some issues with blind counting and observers' interaction with tally team members. To support transparency, as we note below, observers must have a copy of the electronic results for the precinct being audited. In one case, however, an observer told the tally team the correct electronic tally result when the hand tally was incorrect. This was a violation of two procedural rules: observers are forbidden to interact directly with tally teams and the tally must proceed under a blind counting protocol. To prevent this, observers must be reminded explicitly about the rules of engagement between observers and the tally team and about the importance of conducting a blind count.

Problems with randomness: We observed how slight changes in the random selection method resulted in imperfect randomness.

In San Mateo county, an observer who rolled a set of three ten-sided dice mis-read the roll as being invalid; that is, not corresponding to a valid precinct. Despite the rolled digits in fact being valid, this observer immediately picked up the dice and re-rolled them, effectively ruining that roll. From a security perspective, one could imagine attackers might take advantage of such a situation to ensure that certain precincts are not chosen. In this case, the presiding election official had to explicitly reiterate that observers must not pick up the dice unless directed to by an elections official.

In another case, the county followed their selection protocol perfectly but the protocol itself was flawed. Alameda County performs random selection of precincts using a rotating hopper and a set of 10 ping-pong balls with the digits 0-9 written on them. An election official draws a ping pong ball from the hopper and this digit serves as the ones place of a random number. Balls are drawn, corresponding to successive digits, until the random number chosen corresponds to a number between one and approximately 1200 (there are precincts in Alameda). One quirk in this selection method results in imperfect randomness: only when the hundreds place digit is 0, 1 or 2 do they draw a fourth digit. This results in non-uniform random selection. Because any random number where the hundreds place is larger than 2 can correspond to only precincts numbered from approximately 300-999, this ultimately results in those precincts being chosen with twice the probability of the remaining precincts. Upon inquiring about this with Alameda, we were told that the original method devised to select random numbers required starting with the thousands digit and starting over from scratch when the result would have been an invalid precinct number. We have recommended to Alameda county that they return to discarding invalid random numbers entirely rather than conditionally discarding specific digits. We highlight this case along with two illustrative solutions in a separate research memorandum.²²

This highlights a compelling finding for procedural research: certain procedures, especially those related to technical security matters, are very sensitive to changes in the protocol. Small changes to procedures may seem trivial to those not familiar with the technical background but could have severe consequences. In order to avoid these kinds of imperfections, experts and observers with domain knowledge need to be involved when procedures change. In the longer term, it makes sense for these types of sensitive procedures to be specified in law or regulation so that such deviance is minimized.

Tallying and reporting appropriate records: Tallying the proper records of the vote as well as reporting meaningful data about the tally are critical security-relevant aspects of tally audits. Previous security work has emphasized that keeping track of data such as undervotes, overvotes and spoiled ballots/VVPATs can serve to aid detection of particularly subtle dynamic attacks. For example, a dynamic attack that misprints VVPAT records could be detected during an audit via an unusual amount of spoiled VVPAT records.[ 23] Election officials must track these numbers by hand-counting undervotes, overvotes and spoiled ballots as part of the manual tally. Counties do this inconsistently now, but new reporting requirements for the 1% manual tally require counties to report this information to the Secretary of State. To support collection of this data, we assisted the Secretary of State in designing a reporting instrument that includes reporting, among other data, quantities of overvotes, undervotes and spoiled ballots.²³

We observed that one county did not appear to be counting the actual paper records but the totals tapes.²⁴ At least, the method that they used to perform their tally of VVPAT records was very different from the other counties. The tally must be over the records verified by the voter or the necessary confirmation of voter intent is never possible and opportunities for mischief and/or error increase. Like other counties we observed, this particular county started by cutting VVPAT records off of the VVPAT roll. However, they immediately placed the cut VVPATs into manila folders and began cutting up the totals tape, race-by-race, for each VVPAT roll. The tally proceeded by the caller calling out phrases like, ``McCain, 0...Obama, 0...''; that is, instead of calling out votes off of individual VVPATs, the caller appeared to be reading totals off of the totals tape. When we asked election officials about this, they stated that they were ``counting individual votes and not just the summary''. In addition to the security-related concern that voters do not verify the totals tape information the law is very specific that the VVPAT must be used for the manual count.²⁵ We have been unable to confirm what we observed.²⁶

Resistance to insider modification of voted ballots: We observed inconsistent attention to insider attacks on voted ballots during the tally process. When we did see provisions meant to mitigate certain types of insider attacks, we were often surprised by their existence. For example, each of the counties we observed had tally teams use pencil for marking tally documents. We found this puzzling as this permitted talliers to erase marks instead of the more standard accounting process for correcting mistakes: crossing out errors and initialing corrections.²⁷ When we inquired about this, officials responded with an obvious security-based answer: pens could be used to indelibly change voted ballots. In fact, San Mateo's procedures included a prohibition on any indelible marking device in the tally area. While we were surprised by this particular case, there did not seem to be more systematic attention to insider threats to the tally process. Obviously, as is the theme of this work, exhaustive attention to insider threats will undoubtedly conflict with other constraints and priorities. Research examining insider threats to election office activity is a promising area for future work.

4.2 Transparency

As we have stressed in previous work,[ 25] electoral transparency requires supporting access, oversight, accountability and comprehensibility of election processes. In terms of access and oversight, procedural improvements that impact transparency relate mostly to to publication of notice, procedures and data so that members of the public can observe the process in an informed manner and perform their own calculations, if necessary. In terms of accountability and comprehensibility, we found it was important to have clear lines of communication for asking questions or alerting officials to procedural anomalies.

Public notice: Observers wishing to witness the manual count need to have public notice of the time and location of the tally process. When we began this work, it was difficult to know when and where a given jurisdiction's manual tally would be conducted. With a recent addition to the manual tally law, a minimum of 5-day public notice is now required.²⁸ This has largely alleviated such frustrations and we have been able to find such notice for the jurisdictions that we have wanted to observe. However, in addition to traditional means of posting notice, like newspaper advertisements, we recommend wide, varied posting of such notice, such as on the jurisdiction's web site, via press release or op-ed in local community publications.

Procedures publication: Along with knowing when and where the tally will take place, observers will need information about the tally procedures themselves as well as any observer guidelines. Observers might not be familiar with the intricacies of the tally procedure and should be provided with a document that summarizes the process and then goes into detail about how the tally is conducted. The few written tally procedures we have seen, from the counties we observed, range from very detailed to providing a high-level overview.[ 26, 27, 28] Also, observers may be unfamiliar with how they should behave while observing the tally, as was the case mentioned above where an observer inadvertently broke the blind counting rule (see 4.1). We encountered this ourselves in San Mateo when we were told that no discussion is allowed amongst observers during the tally process per their observer guidelines. San Mateo seems to have the best practice here in terms of providing observation guidelines for all observable events in a typical election cycle.²⁹

Data publication: In addition to procedural information, observers need be provided with various data. For the random selection, observers need a hardcopy (or read-only copy) of the mapping between possible random numbers to precinct identifiers.³⁰ This allows observers of the random selection to verify that the selected number indeed corresponds to the correct precinct identifier. In our observations, these materials were made available to observers in Alameda and Marin but not in San Mateo. San Mateo instead had the mapping spreadsheets on a laptop--decidedly not read-only--and would turn the laptop towards observers so that they could verify the selection.³¹

For the tally, observers should have hardcopy (or read-only) copies of the unofficial statement of the vote as well as the detailed precinct results reports for each selected precinct. We have recommended that jurisdictions provide the statement of the vote during the random selection process so that the jurisdiction ``commits to'' or ``vouches for'' the vote totals before any precinct is selected. Observers need detailed precinct reports wile observing so that they can confirm the tally totals in the same manner as the tally supervisor confirms each tally team's manual count results. Again, these materials were available in Alameda and Marin counties but were not present in San Mateo.

Clear lines of communication: Observers need effective lines of communication for asking questions in a timely manner and also escalating issues that they feel are contrary to written procedures or otherwise anomalous. During our observations, we were able to easily ask questions of even the most senior elections personnel. We consistently received thoughtful and considerate responses to our queries and concerns. This notion of transparency seems to be something that counties support well already.

4.3 Efficiency

Efficiency--that is, minimizing waste during the tally in terms of time and resources--was a major concern during this work. We approached this research project knowing that we could not simply make demands of election officials in terms of security and transparency and expect them to adopt our ideas without question. Accordingly, we felt that we should work equally as hard at giving something back. During our initial conversations with elections officials, it became clear that recommending steps that could make their processes more efficient would be greatly appreciated.

The manual tally, as we have mentioned above, can be problematic in terms of time, space and staff. The tally is a time- and resource-intensive effort that requires a significant amount of work from election staff (and/or temporary employees) during a crucial time in which these staff could be assisting with other canvass-related activities. Most of the recommendations we have made in terms of efficiency relate to time efficiencies; that is, if we could find a way to do a task in less time, it would free up staff and other resources.

Randomness can be inefficient: Often our preferred methods of publicly-verifiable random selection can be inefficient. For example, in San Mateo during the random selections in November 2006 and 2007, using 10-sided dice to select the 1% sample went fairly quickly but selecting precincts for races not included in the 1% sample took a considerable amount of time--on the order of an hour--due to the high frequency of invalid rolls. During these selection events with many misrolls, we found ourselves (observers and election officials) making up rules as we went along. For example, if there were two precincts to choose from, we would roll one die and specify that even numbers corresponded to one precinct and odds to the other. But this ad-hoc rule creation was troubling and became difficult to do easily on-the-fly with larger numbers of precincts. Among the recommendations of Cordero et. al, they advised ``binning'' random numbers into equal sized bins to increase the frequency of valid rolls.[ 15] We created a web-accessible script written in PHP to do this for an arbitrarily-large jurisdiction with an arbitrary number of 10-sided dice.[ 30] This script does one thing: given a number of precincts to choose from and a number of 10-sided dice, it calculates the appropriate binning to minimize the frequency of misrolls (it also displays the bin mapping in a format that is easy for election staff to cut-and-paste into a spreadsheet program). San Mateo county used this script for their random selection in the manual tally for their February 2008 election and considerably shortened the time it took to do the selection as well as streamlined the process.³²

Vote results reports should be fine-grained: In some cases, the vendors' EMSs will not report machine-specific results within a precinct. Unfortunately, this often means that a manual tally of, say, four to five VVPAT rolls for a given precinct can be compared only with aggregate precinct totals, instead of on a machine-by-machine basis. Considering that we observed that it might take one tally team of four people over 4 hours to tally one full VVPAT roll, finding a discrepancy after all that effort is ineffective and inefficient; if there is a discrepancy, the EMS report contains no information that would be helpful in locating on which VVPAT roll the discrepancy might be contained. This was the case in San Mateo which uses Hart Inter Civic voting systems; each precinct had 4-5 VVPAT rolls but the Hart system only reports results at the precinct level. This can result, due to blind counting rules mentioned above, in the tally team having to redo the tally for each of that precinct's VVPAT rolls. If the EMS had reported vote totals for each machine in the precinct, the tally team would have instead had to retally a small number of VVPAT rolls.³³

State-of-the-art auditing methodologies can also place distinct requirements on voting systems. For example, statistically conservative audit schemes[ 9] start with a flat percentage audit, then require the auditor to calculate a statistical confidence value and, if needed, increase the sample size of the audit. However, some vendors' EMSs will produce meaningful results only in PDF format, a format useful for presentation of information but not useful for computation. To quickly calculate a statistical quantity with data from hundreds of precincts in such an unusable format would require an army of transcribers. If EMSs had the capability to output vote totals in an open, machine-readable and machine-processable format, such as the OASIS standard Election Markup Language,³⁴ they would better support more sophisticated forms of election audits.

Adverse effects of good team demeanor: We observed subtle effects of tally team members becoming gradually more comfortable with one another. We noticed that, quite naturally, the members of a tally team tended to get progressively more comfortable with each other as tallying progressed and increasingly reluctant to assert certain conflicting aspects of their tally team roles.³⁵ It is natural for a working group to converge, socially or otherwise, on a more comfortable working state, but often it seemed that objections were self-silenced to avoid derailing or slowing down the tally. However, this posed a few unique problems. We noted that occasionally, a tallier would either forget to call out on every multiple-of-ten vote or call out earlier than another tallier. More often than not, this was dealt with informally rather than with a more formal procedure of backing up ten votes and redoing the tally to this point. We also noted that the witness would occasionally question the vote determination made by the caller or mis-stack ballots into a stack of 9 ballots. These hiccups were also often dealt with informally instead of with a more formal challenge procedure for questioning the determination of a caller or rolling back the count to correct for mis-stacking. To help solve this problem, tally team training should emphasize that that talliers must feel comfortable stopping the process at any point for clarification or to question a determination made by other team members.

Using pre-filled tally sheets: A big time saver observed in Marin County's tally was the use of pre-printed tally sheets. In other jurisdictions such as Alameda and San Mateo, tally teams had to fill out generic tally sheets by hand for the precinct they were tallying.³⁶ In a primary election with many candidates and many races, this can take a significant amount of time; e.g., in San Mateo, we observed that this took about one hour or 20 person-hours.³⁷ In contrast, Marin used a digital tally sheet template in which they had one staff member fill out the candidate names and then print copies for the tally team. To save time in terms of person-hours, pre-filled printed tally sheets should be made for all jurisdiction-wide races and then races unique to a particular ballot style can be filled in by hand.

Innovative uses of RFID technology: Finally, Alameda county exposed us to very innovative uses of radio-frequency identification (RFID) technologies in their chain-of-custody procedures. Alameda applied RFID chips to the election media and the pollbook for each precinct. When pollworkers returned precinct materials to drop-off locations, an election staffer with a RFID reader would read the RFID chips from within a sealed security bag, without breaking the seal. This shortened the time needed to check the presence of critical items drastically--from about 30 minutes to less than 5--while preserving one link in the chain of custody. While this use of RFID technology was not in the context of the manual tally, it shows increasing promise in the use of RFID chips for elections-related chain-of-custody. Given the poor quality of current security seal technology,[ 31] we recommend that researchers combine the inventory-tracking capability of RFID technology, the tamper-evidence of sensitive security seals and recent innovations in uncloneable RFIDs to provide the a type of security seal that can be read and cryptographically verified quickly at a distance but that will also ``self-destruct'' upon physical tampering, so that no forged replacement could be crafted.

5 Conclusion

We analyzed the manual tally process as used in a number of California counties to design a generic set of tally procedures that California counties can use. In the process of iterating on the design of procedures with San Mateo as well as observing elections in San Mateo, Alameda and Marin Counties, we developed a number of improvements in terms of security, transparency and efficiency. We also discovered some issues outside the scope of procedure design; for example, the challenges that large counties face in meeting the 28 day canvass deadline or how voting systems could better support manual tally audits. The current procedures resulting from this work exist specifically for San Mateo[ 26] or in a generic form, designed for use by any California county in improving their 1% manual tally process.[ 1] We have attempted to note in the generic procedures where certain ideas are not specific to California.

Acknowledgments

This material is based upon work supported by the National Science Foundation under A Center for Correct, Usable, Reliable, Auditable and Transparent Elections (ACCURATE), Grant Number CNS-0524745. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation.

Considering the almost 2-year time period over which this research was conducted, there are many contributors to acknowledge. Close collaborators in this work included Kim Alexander, Aaron Burstein, Arel Cordero, David Dill, Deirdre Mulligan, Philip Stark and David Wagner. This work would have not been possible without the cooperation and patience of local and state election officials and their staff, such as Warren Slocum, David Tom, Theresa Rabe, Freddie Oakley, Tom Stanionis, Elaine Ginnold, Dave Mac Donald, Jennie Bretschneider, Lowell Finley and California Secretary of State Debra Bowen. In the process of completing this work, the author found discussions with the following people helpful: Judy Bertelsen, Tim Erickson, Michelle Gabriel, Candice Hoke, Meg Holmberg, David Jefferson, Bob Kibrick, Mark Lindeman, John Mc Carthy, Lawrence Norden, Dennis Paull and Pam Smith.

Bibliography

1 Joseph Lorenzo Hall.
The 1% manual audit in california.
UC Berkeley School of Information, April 2008.

2 Ronald L. Rivest and John Wack.
On the notion of ``software independence'' in voting systems, July 2006.

3 Top-to-bottom review of california's voting systems.
California Secretary of State, March 2007.

4 Patrick Mc Daniel, Matt Blaze, Giovanni Vigna, Kevin Butler, William Enck, Harri Hursti, Steve Mc Laughlin, Patrick Traynor, Adam Aviv, Pavol Cerny, Sandy Clark, Eric Cronin, Gaurav Shah, Micah Sherr, Richard Kemmerer, Davide Balzarotti, Greg Banks, Marco Cova, Viktoria Felmetsger, William Robertson, Fredrik Valeur, Joseph Lorenzo Hall, and Laura Quilter.
Everest: Evaluation and validation of election-related equipment, standards and testing (academic final report), December 2007.

5 Software reviews and security analyses of florida voting systems.
Florida State University's Security and Assurance in Information Technology Laboratory, February 2008.

6 Lawrence Norden, Aaron Burstein, Joseph Lorenzo Hall, and Margaret Chen.
Post-election audits: Restoring trust in elections, 2007.

7 Ben Adida and C. Andrew Neff.
Ballot casting assurance.
USENIX/ACCURATE Electronic Voting Technology 2006 (EVT'06) Workshop, 2006.

8 Daniel Tokaji.
The paperless chase: Electronic voting and democratic values.
Fordham Law Review, 57, 2005.

9 Philip B. Stark.
Conservative statistical post-election audits (in press).
The Annals of Applied Statistics, 2008.

10 Javed A. Aslam, Raluca A. Popa, and Ronald L. Rivest.
On auditing elections when precincts have different sizes.
USENIX/ACCURATE Electronic Voting Technology Workshop 2008, July 2008.

11 Roy G. Saltman.
Effective use of computing technology in vote-tallying.
National Bureau of Standards, March 1975.

12 David Jefferson, Elaine Ginnold, Kathleen Midstokke, Kim Alexander, Philip B. Stark, and Amy Lehmkuhl.
Evaluation of audit sampling models and options for strengthening california's manual count.
California Secretary of State, July 2007.

13 Post-election manual tally requirements.
California Secretary of State, October 2007.

14 Sarah P. Everett.
The usability of electronic voting machines and how votes can be changed without detection.
Rice University Ph D Thesis, May 2007.

15 Arel Cordero, David Wagner, and David Dill.
The role of dice in election audits--extended abstract.
IAVoSS Workshop on Trustworthy Elections 2006 (WOTE 2006), June 2006.

16 Joseph A. Calandrino, J. Alex Halderman, and Edward W. Felten.
In defense of pseudorandom sample selection.
USENIX/ACCURATE Electronic Voting Technology Workshop 2008, July 2008.

17 David Wagner.
Thoughts on the nov 16, 2006 1% manual tally in yolo county.
UC Berkeley Department of Computer Science, November 2006.

18 Joseph Lorenzo Hall.
The 1% manual audit in california: Proposed procedures and rationale, November 2006.

19 Rebekah Gordon.
Elections office gets tips from experts.
San Mateo County Times, November 2006.

20 County manual tally reports, April 2008.

21 Summary information--post-election manual tally.
County of Fresno County Clerk / Registrar of Voters, March 2008.

22 Joseph Lorenzo Hall.
Research memorandum: On improving the uniformity of randomness with alameda county's random selection process.
UC Berkeley School of Information, March 2008.

23 Lawrence Norden and Eric Lazarus.
The machinery of democracy: Protecting elections in an electronic world: Brennan center task force on voting system security.
Brennan Center for Justice at NYU School of Law, 2006.

24 Post-election manual tally log: Suggested instructions for post-election manual tally requirements (ccrov-08048).
California Secretary of State, 2008.

25 Joseph Lorenzo Hall.
Transparency and access to source code in electronic voting.
USENIX/ACCURATE Electronic Voting Technology Workshop 2006, June 2006.

26 One percent manual recount procedures.
San Mateo County Clerk Assessor Recorder Elections, December 2007.

27 Procedures for one percent manual tally.
Marin County Registrar of Voters, February 2008.

28 Process overview of the 1% manual tally.
Alameda County Registrar of Voters, February 2008.

29 Election observer handbook.
San Mateo County Clerk Assessor Recorder Elections, February 2008.

30 Joseph Lorenzo Hall.
Dice binning calculator for post-election audits, March 2008.

31 Roger G. Johnston.
Tamper-indicating seals.
American Scientist, 94:515-523, November 2006.

About this document ...

Improving the Security, Transparency and Efficiency of California's 1% Manual Tally Procedures

This document was generated using the LaTeX2HTML translator Version 2002-2-1 (1.71)

Copyright © 1993, 1994, 1995, 1996, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998, 1999, Ross Moore, Mathematics Department, Macquarie University, Sydney.

The command line arguments were:
latex2html -split 0 -show_section_numbers -local_icons -no_navigation jhall_evt08_html

The translation was initiated by Joseph Hall on 2008-07-01

---

Footnotes

$ ... Information ¹: Contact the author at: joehall@berkeley.edu. This paper was submitted on 11 April 2008 to the 2008 USENIX/ACCURATE Electronic Voting Technology Workshop (EVT'08), accepted on 20 May 2008 and submitted in final form on 30 June 2008. This paper will be presented at EVT'08 in San Jose, California (USA) on 28 July 2008; See: http://www.usenix.org/event/evt08/. $ ... county. ²: Due to space limitations, the detailed procedures are presented in a separate document: [ 1] $ ... record. ³: The Verified Voting Foundation keeps an up-to-date list of state paper record laws on its front page (see: http://www.verifiedvoting.org/). Many researchers believe that paper is currently the only feasible form of such an audit record, while others advocate for ``end-to-end'' verification systems or take issue with the lack of verifiability of most paper-based systems for people with disabilities. See: [ 7, 8] $ ... years. ⁴: California's 1% manual tally was introduced in 1965. See: California Statutes 1965, c. 2040, p. 4659, Sec. 1. $ ... program. ⁵: Saltman provides interesting cases of historical punchcard mishaps. See: [ 11] $ ... precincts. ⁶: Note that the current manual tally statute, CA Elec. Code Sec. 15360, was consolidated from a number of historical statutes. See: Former Sec. 15645, enacted by CA Stats. 1994, c. 920, Sec. 2, (derived from: former Sec. 15281 (added by CA Stats. 1976, c. 246, Sec. 3), former Sec. 15417 (added by CA Stats. 1965, c. 2040, p. 4659, Sec. 1), and former Sec. 17190 (added by CA Stats. 1978, c. 847, Sec. 5, amended by CA Stats. 1986, c. 1277, Sec. 14.). $ ... count. ⁷: CA Elec. Code Sec. 336.5. $ ... records. ⁸: CA Elec. Code Sec. 19253(b)(2). $ ... tally. ⁹: See n. 35 for examples of blank tally sheets. $ ... sync. ¹⁰: We are not certain why these counties had such similar counting methods. Upon asking, for example, about the size and make-up of tally teams, election officials tended to respond that this is how they have counted in the past. One hint to this particular question came when we observed one county, Alameda County, using three-member tally teams instead of the more-standard four-member teams. Upon questioning, the county said that the election code did not require them to use four-member teams for the manual tally. This is correct; the election code does not contain much detail about specific procedures for manual tallying. Where might Alameda have thought that the election code speaks at all about, for example, the structure of a tally team? CA Elec. Code 15102 does specify, in a section titled Vote By Mail Processing, that hand tallying of vote by mail ballots shall be done by a team of four people:

When the tally is done by hand, there shall be no less than four persons for each office or proposition to be counted. One shall read from the ballot, the second shall keep watch for any error or improper vote, and the other two shall keep the tally.

$ ... audit. ¹¹: Of course, voters must actually check that the contents of the paper record match their intent and what is displayed on the voting system screen. Recent evidence suggests that few voters do this. See: [ 14]. This concern does not apply to technologies where voters directly-mark the paper record, such as optical scanners. $ ... tallied. ¹²: Note that this election law (CA Elec. Code Sec. 15360(c)) now specifies that election officials must use a ``random number generator or other method'' from regulations adopted by the Secretary of State. This means that there is no prohibition on computer-generated pseudorandom numbers, as we would prefer. $ ... al. <sup>13</sup>: [ 15]. But compare with: [ 16]. $ ... election. <sup>14</sup>: CA Elec. Code Sec. 15372. $ ... County. <sup>15</sup>: We met twice with Yolo county to discuss how what we were learning would apply to Yolo, a much smaller county. This author did not have the opportunity to observe the manual tally in Yolo County, although we benefited from the observations of collaborators. [ 17]. $ ... team <sup>16</sup>: Others involved in this work include David Dill (Stanford Computer Science), David Wagner (UC Berkeley Computer Science), Arel Cordero (UC Berkeley Computer Science), Aaron Burstein (UC Berkeley Law) and Kim Alexander (California Voter Foundation). $ ... procedures. <sup>17</sup>: For example, Alameda County worked with Arel Cordero, David Wagner, David Dill and members of their public advisory committee to design a random selection process using a tumbler with numbered ping-pong balls. We provided a research memo describing how imperfections in their random selection process resulted in non-uniform random selection. We describe this case in Section 4.1. $ ... counted. <sup>18</sup>: By ``ballot types'' we mean ballots cast using a distinct type of voting technology such as in-precinct optical scan ballots, DRE ballots, paper-based provisional ballots, etc. $ ... ballots. ¹⁹: The Secretary of State has published manual tally reports that show many counties performing random selection very soon after election day. [ 20] In one case, a county seems to have even performed the selection before election day (February 5, 2008). [ 21] $ ... regime. ²⁰: This allows beginning the manual tally for ballot types where counting is completed before all ballot types are counted. In February 2008, we participated with Philip Stark (UC Berkeley Statistics) and Elaine Ginnold (Marin County Registrar of Voters) in the first statistical confidence-based audit of an election. In this pilot, each ballot type was sampled in separate strata so that the tally could begin for ballots where counting was completed. $ ... mistake. ²¹: When counting optical scan ballots--where the ballot marking can be less definitive--we observed that it helped to relax the blind counting requirements after two discrepant tallies. This allowed the talliers to try and determine which ballot had been read (or not read) by the machine but not seen (or seen) as a valid vote by the talliers. $ ... memorandum. ²²: [ 22]. Note: we did not discover this independently; two public observers, Meg Holmberg and Tim Erickson, brought this to our attention. $ ... ballots. ²³: Kim Alexander (California Voting Foundation), Philip Stark (UC Berkeley Statistics) and myself assisted the Office of the California Secretary of State in developing the reporting instrument: [ 24] $ ... tapes. ²⁴: We have been unable to confirm with this particular county how they count VVPAT records. $ ... count. ²⁵: CA Elec. Code Sec. 19253(b)(1) which says, in part, ``The voter verified paper audit trail shall be considered the official paper audit record and shall be used for the required 1-percent manual tally [...]''. $ ... observed. <sup>26</sup>: For example, one explanation is that there were very few ballots cast on the VVPAT rolls we observed. In that case, most of the candidates on the ballot would have zero votes. Tallying would entail a brief tally of any candidates that do have votes with a lengthy tally of zero votes for each of the remaining candidates. Without pre-filled tally sheets, the task of tallying votes on VVPAT records becomes more a task of confirming the lack of votes. $ ... corrections. <sup>27</sup>: For example, we observed a tallier in Alameda erasing tally marks after they had to retally a specific candidate. See: http://www.flickr.com/photos/joebeone/2295569830/sizes/l/. $ ... required. <sup>28</sup>: CA Elec. Code Sec. 15360(d). $ ... cycle. <sup>29</sup>: [ 29] In addition, the California Secretary of State requires counties to publish an Election Observation Panel Plan (see: http://www.sos.ca.gov/elections/eop.htm) and the California Election Code imposes certain requirements to facilitate public observation in certain cases. See CA Elec. Code Sec. 15104. $ ... identifiers. <sup>30</sup>: We call this hardcopy mapping a Master Selection Spreadsheet. Recall that once the 1% selection is complete, additional selection may be necessary as there may be races for which no precinct was selected for auditing. If a jurisdiction has decided to follow best practice, rather than the letter of the law, and select precincts randomly for these additional races, they will need to provide similar mapping spreadsheets for each race in the election (we call these Contest Selection Spreadsheets). $ ... selection. <sup>31</sup>: See the image linked in n. 31. $ ... process. <sup>32</sup>: To see our dicebins.php calculator ``in action'' in San Mateo, see: http://www.flickr.com/photos/joebeone/2293490290/sizes/l/. $ ... rolls. ³³: We observed one team quickly recounting stacks of ballots to make sure that they arrived at the result they got with the slower tally process. This might have been developed in response to having to retally an entire precinct. $ ... Language, ³⁴: See: http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=election. $ ... roles. ³⁵: We note that, in one exceptional case, one tally team member was a supervisor working in the local elections department. We found the interaction between this tally team member and the other members of the team to be particularly different from other teams. We were uncertain if the other team members were directly supervised by this individual. If so, this obviously highlights an undesired point of friction. Election officials need to be conscious of these kinds of power dynamics and seek to neutralize them. One option would be to only place supervisors on teams of people that they do not directly supervise (such as temporary employees). Another possible solution is to emphasize in training that all tally team members have equal authority for the period of time in which the tally is conducted. $ ... tallying. ³⁶: For images of these types of tally sheets in Alameda and San Mateo, see the following images, respectively: http://www.flickr.com/photos/joebeone/2266221884/sizes/l/ andhttp://www.flickr.com/photos/joebeone/2240342264/sizes/l/. $ ... person-hours. ³⁷: This observation was based on San Mateo's VVPAT tally area which used 5 tally teams of 4 people each. San Mateo did photocopy the filled-out tally sheets once each tallier had completed them so that they could be reused when the team started a new VVPAT roll.

---

Joseph Hall 2008-07-01

Comments

• hall-ImprovingManualTallyProcedures.pdf: Improving Manual tally procedures in California