Principles and Best Practices for Post-Election Audits

electionaudits.org (2008-09)

This Page: https://copswiki.org/Common/M1729
Media Link: https://copswiki.org/w/pub/Common/M1729/bestpracticesfinal_0.pdf
Remote Link: http://electionaudits.org/files/bestpracticesfinal_0.pdf
More Info: Election Integrity, Election Team, Snapshot Protocol, Snapshot Protocol

• Principles and Best Practices for Post-Election Audits
  

The following is an html version of the document with some comments.

---

Principles and Best Practices for Post-Election Audits

electionaudits.org 2008

Endorsed by the following organizations:

• Brennan Center for Justice,
• Citizens for Election Integrity Minnesota,
• Coloradoans for Voting Integrity,
• Common Cause,
• CTVoters Count.org,
• Florida Voters Coalition,
• Iowans for Voting Integrity,
• Michigan Election Reform Alliance,
• Verified Voting,
• Citizens for Election Integrity Massachusetts.

Statistical portions, principles 5–7 and their best practices, endorsed by the American Statistical Association.

Executive Editors

• Mark Lindeman, Assistant Professor of Political Studies, Bard College*
• Mark Halvorson, Director and Founder, Citizens for Election Integrity Minnesota
• Pamela Smith, President, Verified Voting
• Lynn Garland
• Vittorio Addona, Assistant Professor of Mathematics and Computer Science, Macalester College*
• Dan Mc Crea, President and Co-founder, Florida Voters Foundation

• • *affiliations for identification purposes only

These principles were written to guide the design of high-quality post-election audits. They were developed by an ad hoc group comprising many stakeholders, including election officials, public advocates, computer scientists, statisticians, political scientists and legislators.

Why Audit Election Results:

No voting system is perfect. Nearly all US elections today are counted using electronic voting systems. Such voting systems have produced result-changing errors through problems with hardware, software, and procedures.(1) Errors can also occur in hand counting of ballots or in the compiling of results. Even serious error can go undetected if results are not audited effectively.

Well-designed and properly performed post-election audits can significantly mitigate the threat of error, and should be considered integral to any vote counting system. A post-election audit in this document refers to hand-counting votes on paper records and comparing those counts to the corresponding vote counts originally reported, as a check on the accuracy of election results, and resolving discrepancies using accurate hand counts of the paper records as the benchmark. Such audits are arguably the most economical component of a quality voting system, adding a very small cost(2) for a large set of benefits.

The benefits of such audits include:

• Revealing when recounts are necessary to verify election outcomes
• Finding error whether accidental or intentional
• Deterring fraud
• Providing for continuous improvement in the conduct of elections
• Promoting public confidence in elections

Post-election audits differ from recounts. Post-election audits routinely check voting system performance in contests,(3) regardless of how close margins of victory appear to be. Recounts repeat ballot counting in special circumstances, such as when preliminary results show a close margin of victory. Post-election audits that detect errors can lead to a full recount. When an audited contest is also recounted, duplicate work can be avoided (see Best Practices 9a). Voting systems should have reliable audit records. Best effort audits should be performed even if the technology does not support optimal audits, or even if the laws do not permit optimal remedies.(4) No single model for post-election audits is best for all states. Election traditions, laws, administrative structure and voting systems vary widely. Nonetheless, there are guiding principles that apply across all states. As states develop their own audit models, the public should have the opportunity to help shape those regulations.

Principles

1. TRANSPARENCY: Elections belong to the public. The public must be allowed to observe, verify, and point out procedural problems in all phases of the audit without interfering with the process.
2. INDEPENDENCE: The authority and regulation of post-election audits should be independent of officials who conduct the elections. The actual work of post-election audits may be best performed by the officials who conduct the elections.
3. PAPER RECORDS: Ideally, post-election audits use hand-to-eye counts of voter-marked, voter-verified paper ballots. Where such paper ballots are not available, other forms of voter-verifiable paper records should be used.
4. CHAIN OF CUSTODY & BALLOT ACCOUNTING: Robust ballot accounting and secure chain of custody of election materials and equipment are prerequisites for effective post-election audits.
5. RISK-LIMITING AUDITS: Post-election audits reduce the risk of confirming an incorrect outcome. Audits designed explicitly to limit such risk (risk-limiting audits) have advantages over fixed-percentage or tiered audits, which often count fewer or more ballots than necessary to confirm the outcome.
6. ADDRESSING DISCREPANCIES and CONTINUING THE AUDIT: When discrepancies are found, additional counting and/or other investigation may be necessary to determine the election outcome or to find the cause of the discrepancies.
7. COMPREHENSIVE: All jurisdictions and all ballot types, including absentee, mail-in and accepted provisional ballots, should be subject to the selection process.
8. ADDITIONAL TARGETED SAMPLES: Including a limited number of additional targeted samples of ballots can increase audit effectiveness and public confidence. Such samples may be selected by candidates, issue committees, parties, election administrators, or others as provided by regulation.
9. BINDING ON OFFICIAL RESULTS: Post-election audits must be completed prior to finalizing official election results and must either verify the outcome or, through a 100% recount, correct the outcome.

Best Practices

1. Transparency

Elections belong to the public. The public must be allowed to observe, verify, and point out procedural mistakes in all phases of the audit without interfering with the process. The following conditions must be met:

• a. Detailed auditing procedures are developed well in advance of elections, with reasonable opportunities for public comment. These include procedures for selecting audit units,(5) sorting the paper records and counting the votes, and determining when more units need to be audited and when the audit can end. There is adequate notice to allow the public to witness and verify each phase of the audit.
• b. The public is given sufficient access to witness and verify the random selection of the audit as well as the manual count with reasonable opportunities for public comment. Election officials have the authority to prevent the public from hampering the proceedings.
• c. Final results are reported to the public immediately and posted on the Web. The results include an analysis of all discrepancies as well as recommendations for improvement. The data on the frequency and source of discrepancies can provide jurisdictions with benchmarks for improvement in future elections.(6)
• d. Ideally, a public archive of the audit documents, reports and results is maintained indefinitely in the case of electronic records and for at least two years in the case of paper records.

2. INDEPENDENCE:

The authority and regulation of post-election audits should be independent of officials who conduct the elections. The actual work of post-election audits may be best performed by the officials who conduct the elections and their designees.

• a. The independence of authority and regulation may be satisfied from resources inside or outside state government.
• b. The actual work of post-election audits—i.e. the handling and counting of ballots and reporting the results—may be best performed by the officials who conduct the elections.

3. PAPER RECORDS:

Ideally, post-election audits use hand-to-eye counts of voter-marked, voter-verified paper ballots. Where such paper ballots are not available, other forms of voter-verifiable paper records should be used.

• a. The paper records must be easy to read and handle.
• b. The paper records must reliably reflect the intent of the voters. Care must be taken to urge voters to confirm the record of their votes.

4. CHAIN OF CUSTODY & BALLOT ACCOUNTING:

Robust ballot accounting and secure chain of custody of election materials and equipment are prerequisites for good post-election audits. The following conditions must be met:

• a. There are strict written accounting procedures for paper records to prevent the addition, subtraction, substitution, or alteration of paper records.
• b. To safeguard the ballots and audit records from loss and tampering, paper records and electronic equipment are fully secured(7) at all times when a breach could adversely affect the integrity of the records including from the time the votes are cast until all audit or recount activity is completed and election results are finalized.(8)
• c. The audit begins as soon as possible after the random selection of audit units, which commences as soon as possible after the initial tallies recorded by the voting system are reported. (In some circumstances the audit may be conducted in phases as discussed in Best Practice 5e.)
• d. The secrecy of the ballot is preserved; the order of the votes cast is never compared to the order in which the voters signed in.
• e. There is a reconciliation to ensure that all votes from all audit units are correctly tabulated in the election totals.

5. RISK-LIMITING AUDITS:

Post-election audits reduce the risk of confirming an incorrect outcome. Audits designed explicitly to limit such risk (risk-limiting audits) have advantages over fixed-percentage or tiered audits, which often count fewer or more ballots than necessary to confirm the outcome.

• a. Risk-limiting audits have a large, predetermined minimum chance of leading to a full recount whenever a full recount would show a different outcome.(9)(10) (Correct preliminary outcomes are never overturned.(11)) After any audit, this chance should be calculated and published as part of the audit results to promote continuous improvement.
• b. Audit units (precincts, machines, batches of paper records) should be selected using appropriate random sampling methods.(12) In a risk-limiting audit, the sample size will depend on the margin of victory and other factors; these other factors may include the number of ballots in each precinct and the overall number of ballots in the contests.(13) In general, smaller margins of victory and smaller contest require auditing a larger percentage of the audit units.
• c. To reduce the burden of counting ballots while still auditing a variety of contests, it may be appropriate to use different rules for auditing some contests than others. For example, it may be appropriate to allow more risk for non-statewide contests.(14) Jurisdictions may require audits in some contests and randomly select others to be audited, so that every contest has some possibility of being audited. For smaller contests, it may also be appropriate to use alternative audit methods such as targeted sampling (see Best Practice 8) or random sampling based on a fixed number or percentage of audit units.
• d. The selected audit units must be fully and manually counted.(15) For each selected audit unit, the audit must compare vote count subtotals from the preliminary reported election results with hand-to-eye counts of the corresponding paper records.
• e. For efficiency, large groups of ballots can be divided into batches, each comprising an audit unit. In this case, the subtotals for each batch must be reported prior to the audit as part of the election results. For instance, absentee ballots (if not sorted and counted by precinct) can be divided into batches.

6. ADDRESSING DISCREPANCIES and CONTINUING THE AUDIT:

When discrepancies are found, additional counting and/or other investigation may be necessary to determine the election outcome or to find the cause of the discrepancies.

• a. Audit protocols must clearly state what will result in counting more audit units. Such factors might include the number of discrepancies and their distribution across the sample. Protocols must also specify the method to determine how many additional audit units will be selected and under what circumstances a full recount will be conducted. For a risk-limiting audit, the decision of whether to count more audit units is based on a calculation of the risk; the number of additionally selected audit units depends crucially on the discrepancies that have been uncovered.
• b. The plan for continuing the audit must ensure that all stages in counting take place before reporting final results. Moreover, the plan should aim to control the cost of post-election audits while achieving any specified risk limit.

7. COMPREHENSIVE:

All jurisdictions and all ballot types, including absentee, mail-in and accepted provisional ballots, should be subject to the selection process.

• a. Ballots from different jurisdictions and ballot types can be divided into distinct groups that are audited in separate phases. In each phase, the random selection of units to audit must not commence until preliminary results for each audit unit in that group have been reported to the public.
• b. All types of ballots, even those used by few voters, should be subject to the selection process.(16) These might include overseas or military ballots, faxed ballots, telephone ballots, ballots transmitted over the Internet, ballots cast through accessible interfaces “voter-verified paper audit trail” ballot images, and ballots cast using any other future technology.(17)

8. ADDITIONAL TARGETED SAMPLES:

Including a limited number of additional targeted samples of ballots can increase audit effectiveness and public confidence. Such samples may be selected by candidates, issue committees, parties, election administrators, or others as provided by regulation.

• a. This type of sample can be used either in conjunction with a random audit, or by itself for a contest not required by regulation to be audited using a random method.(18)
• b. Targeted audit units might be chosen based on such factors as major election day problems or preliminary results that deviate significantly from historical voting patterns.

9. BINDING ON OFFICIAL RESULTS:

Post-election audits must be completed prior to finalizing official election results and must either verify the outcome or, through a 100% recount, correct the outcome.

• a. The audit procedures and timing must be integrated with recount law. For each contest, an audit unit normally should be counted only once, even if it is included in both an audit and a recount. If there are unexplained discrepancies in the vote count, a manual count may need to be repeated to reduce the likelihood of a counting error.

Comments

Notes

1 : For example, in Pottawattamie County, Iowa, in the June 2006 primary election for County Recorder, the original optical scan count showed challenger Oscar Duran defeating the incumbent, John Sciortino. A hand count showed that Sciortino actually had won handily; the scanners had been misprogrammed. In Napa County, California, after the March 2004 primary, the 1% manual tally discovered that the optical scanners had been miscalibrated and were failing to detect the dye-based ink commonly used in gel pens. The ensuing recount recovered almost 6700 votes (but no outcomes changed).

2 : For instance, in Minnesota after the 2006 general election, the cost of the wages for election judges (pollworkers) to count votes has been estimated at $24,500 to $27,000 statewide – 9 to 10 cents per hand-counted vote, and about 1.2 cents per voter in the election (http://www.ceimn.org/files/CEIMNAuditReport2006.pdf). While audit costs will vary depending on the scope of the audits and other considerations, they can be expected to be a small fraction of election administration costs.

3 : We will use “contest” to refer to any ballot item (such as an election to public office or a ballot initiative) – not to a challenge to the results, as in some states.

4 : The proposal of best practices for auditing a given system does not imply an endorsement of the system.

5 : In post-election audits, each ballot (or paper record) is assigned to an audit unit – a group of paper records from a precinct, counting machine, or batch of ballots. On batches, see the discussion in Best Practice 5e.

6 : In addition to the number of miscounts per machine and the analysis of the source of these discrepancies, it is important to collect and report the number of spoiled ballots, canceled VVPATs, unreadable VVPATs, overvotes, undervotes and voter-mismarked paper ballots (for instance, if the candidate’s name is circled but the oval is left blank).

7 : Procedures regulating access to ballots and equipment could include requiring signatures for access and documenting the reason for it, preventing access by a single person, requiring that access be observed by members of opposing parties, or the use of surveillance cameras to guard storage areas.

8 : This includes the expiration of all legal recourse to challenge or correct the election.

9 : “Outcome” refers to which candidates or ballot propositions won or lost, not necessarily a specific vote tally. Here we refer to the outcome as “correct” or “incorrect” depending on whether it corresponds with what would be the outcome from a complete manual recount. Note that the outcome from a complete manual recount may not always match the will of the voters. To ensure that outcomes reflect the will of the voters, additional conditions must be met including rigorous ballot accounting, accurate registration data, elimination of unreasonable delays at the polls, good ballot design, and controls on chain of custody for all election equipment and materials.

10 : Fixed-percentage samples are inadequate for risk-limiting audits, because the audit size needed to verify an election outcome depends on the apparent margin of victory, as well as the number of audit units and the amount of error each audit unit can harbor. However, auditing some minimum percentage of votes or audit units regardless of jurisdictional size or election margin may be useful to monitor election accuracy. Generally, requiring a smaller chance of error (e.g. 1% versus 5%) will entail auditing more ballots.

11 : If audit results indicate that the initial outcome is incorrect, ultimately a full recount would be required to determine the final outcome. Preliminary outcomes cannot be overturned based on audit samples alone.

12 : In the selection, some units may be weighted more than others based on their size and the amount of error they could harbor. Random sampling is unnecessary if all audit units will be manually counted, or if so many audit units are counted that the remaining units cannot change the outcome.

13 : Discrepancies found during the audit can also affect the sample size, as discussed in 6a.

14 : All else being equal, contests spanning fewer audit units – for instance, local contests as opposed to statewide contests – require proportionally larger audits to ensure that the chance of confirming an outcome that is incorrect is low.

15 : “Manual counting” or “hand counting” refers to human visual inspection of paper records to interpret voter intent, followed by a tabulation of the individual vote interpretations. Only the tabulation portion is sometimes assisted by independent and well trusted equipment such as calculators and spreadsheets. All hand counts should be done blind to the expected result.

16 : When auditing less common ballot types or very small precincts, care must be taken to preserve voter anonymity and the secrecy of the individual voter’s ballot. Also, it may be possible to confirm the election outcome without sampling some types of ballots, if these types do not contain enough ballots to alter the outcome (see footnote 12). However, for fairness and to provide valuable information about the quality of the election process, all ballot types should be routinely audited.

17 : In all cases, voter-verified paper ballots or records must be available for the audit. Auditability — the ability to conduct reliable and efficient audits — should be a crucial criterion when selecting voting technologies.

18 : One way to contain the cost of targeted samples is to require that the requesting candidate or group pay for the additional ballots to be audited. Such a law was passed in Minnesota in the 2008 legislative session; under this law, the requesting candidate would be refunded by the jurisdiction conducting the recount if the recount leads to the initial result being overturned.

---