Election Audits

Direct link to this page: https://copswiki.org/Common/ElectionAudits

Latest News

This project provides a place to gather up and organize links to various initiatives regarding audits, including statistical and exhaustive approaches.

Background

• http://bcn.boulder.co.us/~neal/electionaudits/rlacalc.html -- rlacalc: Online Calculator for Risk-Limiting Audit Sample Sizes
• https://www.usenix.org/legacy/events/evtwote10/tech/full_papers/Stark.pdf -- Stark.pdf - "Super-Simple Simultaneous Single-Ballot Risk-Limiting Audits"
• https://www.stat.berkeley.edu/~stark/Preprints/gentle12.pdf -- A Gentle Introduction to Risk-limiting Audits -- Mark Lindeman and Philip B. Stark
• https://github.com/ron-rivest/2018-bctool/blob/master/code/bctool.py -- Bayesian Comparison audit tool
• https://cran.r-project.org/web/packages/bayesm/bayesm.pdf -- Bayesian toolkit

SUITE: Risk-Limiting Audits by Stratified Union-Intersection Tests of Elections (SUITE) | Springer Link

• https://link.springer.com/chapter/10.1007/978-3-030-00419-4_12
• https://arxiv.org/pdf/1809.04235.pdf

Several other methods at:

• Next Steps for the Colorado Risk-Limiting Audit (CORLA) Program
• https://arxiv.org/abs/1803.00698

Work continues studying Bayesian methods that are much more efficient, but aren't yet known to be risk-limiting.

---

From Harvie Branscomb:

The following linked document is a comprehensive set of steps for a risk limiting ballot comparison tabulation audit that I assembled prior to the second meeting of the audit advisory group appointed by the CO SOS in January 2017. I believe it largely complies with the guidelines listed in the Lutz memo just sent out. And the CO audit is actually being implemented along the lines of these steps as well. The major differences between these steps and the actual implementation are the use of the RLATool special purpose software rather than basic paper forms for capture of sampled voter intent, and the selection of contests to drive the audit done by the SOS rather than by choice of narrowest margin countywide contest. I think it may be useful for designers of other risk limiting post election tabulation audits to read this document:
https://www.sos.state.co.us/pubs/elections/VotingSystems/riskAuditFiles/20170217Branscomb-StepsRiskLimitingVoteTabulationAudit.pdf

There is a shorter more abstracted version was also contemporaneously created by members of SAWG linked here:
https://www.sos.state.co.us/pubs/elections/VotingSystems/riskAuditFiles/20170217McBurnett-BriefCORLASummary.pdf

The public records of the audit advisory group including the two above are located here (many other comment documents may be useful):
https://www.sos.state.co.us/pubs/elections/VotingSystems/riskAuditGroup.html

There is a vast public record rulemaking that implemented regulations for the Colorado audit.
There is extensive written testimony regarding proposed rules for implementation of the RLTA such as this here:
http://www.sos.state.co.us/pubs/rule_making/written_comments/2017/20170718BranscombMcCarthy.pdf

and in more detail here, focusing on the discussion of the text of Rule 25 that implements the CO RLTA:
http://www.sos.state.co.us/pubs/rule_making/written_comments/2017/20170710Branscomb.pdf

and this followup to the above in response to comments by Boulder County:
http://www.sos.state.co.us/pubs/rule_making/written_comments/2017/20170717Branscomb.pdf

The following linked writing discusses the need for disassociation of tabulation batches from eligibility batches, both in numbering and in content such that batches cannot be linked in a mail ballot or central count process with mixed styles (e.g.vote center). Sufficient disassociation will allow paper ballots to be viewed in public, as the best practices for audit require:
http://www.sos.state.co.us/pubs/rule_making/written_comments/2017/20170718Branscomb.pdf

Above documents are all from this rulemaking page:
http://www.sos.state.co.us/pubs/rule_making/hearings/2017/ElectionsRulesHearing20170711.html

And the the resulting Rule 25 that implements the audit in current form is found here:
http://www.sos.state.co.us/pubs/rule_making/CurrentRules/8CCR1505-1/Rule25.pdf

I think Ray has asked me to provide this kind of material in the past, and I apologize for not sending it earlier. Harvie Branscomb

---

Colorado Audt Center

• http://www.sos.state.co.us/pubs/elections/auditCenter.html -- will take some work to fully undestand what they are providing and if we can provide sufficient oversight.
• Colorado RLA (CORLA) -- http://bcn.boulder.co.us/~neal/elections/corla/

Summary of articles submitted (Add | All):

• (M2042) 2025-12-04 Ballot Image Audit of 2023-11-07 election in Broomfield County, CO -- Raymond C Lutz, Citizens Oversight,
• (M2040) 2025-10-30 Oversight of San Diego, 2025 General Election -- Ray Lutz, Citizens Oversight,
• (M2039) 2025-10-23 Analysis of CA in 2024 General Election -- , ,
• (M2036) 2025-09-02 Audits of 2024 Election in Burlington and Sussex Counties, NJ -- Ray Lutz, Citizens Oversight,
• (M2033) 2025-02-17 Clark County Inquiry Regarding the 2024 Election -- Ray Lutz, Citizens Oversight,
• (M2032) 2025-01-30 Public Records Request to NV SOS Regarding 2024 Election in NV -- Ray Lutz, Citizens Oversight,
• (M2027) 2024-09-17 Comparison of Ballot Image Viewers -- Ray Lutz, Citizens Oversight,
• (M2012) 2024-06-01 Letter to Ken Bennett re: BIA Classifications, Dallas Data, AZ pilot (M2012) -- Ray Lutz, Citizens Oversight,
• (M2011) 2024-04-09 To Raffensperger -- GA Ballot Image Audits including OCR of BMD ballots in light of HB 974 (M2011) -- Ray Lutz, Citizens Oversight,
• (M2020) 2024-03-05 Dallas, TX 2024-03-05 Primary Election Data -- Ray Lutz, Citizens Oversight,
• (M2004) 2023-12-05 Letter to CO SOS Griswold re Arvada Mayor's Contest 2023 -- Ray Lutz, Citizens Oversight,
• (M2030) 2023-12-05 Pr Rel- Citizens Oversight Announces Independent Audits of 2024 Election -- Ray Lutz, Citizens Oversight,
• (M2026) 2023-10 Technical Report: 100% Retabulation Audits: 2022 Primary and General Election Audit Data and Ballot Images from Leon County, FL -- Lonna Rae Atkeson, et al, Florida State University, Includes a review and summary of the document.
• (M2000) 2023-08-29 Letter To Monmouth County Clerk (M2000) -- Ray Lutz, Citizens Oversight,
• (M2009) 2023-07-04 NJ 2022 Audit of Three Counties, Burlington, Mercer, Monmouth -- Ray Lutz, Citizens Oversight,
• (M2041) 2022-11-17 1% Manual Tally Observations in San Diego -- 2022-11-17 -- Ray Lutz, Citizens Oversight,
• (M2019) 2022-11-12 CVR Analysis Mistakes -- Ray Lutz, Citizens Oversight, Review of Bogus CVR analysis done by Phil Evans and Jeff "the Racoon" O'Donnell
• (M1989) 2022-11-06 Dual Tabulation Issue in ES&S Voting Equipment -- Ray Lutz, Citizens Oversight,
• (M1986) 2022-09-30 2020 General Election Audit Reports for Bartow GA, Fulton GA and Dane WI -- Ray Lutz, Citizens Oversight, Includes general background and narrative reports for Bartow GA, Fulton GA and Dane WI
• (M1997) 2022-08-12 The Antrim County 2020 Election Incident: An Independent Forensic Investigation -- J. Alex Halderman, University of Michigan,
• (M1987) 2022-07-08 Review of Jovan Pulitzer's Paper Analysis Report for Maricopa County -- Ray Lutz, Citizens Oversight,
• (M1984) 2022-05-26 Election Hazards -- Ray Lutz, Citizens Oversight,
• (M1983) 2022-02-18 GA_Bartow_20201103 Audit Narrative Report -- Ray Lutz, Citizens Oversight,
• (M1982) 2022-01-27 GA_Bartow_20201103 Audit Report -- Ray Lutz, Citizens Oversight,
• (M1980) 2021-10-03 Review of Arizona Audit of the 2020 General Election in Maricopa County -- Ray Lutz, Citizens Oversight, Maricopa County Audit by Cyber Ninjas
• (VoterIntent) 2021-09-16 Voter Intent -- Ray Lutz, Citizens Oversight,
• (M1978) 2021-09-10 Letter to SOS Weber regarding Recall auditing procedures. -- Ray Lutz, Citizens Oversight,
• (M1979) 2021-08-28 Ballot auditor: Volusia County supervisor of elections did a good job -- Noah Hertz, The West Volusia Beacon,
• (M1977) 2021-08-05 Comments by Citizens' Oversight on RLA regulations -- Ray Lutz, Citizens Oversight,
• (M1976) 2021-07-20 Critical Review of the paper 'Unclear Ballot' -- Ray Lutz, Citizens Oversight,
• (M1973) 2021-06-15 Ballot Images in Florida Regulations -- Ray Lutz, Citizens Oversight,
• (AuditEngineFAQs) 2021-06-11 AuditEngine Frequently Asked Questions -- Ray Lutz, Citizens Oversight, Included from rendered markdown.
• (M1970) 2021-05-29 AuditEngine Three Florida Counties -- Case Study Report & Videos (M1970) -- Ray Lutz, Citizens Oversight, Includes Volusia, Collier, and Port St. Lucie Counties
• (M1969) 2021-05-28 Maricopa County Audit -- 2020 General Election -- Ray Lutz, Citizens Oversight,
• (M1971) 2021-05-28 Another election audit coming? Arizona Senate considers using new technology for digital recount -- Jen Fifield Robert Anglen, Arizona Republic,
• (M1968) 2021-04-13 Ballot Images: Essential or Worthless? -- Ray Lutz, Citizens Oversight,
• (M1967) 2021-04-08 Public Records Request to Bartow County GA for 2020 Ballot Images -- Ray Lutz, Citizens Oversight,
• (M1965) 2020-12-22 Audits in 2020 General Election -- Ray Lutz, Citizens Oversight,
• (M1954) 2020-11-03 Tabulation Audit Oversight and Snapshot Protocol Tutorial -- Ray Lutz, Citizens Oversight,
• (OversightFor2020GeneralElection) 2020-10-29 Oversight for 2020 General Election -- Ray Lutz, Citizens Oversight,
• (M1950) 2020-07-11 2020 General Election Oversight Plan -- Ray Lutz, Citizens Oversight,
• (M1947) 2020-07-07 Letter to Gov. Newsom to include all ballots in the audit -- Ray Lutz, Citizens Oversight, Please read this topic for specific actions to help us make a difference!
• (M1946) 2020-06-22 Citizens' Oversight's Comments on VVSG 2.0 Draft -- Ray Lutz, Citizens Oversight,
• (M1981) 2020-06-22 Guidelines and Terminology on the Ease of Voter-verification -- Ray Lutz, Citizens Oversight, Voter Intent
• (M1944) 2020-03-26 Comment on AB-2400, RLA revisions -- Ray Lutz, Citizens Oversight,
• (M1943) 2020-03-18 Request for 2020 Primary Election Ballot Images to CA Counties -- Ray Lutz, Citizens Oversight,
• (M1940) 2019-12-06 Uniform Audit Report -- Ray Lutz, Citizens Oversight,
• (M1938) 2019-11-30 The Four Fatal Flaws of RLA Audits -- Ray Lutz, Citizens Oversight,
• (M1936) 2019-11-11 Securing Digital Ballot Images to Enable Auditing -- Ray Lutz, Citizens Oversight,
• (M1931) 2019-11-04 Earth Summit Presentation: Elections: Stop Scams with Oversight and Audits -- Ray Lutz, Citizens Oversight,
• (M1928) 2019-10-05 AuditEngine Announcement at 3rd National Election Integrity Conference, Berkeley -- Ray Lutz, Citizens Oversight,
• (M1926) 2019-09-19 Staff Report and Comments by Ray Lutz at Rhode Island BOE on Audits -- Ray Lutz, Citizens Oversight,
• (M1923) 2019-09-13 ES&S Voting Machine Equipment Demo -- Ray Lutz, Citizens Oversight,
• (M1925) 2019-09-13 Clear Ballot Voting System Demonstration -- Ray Lutz, Citizens Oversight,
• (M1919) 2019-08-21 The Surprising Wisdom of Batch Comparison Election Audits -- Ray Lutz, Citizens Oversight,
• (M1917) 2019-08-19 Ballot Image Samples -- Ray Lutz, Citizens Oversight,
• (M1916) 2019-08-18 Tutorial on Risk Limiting Audits -- Ballot Polling -- Ray Lutz, Citizens Oversight,
• (M1914) 2019-08-16 Letter to Rhode Island Board of Elections regarding Risk Limiting Audits -- Ray Lutz, Citizens Oversight,
• (M1918) 2019-08-14 Rhode Island Board of Elections on RLA report -- Rhode Island Board of Elections, Citizens Oversight,
• (M1913) 2019-08-12 Comprehensive Risk Estimation of Election Audits -- Ray Lutz, Citizens Oversight,
• (M1915) 2019-08-07 Presentation by Steven Rosenfeld on Ballot Image Audits in Maryland -- Steven Rosenfeld, Audit Usa, This video is queued to start at about 51 minute offset. Powerpoint slides from Maryland included
• (M1939) 2019-06-15 Uniform Audit Tally Sheets -- Ray Lutz, Citizens Oversight,
• (M1900) 2019-04-09 Letter to San Diego City Council re: Item 53 on April 9 (Election Integrity) -- Ray Lutz, Citizens Oversight,
• (M1899) 2019-04-05 Letter to SDROV requesting party affiliation breakdown by precinct for years 2012-2018 -- Ray Lutz, Citizens Oversight, Tracking #: 9214890142980447171210
• (M1896) 2019-03-26 Letter to San Diego BOS on Election Integrity for 2019-03-26 meeting -- Ray Lutz, Citizens Oversight,
• (M1893) 2019-03-17 Letter to LA ROV requesting Batch Report for Nov 2018 -- Ray Lutz, Citizens Oversight,
• (M1894) 2019-03-17 2019-03-17 (M1894) Letter to SD ROV requesting Batch report -- Ray Lutz, Citizens Oversight,
• (M1895) 2019-03-17 Letter to OC ROV requesting Ballot Images -- Ray Lutz, Citizens Oversight,
• (M1892) 2019-03-10 Election Integrity Update -- Ray Lutz -- Ray Lutz, Citizens Oversight,
• (M1879) 2018-09-10 White Paper: Election Audit Strategy and BRAWL -- Balanced Risk Audit with Workload Limitation -- Ray Lutz, Citizens Oversight,
• (M1949) 2017-12-29 Random drawing scheduled to break tie in disputed Va. House race -- Denise Lavoie, Associated Press, VA 94th District House of Delegates was at first one vote in favor of Shelly Simonds, then they found this one questionable ballot
• (M1884) 2017-01-31 CLIP Audit -- Ronald Rivest, MIT,
• (M1693) 2016-09-06 CPRA -- Procedures of San Diego Registrar of Voters (2016) -- Ray Lutz, Citizens Oversight,
• (ElectionAuditBestPractices) 2016-08-08 Election Audit Best Practices -- Ray Lutz, Citizens Oversight,
• (M1678) 2012-10 Risk-Limiting Post-Election Audits: Why and How -- Risk-Limiting Audits Working Group, Risk-Limiting Audits Working Group,
• (M1807) 2012 BRAVO: Ballot-polling Risk-limiting Audits to Verify Outcomes -- Mark Lindeman, Philip Stark Vincent Yates, Department of Statistics University of California, Berkeley,
• (M1729) 2008-09 Principles and Best Practices for Post-Election Audits -- , electionaudits.org,
• (M1503) 2008-07-01 Improving the Security, Transparency and Efficiency of California's 1% Manual Tally Procedures -- Joseph Lorenzo Hall, UC Berkeley School of Information, Manual Tally Procedures
• (M1683) 2008 Procedures for California's 1% Manual Tally -- Joseph Hall, UCBerkeley,
• (M1883) 2007-11-05 Elections board workers take plea deal -- Karl Turner, cleveland.com, Features Michael Vu
• (M1922) 2000-04-05 Probabilistic Risk Assessment: What Is It And Why Is It Worth Performing It? -- Dr. Michael Stamatelatos, NASA Office of Safety and Mission Assurance,

Discussion List

See List Serve for all email discussion lists and to add more.