AuditEngine
This Page: copswiki.org/Common/AuditEngine
AuditEngine is an independent, comprehensive ballot-image auditing system developed by Citizens' Oversight. It examines digital images of paper ballots, independently determines the votes recorded on each ballot, and compares those findings with the official cast vote records (CVRs) produced by the voting system.
AuditEngine can analyze both hand-marked paper ballots and ballots produced by ballot marking devices (BMDs). For BMD ballots, it reads the human-readable selections rather than relying on encoded barcodes or QR codes.
The resulting reports identify discrepancies, support reconciliation of election records, and help election officials and the public evaluate the accuracy of reported election results.
How AuditEngine Works
AuditEngine can be understood as a series of four major phases.
Input Data
AuditEngine requires two primary data sources, both exported from the voting system:
- Cast Vote Records (CVRs): Machine-readable records containing the votes interpreted by the voting system, along with associated metadata such as precinct, ballot style, and voting group. CVRs are typically provided in CSV, spreadsheet-compatible, or JSON formats.
- Ballot Image Archives (BIAs): Digital images of ballot sheets produced by the voting system, typically including both front and back images, organized into one or more ZIP archives.
Phase 1: Indexing
AuditEngine examines the CVR metadata and ballot image archives to create a unified index of ballot sheets. The votes reported in the CVR are kept separate from the independent vote-extraction process until Phase 4.
- CVR Metadata: Extracts ballot identifiers, precincts, ballot styles, and other metadata without using the CVR vote selections.
- Image Inspection: Examines every ballot image to identify its dimensions, encoded ballot-style information, and cryptographic hash.
- Ballot Classification: Distinguishes full-face ballots from barcoded ballots.
- Detect Exact Repeats: Identifies repeated ballot identifiers and identical ballot images.
- Indexing Report and Election Information File (EIF): Summarizes the input data and creates an EIF describing the election's contests and candidate or response options.
Phase 2: Mapping
- Ballot Layout Analysis: Identifies ballot layouts and the locations of voting targets, such as ovals.
- Optical Character Recognition (OCR) Prep: References for comparing printed text to identify voter selections without using barcodes.
- Independent Mapping: Associates voting targets with contests and options without relying on the voting system's proprietary target maps or encoded vote selections.
- Templates: Creates a template for each ballot style, combining many images of that style.
- Automated and Manual Mapping: Uses automated analysis and, when necessary, the browser-based TargetMapper application to establish accurate mappings.
- Proof Review: Generates visual proofs that reviewers can inspect to verify the accuracy of ballot mappings.
Phase 3: Extraction
- Hand-Marked Ballots: Examines the original ballot images and evaluates voter marks at the mapped target locations.
- BMD Ballots: Uses OCR to interpret the human-readable vote selections printed by ballot marking devices, without relying on barcodes or QR codes.
- Parallel Processing: Processes ballot images concurrently using scalable computing resources, allowing large elections to be analyzed efficiently.
Phase 4: Comparison
- Independent Comparison: Compares AuditEngine's independently extracted vote selections with the corresponding official CVR selections.
- CVR Vote Isolation: The official CVR votes enter the audit process at this stage, after independent extraction is complete.
- Discrepancy Classification: Identifies and categorizes differences between the two interpretations, including missing records, unmatched ballots, and vote-selection discrepancies.
- Repeated Scanned Ballots: Identifies batches scanned twice and adjusts reports.
- Reconciliation and Contest Reports: Produces detailed reports of discrepancies, ballot-level findings, and contest-level results, including differences that could affect election outcomes.
Optional Input Data
- File Hash Values: We prefer to have a file of hash values we can check with the official ballot images and cast vote records.
- Ballot Style Masters: Mapping can use these files to assist our own automated approach, particularly if the images have invisible ovals or an unusual layout, such as landscape grid layouts.
- Results Summary Files: We can parse a variety of files providing the aggregated results, which may differ from the CVR.
- Digital Poll Tapes: Polling place reports generated by the Election Management System, as a cross-check.
- Verification Scans: Batches of ballots can be scanned manually and provided as verification scans, for comparison with the voting system's batches.
More on each phase: How AuditEngine Works, in the AuditEngine documentation.
How the Voting System and AuditEngine Work Together
The diagram illustrates two separate processes.
Voting System (upper portion): The voting system scans paper ballots, produces digital ballot images, interprets vote selections, and generates cast vote records (CVRs). The CVRs are used to tabulate the official election results.
AuditEngine (lower portion): AuditEngine receives the ballot images and CVR files, analyzes ballot layouts, and independently extracts vote selections from the images. During this independent processing, the official CVR vote selections remain separate.
Only after independent extraction is complete does AuditEngine compare its findings with the official CVRs. Differences are classified and reported for further examination, including human review when appropriate.

Audit Projects and Results
- Auditing the 2026 General Election with AuditEngine: records requests to counties, and the offer to audit Alaska.
- Audits of Prior Elections: each election audited so far, where its audit stands, and its reports.
- San Francisco Ballot Image Audits: the 2024 General Election audit, including batches scanned twice.
- Audit Projects on AuditEngine.org: the audits done with AuditEngine.
AuditEngine Resources and Documentation
AuditEngine Website
- AuditEngine.org: visit the official website.
Technical Documentation
- AuditEngine Documentation: how AuditEngine works and how to use it.
- AuditEngine Background: start here.
- Frequently Asked Questions
- Getting the data together: the files an audit needs.
- Exporting: step-by-step exports from each voting system.
- Uploading: sending the files to AuditEngine.
- Creating an account, a job, and processing an election
- Election Information File and Ballot Options File
AuditEngine Application
- The AuditEngine app: for authorized users, to upload election files and start audits.
How you can help
AuditEngine depends on collaboration among election officials, technical contributors, researchers, and members of the public. Volunteers can help obtain election records, process ballot images, review audit results, and investigate discrepancies. The roles below describe ways to participate and the experience required. To help, complete the form: SignUp
| Role | Responsibilities | Experience Needed |
|---|---|---|
| Public Advocate | Encourage election officials to preserve ballot images and make them available for independent review. | None, but may need attorneys to assist when officials claim that ballot images and cast-vote record files are not public records or cannot be made available for review. |
| Requestor | Submit public-records requests for ballot image archives and CVRs. | Limited technical expertise, but requires knowledge of public records requests and the desired file types, flash drives, and zip files. Assistance from attorneys may be required if officials resist. |
| Uploader | Organize and upload election data into AuditEngine. | Limited technical expertise, no legal background required. |
| Observer | Monitor audit progress and review publicly available findings. | Common sense |
| Auditor | Configure audit jobs and supervise processing. | Moderate technical expertise, knowledge of ballot formats and program settings. |
| Mapper | Identify ballot targets and associate them with contests and options using TargetMapper. | Detail-oriented and able to use a mouse. |
| Reviewer | Verify mapping proofs and examine reported discrepancies. | Must be detail-oriented. |
| Adjudicator | Evaluate disputed ballot interpretations according to applicable election rules. | Common sense required. |
| Subscriber | Follow audit projects and review published reports. | None required. |
Historical Reports and Announcements
2020 General Election Audit Reports
AuditEngine's 2022 reports on the 2020 General Election in Bartow County, GA, Fulton County, GA, and Dane County, WI.


M1986 2022-09-30 2020 General Election Audit Reports for Bartow GA, Fulton GA and Dane WI -- RayLutz CitizensOversight
AuditEngine Case Study Report
This 2021 case study examines AuditEngine's analysis of elections in Collier, Volusia, and St. Lucie Counties, Florida. The report compares AuditEngine's interpretations with official voting-system records and examines cases in which the two systems disagreed.
Historical introduction: 2021 Florida case study and videos


M1970 2021-05-29 AuditEngine Three Florida Counties -- Case Study Report & Videos (M1970) -- RayLutz CitizensOversight
The first video is a good introduction to the case study:
- Part 1: Introduction and Theory of Operation: https://youtu.be/dDEFGsqdCsg
AuditEngine Frequently Asked Questions
We have started to compile the questions that tend to asked on this page: AuditEngineFAQs
AuditEngine Press and Announcements
- Jennifer Cohn interview on AuditEngine:
- AuditEngine was first announced and demonstrated at the National Election Integrity Conference in Berkeley, CA on Oct 6, 2019, which was sponsored by the National Voter Rights Task Force. Read press release here: Press Release.
Summary of articles submitted for this project:
The following archive contains AuditEngine reports, correspondence, research, announcements, and supporting documents, listed from newest to oldest.
M2070 2026-11-16 Letter to San Francisco: Batches Scanned Twice in the November 5, 2024 General Election -- RayLutz CitizensOversight
M2068 2026-11-11 Audits of Prior Elections -- RayLutz CitizensOversight
M2069 2026-11-11 San Francisco Ballot Image Audits -- RayLutz CitizensOversight
M2067 2026-11-06 Auditing the 2026 General Election with AuditEngine -- RayLutz CitizensOversight 
M2066 2026-11-05 Offer To Audit The 2026 General Election Using Ballot Images -- RayLutz CitizensOversight
M2064 2026-09-29 Letter to Thad Kousser and Lauren Prather at Center for Transparent and Trusted Elections at UCSD -- RaymondLutz CitizensOversight 
M2062 2026-09-03 Letter to Shawn K. Brom, San Diego ROV, RE: RIVERI -- RaymondLutz CitizensOversight 
M2065 2026-06-19 Critical Ballot Design Issues in New Jersey -- RayLutz CitizensOversight 
M2051 2026-04-06 Passaic County NJ BIA Narrative Report 2025-11-04 -- RayLutz CitizensOversight 
M2052 2026-04-06 Hudson County NJ BIA Narrative Report 2025-11-04 -- RayLutz CitizensOversight 
M2036 2025-09-02 Audits of 2024 Election in Burlington and Sussex Counties, NJ -- RayLutz CitizensOversight
M2018 2024-08-06 Letter to Jocelyn Benson, MI SOS -- RayLutz CitizensOversight 
M2017 2024-07-30 Letter to Joseph Kirk 20240730 -- RayLutz CitizensOversight 
M2016 2024-06-25 AuditEngine for Georgia (Proposal) (M2016) -- RayLutz CitizensOversight 
M2014 2024-06-18 Letter to Janice Johnston re BIA Classifications, Dallas Data, GA pilot (M2014) -- RayLutz CitizensOversight 
M2015 2024-06-18 Letter to Monmouth County Board of Elections re AuditEngine (M2015) -- RayLutz CitizensOversight 
M2012 2024-06-01 Letter to Ken Bennett re: BIA Classifications, Dallas Data, AZ pilot (M2012) -- RayLutz CitizensOversight 
M2011 2024-04-09 To Raffensperger -- GA Ballot Image Audits including OCR of BMD ballots in light of HB 974 (M2011) -- RayLutz CitizensOversight 
M2010 2024-01-23 To GA SOS Raffensperger re BMD Audits (M2010) -- RayLutz CitizensOversight 
M2008 2023-12-21 Letter to El Paso County Clerk Chuck Broerman -- RayLutz CitizensOversight 
M2007 2023-12-12 !AuditEngine Findings -- RayLutz CitizensOversight 
M2005 2023-12-08 MD Pilot Audit 20231107 Narrative Report -- RayLutz CitizensOversight 
M2030 2023-12-05 Pr Rel- Citizens Oversight Announces Independent Audits of 2024 Election -- RayLutz CitizensOversight 
M2004 2023-12-05 Letter to CO SOS Griswold re Arvada Mayor's Contest 2023 -- RayLutz CitizensOversight 
M2003 2023-12-05 Letter to PA re Northampton County -- RayLutz CitizensOversight 
M2000 2023-08-29 Letter To Monmouth County Clerk (M2000) -- RayLutz CitizensOversight 
M1999 2023-07-31 Letter to State Board of election of NY re ExpressVote XL -- RayLutz CitizensOversight
M2009 2023-07-04 NJ 2022 Audit of Three Counties, Burlington, Mercer, Monmouth -- RayLutz CitizensOversight 
M1989 2022-11-06 Dual Tabulation Issue in ES&S Voting Equipment -- RayLutz CitizensOversight
M1988 2022-10-03 Election Data Use Case for IETF SCITT Working Group -- RayLutz CitizensOversight 

M1986 2022-09-30 2020 General Election Audit Reports for Bartow GA, Fulton GA and Dane WI -- RayLutz CitizensOversight
M1985 2022-05-27 Letter to Clackamas County, OR regarding blurry barcodes issue -- RayLutz CitizensOversight 
M1984 2022-05-26 Election Hazards -- RayLutz CitizensOversight 
M1983 2022-02-18 GA_Bartow_20201103 Audit Narrative Report -- RayLutz CitizensOversight
M1982 2022-01-27 GA_Bartow_20201103 Audit Report -- RayLutz CitizensOversight 
M1980 2021-10-03 Review of Arizona Audit of the 2020 General Election in Maricopa County -- RayLutz CitizensOversight
VoterIntent 2021-09-16 Voter Intent -- RayLutz CitizensOversight 
M1979 2021-08-28 Ballot auditor: Volusia County supervisor of elections did a good job -- Noah Hertz The West Volusia Beacon 
M1976 2021-07-20 Critical Review of the paper 'Unclear Ballot' -- RayLutz CitizensOversight 
M1975 2021-06-29 Letter to AZ Sen. Karen Fann re: Deep Rig False Claims -- RayLutz CitizensOversight 
M1974 2021-06-25 Letter to AZ Sen. Karen Fann re: Arizona Statewide Ballot Image Audit -- RayLutz CitizensOversight
M1973 2021-06-15 Ballot Images in Florida Regulations -- RayLutz CitizensOversight 
M1972 2021-06-15 Letter to AZ Sen. Karen Fann re: Maricopa Ballot Image Audit -- RayLutz CitizensOversight
AuditEngineFAQs 2021-06-11 AuditEngine Frequently Asked Questions -- RayLutz CitizensOversight 

M1970 2021-05-29 AuditEngine Three Florida Counties -- Case Study Report & Videos (M1970) -- RayLutz CitizensOversight- M1971 2021-05-28 Another election audit coming? Arizona Senate considers using new technology for digital recount -- Jen Fifield Robert Anglen ArizonaRepublic
- M1969 2021-05-28 Maricopa County Audit -- 2020 General Election -- RayLutz CitizensOversight

M1968 2021-04-13 Ballot Images: Essential or Worthless? -- RayLutz CitizensOversight
M1967 2021-04-08 Public Records Request to Bartow County GA for 2020 Ballot Images -- RayLutz CitizensOversight - M1965 2020-12-22 Audits in 2020 General Election -- RayLutz CitizensOversight
M1966 2020-11-30 Video of SB 283 High Speed Scanner Test in New Hampshire -- CitizensOversight - OversightFor2020GeneralElection 2020-10-29 Oversight for 2020 General Election -- RayLutz CitizensOversight
- M1953 2020-08-20 Instructions to upload to Audit Engine -- RayLutz CitizensOversight

M1947 2020-07-07 Letter to Gov. Newsom to include all ballots in the audit -- RayLutz CitizensOversight
M1946 2020-06-22 Citizens' Oversight's Comments on VVSG 2.0 Draft -- RayLutz CitizensOversight
CitizensOversightStatusUpdate2019 2020-04-13 Citizens Oversight Status Update 2019 -- RayLutz CitizensOversight 
M1943 2020-03-18 Request for 2020 Primary Election Ballot Images to CA Counties -- RayLutz CitizensOversight
M1940 2019-12-06 Uniform Audit Report -- RayLutz CitizensOversight
M1938 2019-11-30 The Four Fatal Flaws of RLA Audits -- RayLutz CitizensOversight
M1936 2019-11-11 Securing Digital Ballot Images to Enable Auditing -- RayLutz CitizensOversight 
M1931 2019-11-04 Earth Summit Presentation: Elections: Stop Scams with Oversight and Audits -- RayLutz CitizensOversight 
M1928 2019-10-05 !AuditEngine Announcement at 3rd National Election Integrity Conference, Berkeley -- RayLutz CitizensOversight
M1917 2019-08-19 Ballot Image Samples -- RayLutz CitizensOversight 
M1913 2019-08-12 Comprehensive Risk Estimation of Election Audits -- RayLutz CitizensOversight
M1939 2019-06-15 Uniform Audit Tally Sheets -- RayLutz CitizensOversight 
M1895 2019-03-17 Letter to OC ROV requesting Ballot Images -- RayLutz CitizensOversight
M1949 2017-12-29 Random drawing scheduled to break tie in disputed Va. House race -- Denise Lavoie AssociatedPress
Discussion List
See ListServe for all email discussion lists and to add more.
- ESS_ Exporting Guide for AuditEngine.pdf: ESS_ Exporting Guide for AuditEngine.pdf
- Sending Election Data to AuditEngine.pdf: Sending Election Data to AuditEngine.pdf
