Technical Report: 100% Retabulation Audits: 2022 Primary and General Election Audit Data and Ballot Images from Leon County, FL

Florida State University (2023-10) Lonna Rae Atkeson, et al

This Page: https://copswiki.org/Common/M2026
Media Link: https://2022voterdata.lci.fsu.edu/wp-content/uploads/2023/10/Technical_Report_LA4.pdf%20type%3d%22html%22
More Info: Election Integrity

Abstract: We developed a 2022 audit primary and a general election data dashboard that includes images of the ballots voted in Leon County. These data are embedded in a web page on a website we built for this purpose. Additional pages on the website describe postelection audit processes, data produced by election officials to verify the election outcomes, interesting ballots found in the data, and information on how to use the dashboard. Throughout the project, we faced many obstacles to presenting these types of data in an accessible format, including protecting voter privacy and ballot secrecy, correcting ballot orientation, managing the quantity of the data presented, speed and performance in data presentation, developing the dashboard design, and creating a web design and useable layout.

Points of interest:

• There is no standardized format across jurisdictions to present and inform voters of the outcome.
• less than half of the states make data from the postelection audits available in usable form for the public or researchers to examine (Jaffe et al 2023)
• Leon County, FL had about 203,000 registered voters and 66,871 ballots cast in the primary and 117,456 votes in the general election.
• The ClearAudit system is used for a ballot image audit (BIA), by rescanning ballots in the central office.
  • Ballot images include an image from both sides of the ballot. (What I have seen on this is a separate image for each side).
  • An audit summary that includes aggregate comparison of the vote totals, undervotes, and overvotes for each candidate between the official vote count and the BIA.
  • Audit cast vote records (CVR). The audit CVR also includes information on precinct, vote mode, and language associated with each ballot.
  • The audit oval confidence files that include a confidence score or ranking for each oval within a contest (i.e., candidate or choice)6, grouped by vote mode (Election Day, Early Voting, or Vote-by-Mail) and ballot disposition or vote type (vote, undervote, overvote, or vote for an alternative candidate).
• The report says "We first cleaned the audit summary ... and merged the audit CVR with audit oval confidence files ... and append the precinct-level election results to the audit CVR and oval confidence files." -- It was not clear if this was done by hand or if they set up automated cleaning, merging and appending.
• They claim that "!ClearAudit software uses a larger zone fully encompassing the oval for each contest to search for and identify a vote for each candidate. ... if someone circles the oval instead of filling it in" -- but we did not see any examples of circled ovals in their oval confidence files.
• The ballots are scanned into the "!ClearAudit Tabulator" and each scan is given a file name based on the order that they were scanned. This means that any paper ballot can be easily accessed later by counting through the stack, as long as the stack is maintained in the same order.
• However, the ballot images cannot be correlated with the voting system CVR, and so there is no ballot-level comparison.

App Layout

top filters:

• Contest Selector, All or select contests.
• Party of Contest (only provided for primary)
• Candidate Selector. (this seems redundant. The Contest selector should be enough.)

contest table

Contest

Candidate

Party

Ovals Counted Dominion

Ovals Counted Clear ballot

Difference in ovals counted

Votes Recorded Dominion

Audit Votes Recorded Clear Audit

Difference in Votes recorded

Over Vote

Under Vote

ballot image filters

• Oval Confidence Rank (all, 1-20) Radio
• Voting Method (Election Day, Early, Vote by Mail) Checkboxes
• Vote Type (Voted for Candidate, Voted for other candidate, overvote, undervote) Checkboxes.

ballot table

Ballot ID

Contest

Candidate Last Name

Vote Type

Ballot Link

Oval Confidence Rank

Voting Method

The ballot table does NOT List one line per ballot, but one line per contest and candidate.

• Unlike other apps, this one includes a map of the county showing the precincts.
• They do not provide a means to filter the results by precinct, and apparently the map is used to select precincts to be included.
• "Because of ballot privacy issues, we had to exclude an interactive map for ballot selection by precinct for the primary election, but it does work for most of the precincts in the general election. For 21 smaller precincts, we had to combine them into groups of two or three to protect voter privacy in the general election"
• The map might be a convenient way to help voters to find results in their area, but this somewhat defeats the purpose.
• Considering results by precinct usually does not provide very much information, and considering results by contest is more important (Ray's Opinion).
• They had a section about "Finding Discrepant Ballots" and they described a very difficult methodology where up to 20 of the ballots with ovals marked with distinctive marks might be compared if they would "sort and search through ballot data and images to attempt to identify any discrepancy." and "We limited it to 20 because after the first few ovals it is impossible for the human eye to discern differences in oval quality."
• They mentioned that "To protect voter privacy, states often have arbitrary reporting laws that specify when precinct information needs to be hidden. Florida has a 30-voter threshold for reporting ballot type or precinct data." However, they did not attempt to evaluate whether this arbitrary criteria was appropriate.
• To comply with the over-reaching state law, they said that for the precinct data, "we removed precinct and party information in the upper righthand corner of the ballot, the tick marks at the bottom of the ballot, and disabled the Leon County precinct map. Finally, we had to withhold 43 ballots because the unique contest combinations on these ballots could reveal individual vote choices and violate voter privacy and state law.
• In the general election, they said they removed precinct numbers and barcodes for 30 precincts out of 135 precincts, affecting 4,422 ballots out of 118,216, or 3.7%.
• They set up a machine learning model using the YOLOv8 platform.
  • They manually reviewed primary data to find signatures.
  • In the general, they used 1500 ballots and simulated signatures, and trained the model.
  • They used the model to locate signatures in the primary ballots and they said the model "performed very well" but did not provide any detailed results.

Obstacles

• The said the data sets were large. However, the number of ballots was sort of average or small and not large. So scaling issues may be a problem.
• Voter privacy issues were difficult to address due to the requirement for at least 30 ballots in the smallest groups.
• They had to develop ballot orientation detection and correction.
• Finding discrepancies was not mentioned but was obviously a big limitation.

Comments