Share Button

The Open Ballot Initiative vs. Hand Counted Paper Ballots

Citizens Oversight (2016-12-25) Ray Lutz

This Page:
More Info: Election Integrity

Here, we compare The Open Ballot Initiative (TOBI) with a commonly promoted approach, All Hand Counted Paper Ballots (AHCPB) in the precinct.

In a nutshell:
  • Both approaches use durable paper ballots.
  • The Open Ballot Initiative:
    • uses document image scanners to create a picture (image) of both sides of each ballot sheet
    • Images are saved likely as a PDF file.
    • Preferably, the ballots are imprinted with a unique sequence number prior to scanning so the number appears in the image, can be found on the ballot, and corresponds with the filename of the image file.
    • The image files are secured against tampering using cryptographic means (SHA signatures & block-chain signature).
    • The resulting images must still be checked for correspondence with the original ballots using a sampling method preferably by a third party.
    • Since ballots can be secured away, ballot images can be made available to the public on a public server.
    • The public can count these "by hand" (by viewing on the screen) if they want to, or they can use software to assist in the process.
    • Includes structured mechanism for continual 100% audits by third parties of the tabulation.
    • This eliminates central tabulator vulnerabilities, reduces oversight costs, and is palatable to election officials who are cost-conscious.
  • All Hand-Counted Paper Ballots (AHCPB)
    • Requires that all ballots are transported to precincts and workers count these by hand, possibly using visual aids (projector, for example).
    • Fastest and most accurate method is "read and tally" method but "sort and stack" is an alternative method that is slower and less accurate.

An example of a PDF file which is an actual image scan from a ES&S DS 200 scanner is attached to this topic. The election industry is moving in this direction as computer recognition of voter intent is much better when a high-resolution image is used rather than simplistic reflected-light detectors as was used in the old scan-tron-like ballot readers.

  • 1433i.pdf: Sample of a PDF containing two sides of a ballot in an image file.

Characteristic TOBI AHCPB Comments
Uses durable paper ballots Yes Yes Both very similar in this regard. Durable paper ballots can be completed by hand or using touch-screen assistant equipment which is coupled with a ballot printer, and prints a full ballot identical to one completed by hand except that the bubbles are filled in without any variation. In the precinct, another approach is to have voters fill in the ballots by hand but run them through a quality-control scanner which checks for over and under votes. Vote-by-Mail (VBM) is extremely popular in CA and rural states, and is the only method used in some states like Oregon. Durable paper ballots are compatible with VBM.
Complexity Higher Lower AHCPB is easy to understand. This is a key benefit to the AHCBP approach over TOBI. However, we believe the concept of capturing all ballot images is still easy to understand.
Supports Early VBM processing Yes Yes* AHCPB requires that VBM ballots that are processed and validated are brought back to the precinct for hand-counting.
Supports Later VBM processing Yes No In CA for example, VBM ballots can be accepted with a valid postmark up to three-days after election day. Also, if no signature is provided, then the voter is notified, and has up to eight days to provide a signature. These ballots, as well as any which are not fully processed in terms of signature validation prior to election day, are not processable by AHCPB in the precinct.
Supports All-Mail precincts Yes No Some states, like Oregon, use all-mail ballot election processing. AHCPB is not compatible with these states. Also, any precincts which are all-mail are not hand-countable in the precinct and would require teams to be set up by elections officials.
Cost Low High AHCPB requires manual labor which is feasible only in small districts and with simple ballots (with limited number of ballot options)
Error Rate ~0% 1 to 2% Although Hand-counted paper ballots is viewed as the "gold standard," in practice counting votes on paper ballots, the error rate is 1% to 2% according to recent scientific studies. OBI has the potential to eliminate nearly all errors due to the redundant recognition phase and comparison between differing groups, and then joint resolution of each ballot where any difference is detected between competing groups. Citizens Oversight conducted a study in San Diego sampling 5% of the precincts (85) in 2008. In the sample, only 15% of the precincts were able to balance the number of ballots issues, voted, spoiled, blank and the number of signatures matched the number of voted ballots and the number of scanned ballots. We have very little confidence that temporary workers in precincts can count the vote on complex, multi-page ballots.
Recount Cost Very Low High This is where TOBI really shines. As seen in recent multi-state recount, it is very expensive ($millions) to recount an entire state. MI charged $125 per precinct to recount. In TOBI, the cost to recount is minimal, and recounts would likely never need to be performed because the process INCLUDES redundant tabulations, i.e continual recounts.
Precinct Chain of Custody Requirements Similar Similar The number of Issued Ballots should equal the number of blank, voted, spoiled, and completed ballots. Also, the number of ballots included in the vote tabulation should equal the number of voted ballots and the number of voters who signed-in.
Post-election day chain of Custody Low High To allow for recounting, AHCPB requires that the ballots remain in secure custody as they are the only record of the election. In TOBI, once the ballots have been resolved to images, the actual physical ballots are only required for the audit process. The image data can be stored redundantly and can persist even if the original ballots are destroyed in a fire or flood. Also, there is little cost involved in maintaining the image data longer than the 22 months required for other election materials like ballots. by federal election code.
Supports Provisional Ballots Yes No Provisional ballots are important if there is any question as to the validity of the voter or any other reason, to preserve the right of the voter to cast a ballot. The number of provisional ballots can be minimized by using equipment that will print the appropriate ballot at the voting location. However, provisionals are still necessary for voters who are normally VBM voters who do not have their mail ballot to surrender.
Ranked Voting Yes No* AHCPB can support Rank-Choice voting but only at high cost and time. TOBI can resolve rank-choice voting with redundant cross-checking. AHCPB can't do RCV over large regions or with any time delays, and it is virtually impossible to provide enough oversight to be confident that RCV results are correct. In fact, TOBI is the ONLY method we are aware of that can cost-effectively support RCV and also provide sufficient transparency to allow 100% continual audits of the result by third parties. Note: Support of TOBI does NOT imply the use of RCV.
Other voting schemes Yes Mixed There are a variety of voting schemes such as Single Transferrable Vote (STV), Approval Voting, and proportional representation. AHCPB can support these if easy-to-count voting methods are but not effectively if RCV is used.
Effective Audits Easy, low cost High cost AHCPB relies on the notion that hand-counting is a reliable approach in the precinct and that with many people present, it is unlikely that they will cooperate and stuff the ballot box. To audit AHCPB, it is required that some random precincts are re-hand-counted. TOBI is based on continual independent 100% audits by third parties of the image data, and still requires the inspection of ballots to insure there is correspondence with physical ballots.
Supports Risk Limiting Audits Yes -- Election officials may want to utilize risk-limiting audits (RLA) to check their work before certification. The use of RLA does not eliminate the need for TOBI, however, because RLAs require that election officials are trusted to conduct the RLA properly. AHCPB probably would not use RLA because they are already hand-counting all ballots.
Oversight Cost Low High It is very difficult to provide oversight to thousands of teams and precincts, which is required in the AHCPB scenario. The underlying assumption in AHCPB relied upon is that counts in precincts is relatively reliable due to the presence of citizen volunteers and structured oversight is not required. We disagree with this assumption and assert that structured oversight is required for each hand-counting team. There is no way to check the result at the end.
Risk of Fraud Lower Higher Both have low risk, but since it is very difficult to confirm counts in AHCPB approach, we see that fraud could more easily occur if there is cooperation at precincts among poll workers. There is a slightly higher risk as ballots are transported from polling places to a central scanning facility in TOBI but lower risk as once the ballots are scanned, the tabulation can be more easily checked and once frozen using Block-Chain and Secure Hash Algorithm technology, it is essentially impossible to modify the election.
Needs "Open Source" No No TOBI can use Commercial Off-the-Shelf (COTS) scanners without any risk that corporations with an agenda will be able to manipulate the images. Once the image data is determined and frozen, it matters not what sort of software and equipment is used to recognize the vote and create the final Cast Vote Record, as anyone can perform their own analysis of the image data.
Positive Ballot Tracking Yes No TOBI proposes that each ballot should be endorsed either prior to scanning or as it is scanned with a unique numerical identifier, the ballot number. This means the number is actually imprinted on the ballot itself and incorporated into the image. It may be appropriate to also include a barcode or other means to easily extract the ballot number from the image when the vote is extracted, but we assume the number can be recognized using OCR. Thus, each and every ballot can be tracked through the system and retrieved if necessary to compare with the images. AHCPB has no mechanism to track every ballot and relies on chain of custody to maintain control of ballots.
Scalable Yes No AHCPB is not scalable because it expands linearly with ballot options, even if you have unlimited teams. TOBI expands with number of pages on the ballot but that can be reduced by adding machines.

More detail on processing time

Time to process an election using AHCPB expands linearly with the number of ballot options. (The following calcs do not include time to process VBM signatures or provisional validation, breaks, etc)
t(total AHCPB) ~ t(count one vote)*voters(per precinct)*options(avg per ballot) * tries(avg) / teams(per precinct).

Based on our experience observing manual tally accuracy, it is generally required that two teams count all votes and compare to detect errors. The fastest and most accurate AHCPB procedure (the read-and-tally method) processes each ballot in its entirety before going to the next. If ballots have multiple pages, they can be split up into separate teams but sorting by pages is also time consuming. When the number of options per ballot is high, the time per ballot increases and there is no way to improve that. There is never any improvement in AHCPB due to technology improvements.

TOBI process t(total TOBI) ~ voters * [ sheets(per ballot) / (num_imagescanners * ppm) + t(recognize votes in one ballot) / (imaging processing computers)].
TOBI leverages technology and time to process an election can be much lower.


Being as optimistic as possible, hand tallying races proceeds at about 2.5 to 5 seconds per ballot. We can use an estimate of 3.5 seconds. This is based on a quote from this document:

Wisconsin Election Integrity is using a projector - and projecting ballot images - and reporting cutting the hand counting time by more than half. advantage is in terms of speed - one disadvantage is that the chain of custody issues around the ballot images are a significant problem. However, we must note, this is not an AHCPB method, but a proposed audit methodology.

Page 7:
"When two teams of two auditors (four people) counted votes for two candidates in a mayoral primary, they reached agreement on the visual count for both candidates at a rate of 125 ballots every five minutes, so that a reporting unit with 1,000 ballots could be verified in 40 minutes. A hand count of the same precinct would have taken the same number of people at least two hours."
They are claiming 2.4 seconds per ballot per race, in a special case of only two candidates in one race. This is faster than the 3.5 seconds we have chosen to use below. Please note that the quote above is for an AUDIT of races already scanned. So the race need only be hand-counted one time and then compared with the machine count. If you use ONLY hand-counted ballots, then you must count AT LEAST TWICE so you can have the teams compare to make sure their result is reliable. Teams MUST work independently, so the time savings claimed here goes away.

Characteristic TOBI AHCPB Comments
Parameter Value Unit Comment
tally time per option 3.5 sec See above.
tries per ballot 2.2 tries we assume two teams do every precinct until they match, and 20% of the time they have a do-over
teams per precinct 2 team  
worker per team 4 workers  
Cost per worker / hour 19.5 $15 * 1.3 = $19.50 Only direct costs per worker, not fully burdened govt style
Parameter Value Unit Comment
image scanner rate 250 image/min Assumes double side ballot
recognition rate 1200 ballot/min Not sure of this number, highly dependent on machine and software
Cost per worker /hour 60 On per-machine basis Only direct costs per worker, not fully burdened govt style

Characteristic TOBI AHCPB Comments
Location Voters Ballot
Sides Options
Precincts Percent
scanners Image
per precinct
Total ($)
per Voter
Scanning Time
Recognition Time
Total Time (hrs)
Election Night (hrs)
Cost Total ($)
Per Voter ($)
Los Angeles 5500000 1 t2 30 4000 60% 20 15 11.03 1720.47 6881875 1.25125 36.67 5.09 41.76 16.70 50111.11 0.0091
San Diego 1500000 2 2 30 1550 60% 10 8 7.76 1210.89 1876875 1.25125 40.00 2.60 42.60 17.04 25562.50 0.0170

Actual Data

Please see the attached detailed manual tally spreadsheet for San Diego in the 2016 primary election.

It lists the number of contests and ballots. This election, a single ballot had two sheet, each double sided (four sides total). The size of the ballot results in very long times for manual tallies and takes into account the fact that they had to re-tally these precincts many times because they did not agree with the computer count. On the other hand, if they reached agreement immediately, then they could hand-tally only one time, whereas if you relied upon ONLY AHCPB, it would take AT LEAST two teams tallying the same precinct so they could check each other's result.

seq precinct contests ballots time (hrs) time (minutes) total ballot-races time per ballot-race (min)
32 110150 26 150 9.58 575 176 3.27
363 237200 25 182 18.16 1,090 207 5.26
368 240000 26 203 13.33 800 229 3.49
418 270510 27 281 37.50 2,250 308 7.31
597 376700 27 226 12.42 745 253 2.94
637 403500 12 138 6.50 390 150 2.60
670 404230 11 70 4.25 255 81 3.15
686 405400 13 148 7.67 460 161 2.86
857 420520 12 150 11.42 685 162 4.23
877 423900 11 47 2.17 130 58 2.24
991 442800 16 159 5.75 345 175 1.97
1229 487000 15 191 15.35 921 206 4.47
1332 528200 12 137 8.83 530 149 3.56
1418 538500 13 128 7.5 450 141 3.19
1431 546600 13 92 3.75 225 105 2.14
1454 549280 14 123 5.67 340 137 2.48
            AVERAGE: 3.45

We could figure the other races the same way. Based on this ACTUAL DATA, let's be optimistic and say only 1 min per race-ballot total, so no one can claim we are being too conservative.

In the primary, we had about 775K ballots processed (1.5 million in general election).
  • 1550 precincts. 775K/1550 ~ 500 ballots (total) per precinct. What you see above is only for one class of ballots.

  • 500 ballots x 25 races x 1 min = 12,500 minutes = 208 hours = 26 days (assuming 8 hour days).
  • 500 ballots x 15 races x 1 min = 7500 minutes = 125 hours = about 16 days.

Let's say you had to do it on election night, so the results were available the next day. This is not reasonable for other reasons, but let's go with it.
  • Number of races at average of 20. How many ballots?
  • 8 hours = 480 minutes = 480 race/ballots. For 20 races, you can do 24 ballots. For one race, you can do 480 ballots.

Thus, AHCPB is only viable if you have:
  • very simple ballots
  • very small precincts
  • plenty of time and lots of workers
  • no need to ever do recounts
  • Trustworthy workers.

San Diego uses the read/tally method which is the fastest and most accurate method. The reason these races took so long were because they had to do them over and over because they could not get the same result as the computer.

Comment on OPEN SOURCE

If all ballots are imaged, COTS (commercial off-the-shelf) scanning equipment can be used to create ballot images. In fact, it is safer to do so because COTS products are likely not hacked for the ballot imaging application, whereas any custom scanner would include possible hacking back-doors. This includes the current generation of ES&S DS850 type scanners that perform two steps at once, first pass being to create ballot images, and the second pass to do the recognition. We prefer that for security, ballot images are created first on COTS document scanner equipment that has no knowledge of the election processing application and do not have the capability to recognize the vote, followed by a pass over OMR (Optical Mark Recognition) software that recognizing voter intent in their marking of the ballot. The ES&S machines do both at once and outstack some ballots if they can't be imaged correctly, and they do not include these in the ballot set.

As long as all ballot images are released to the public and the imaging process includes certain parameters, then it really does not matter if any of their software is open or closed source. This issue goes away.

Open source software does not mean it is safe. Opening the source just means someone who is a programmer might be able to read and understand it and might be able to find back doors. There are very few people who can do this, and even if you have open source and an experienced team of software analysts, it does not mean you will find hidden weaknesses. Some vulnerabilities are due to bugs and defects in the code that are difficult to find through analysis and only are found in functional tests (actual operation).

Open source also allows hackers to understand the code and leverage weaknesses. Open Source is a very limited solution that is very difficult to use in any meaningful way to insure safety. I know many groups champion this but just ask around and see how many people have actually looked and analyzed any open source code for weaknesses. Pretty much zero, and again, just having someone review it does not mean they will understand it enough to catch what it is doing or any hidden bugs and vulnerabilities.

The only truly safe way to deal with this problem is with OPEN BALLOT IMAGES which are released to the public.


TOBI has significant benefits over AHCPB and very few drawbacks. We believe that the only true benefit to AHCPB is that it is a simpler concept. However, we believe that the public will understand the underlying concept of capturing ballot image data and making it available for competing groups to review. OPEN SOURCE is a false solution that does not guarantee anything and may actually make it easier to hack. Thus, open source is not a target of TOBI.

Media Form edit

Title The Open Ballot Initiative vs. Hand Counted Paper Ballots
Publisher Citizens Oversight
Author Ray Lutz
Pub Date 2016-12-25
Media Link
Remote Link
Embed HTML
Forum Link
Keywords Election Integrity
Media Type Article
Media Group Blog Entry
Author Name Sortable
Publish Status Published
Thumbnail Link
I Attachment Action Size Date Who Comment
1433i.pdfpdf 1433i.pdf manage 299 K 05 Jan 2017 - 16:08 Raymond Lutz Sample of a PDF containing two sides of a ballot in an image file.
1_Manual_Tally_Spreadsheet_Report_June_2016-_San_Diego_County_Final.pdfpdf 1_Manual_Tally_Spreadsheet_Report_June_2016-_San_Diego_County_Final.pdf manage 575 K 04 Jan 2017 - 23:28 Raymond Lutz San Diego Manual Tally Report spreadsheet
DevelopmentReport-DigitalImageAudits-WI-audit-proposal.pdfpdf DevelopmentReport-DigitalImageAudits-WI-audit-proposal.pdf manage 987 K 05 Jan 2017 - 22:48 Raymond Lutz Proposal for digital image approach for audits in WI
Topic revision: r14 - 18 Jan 2017, RaymondLutz
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Cops? Send feedback