The Open Ballot Initiative vs. Hand Counted Paper Ballots

Citizens Oversight (2016-12-25) Ray Lutz

This Page: https://copswiki.org/Common/TheOpenBallotInitiativeVsHandCountedPaperBallots
More Info: Election Integrity

Here, we compare The Open Ballot Initiative (TOBI) with a commonly promoted approach, All Hand Counted Paper Ballots (AHCPB) in the precinct.

In a nutshell:

• Both approaches use durable paper ballots.
• The Open Ballot Initiative:
  • uses document image scanners to create a picture (image) of both sides of each ballot sheet
  • Images are saved likely as a PDF file.
  • Preferably, the ballots are imprinted with a unique sequence number prior to scanning so the number appears in the image, can be found on the ballot, and corresponds with the filename of the image file.
  • The image files are secured against tampering using cryptographic means (SHA signatures & block-chain signature).
  • The resulting images must still be checked for correspondence with the original ballots using a sampling method preferably by a third party.
  • Since ballots can be secured away, ballot images can be made available to the public on a public server.
  • The public can count these "by hand" (by viewing on the screen) if they want to, or they can use software to assist in the process.
  • Includes structured mechanism for continual 100% audits by third parties of the tabulation.
  • This eliminates central tabulator vulnerabilities, reduces oversight costs, and is palatable to election officials who are cost-conscious.
• All Hand-Counted Paper Ballots (AHCPB)
  • Requires that all ballots are transported to precincts and workers count these by hand, possibly using visual aids (projector, for example).
  • Fastest and most accurate method is "read and tally" method but "sort and stack" is an alternative method that is slower and less accurate.

An example of a PDF file which is an actual image scan from a ES&S DS 200 scanner is attached to this topic. The election industry is moving in this direction as computer recognition of voter intent is much better when a high-resolution image is used rather than simplistic reflected-light detectors as was used in the old scan-tron-like ballot readers.

• 1433i.pdf: Sample of a PDF containing two sides of a ballot in an image file.

Characteristic	TOBI	AHCPB	Comments
Uses durable paper ballots	Yes	Yes	Both very similar in this regard. Durable paper ballots can be completed by hand or using touch-screen assistant equipment which is coupled with a ballot printer, and prints a full ballot identical to one completed by hand except that the bubbles are filled in without any variation. In the precinct, another approach is to have voters fill in the ballots by hand but run them through a quality-control scanner which checks for over and under votes. Vote-by-Mail (VBM) is extremely popular in CA and rural states, and is the only method used in some states like Oregon. Durable paper ballots are compatible with VBM.
Complexity	Higher	Lower	AHCPB is easy to understand. This is a key benefit to the AHCBP approach over TOBI. However, we believe the concept of capturing all ballot images is still easy to understand.
Supports Early VBM processing	Yes	Yes*	AHCPB requires that VBM ballots that are processed and validated are brought back to the precinct for hand-counting.
Supports Later VBM processing	Yes	No	In CA for example, VBM ballots can be accepted with a valid postmark up to three-days after election day. Also, if no signature is provided, then the voter is notified, and has up to eight days to provide a signature. These ballots, as well as any which are not fully processed in terms of signature validation prior to election day, are not processable by AHCPB in the precinct.
Supports All-Mail precincts	Yes	No	Some states, like Oregon, use all-mail ballot election processing. AHCPB is not compatible with these states. Also, any precincts which are all-mail are not hand-countable in the precinct and would require teams to be set up by elections officials.
Cost	Low	High	AHCPB requires manual labor which is feasible only in small districts and with simple ballots (with limited number of ballot options)
Error Rate	~0%	1 to 2%	Although Hand-counted paper ballots is viewed as the "gold standard," in practice counting votes on paper ballots, the error rate is 1% to 2% according to recent scientific studies. OBI has the potential to eliminate nearly all errors due to the redundant recognition phase and comparison between differing groups, and then joint resolution of each ballot where any difference is detected between competing groups. Citizens Oversight conducted a study in San Diego sampling 5% of the precincts (85) in 2008. In the sample, only 15% of the precincts were able to balance the number of ballots issues, voted, spoiled, blank and the number of signatures matched the number of voted ballots and the number of scanned ballots. We have very little confidence that temporary workers in precincts can count the vote on complex, multi-page ballots.
Recount Cost	Very Low	High	This is where TOBI really shines. As seen in recent multi-state recount, it is very expensive ($millions) to recount an entire state. MI charged $125 per precinct to recount. In TOBI, the cost to recount is minimal, and recounts would likely never need to be performed because the process INCLUDES redundant tabulations, i.e continual recounts.
Precinct Chain of Custody Requirements	Similar	Similar	The number of Issued Ballots should equal the number of blank, voted, spoiled, and completed ballots. Also, the number of ballots included in the vote tabulation should equal the number of voted ballots and the number of voters who signed-in.
Post-election day chain of Custody	Low	High	To allow for recounting, AHCPB requires that the ballots remain in secure custody as they are the only record of the election. In TOBI, once the ballots have been resolved to images, the actual physical ballots are only required for the audit process. The image data can be stored redundantly and can persist even if the original ballots are destroyed in a fire or flood. Also, there is little cost involved in maintaining the image data longer than the 22 months required for other election materials like ballots. by federal election code.
Supports Provisional Ballots	Yes	No	Provisional ballots are important if there is any question as to the validity of the voter or any other reason, to preserve the right of the voter to cast a ballot. The number of provisional ballots can be minimized by using equipment that will print the appropriate ballot at the voting location. However, provisionals are still necessary for voters who are normally VBM voters who do not have their mail ballot to surrender.
Ranked Voting	Yes	No*	AHCPB can support Rank-Choice voting but only at high cost and time. TOBI can resolve rank-choice voting with redundant cross-checking. AHCPB can't do RCV over large regions or with any time delays, and it is virtually impossible to provide enough oversight to be confident that RCV results are correct. In fact, TOBI is the ONLY method we are aware of that can cost-effectively support RCV and also provide sufficient transparency to allow 100% continual audits of the result by third parties. Note: Support of TOBI does NOT imply the use of RCV.
Other voting schemes	Yes	Mixed	There are a variety of voting schemes such as Single Transferrable Vote (STV), Approval Voting, and proportional representation. AHCPB can support these if easy-to-count voting methods are but not effectively if RCV is used.
Effective Audits	Easy, low cost	High cost	AHCPB relies on the notion that hand-counting is a reliable approach in the precinct and that with many people present, it is unlikely that they will cooperate and stuff the ballot box. To audit AHCPB, it is required that some random precincts are re-hand-counted. TOBI is based on continual independent 100% audits by third parties of the image data, and still requires the inspection of ballots to insure there is correspondence with physical ballots.
Supports Risk Limiting Audits	Yes	--	Election officials may want to utilize risk-limiting audits (RLA) to check their work before certification. The use of RLA does not eliminate the need for TOBI, however, because RLAs require that election officials are trusted to conduct the RLA properly. AHCPB probably would not use RLA because they are already hand-counting all ballots.
Oversight Cost	Low	High	It is very difficult to provide oversight to thousands of teams and precincts, which is required in the AHCPB scenario. The underlying assumption in AHCPB relied upon is that counts in precincts is relatively reliable due to the presence of citizen volunteers and structured oversight is not required. We disagree with this assumption and assert that structured oversight is required for each hand-counting team. There is no way to check the result at the end.
Risk of Fraud	Lower	Higher	Both have low risk, but since it is very difficult to confirm counts in AHCPB approach, we see that fraud could more easily occur if there is cooperation at precincts among poll workers. There is a slightly higher risk as ballots are transported from polling places to a central scanning facility in TOBI but lower risk as once the ballots are scanned, the tabulation can be more easily checked and once frozen using Block-Chain and Secure Hash Algorithm technology, it is essentially impossible to modify the election.
Needs "Open Source"	No	No	TOBI can use Commercial Off-the-Shelf (COTS) scanners without any risk that corporations with an agenda will be able to manipulate the images. Once the image data is determined and frozen, it matters not what sort of software and equipment is used to recognize the vote and create the final Cast Vote Record, as anyone can perform their own analysis of the image data.
Positive Ballot Tracking	Yes	No	TOBI proposes that each ballot should be endorsed either prior to scanning or as it is scanned with a unique numerical identifier, the ballot number. This means the number is actually imprinted on the ballot itself and incorporated into the image. It may be appropriate to also include a barcode or other means to easily extract the ballot number from the image when the vote is extracted, but we assume the number can be recognized using OCR. Thus, each and every ballot can be tracked through the system and retrieved if necessary to compare with the images. AHCPB has no mechanism to track every ballot and relies on chain of custody to maintain control of ballots.
Scalable	Yes	No	AHCPB is not scalable because it expands linearly with ballot options, even if you have unlimited teams. TOBI expands with number of pages on the ballot but that can be reduced by adding machines.

More detail on processing time

Time to process an election using AHCPB expands linearly with the number of ballot options. (The following calcs do not include time to process VBM signatures or provisional validation, breaks, etc)

t(total AHCPB) ~ t(count one vote)*voters(per precinct)*options(avg per ballot) * tries(avg) / teams(per precinct).

Based on our experience observing manual tally accuracy, it is generally required that two teams count all votes and compare to detect errors. The fastest and most accurate AHCPB procedure (the read-and-tally method) processes each ballot in its entirety before going to the next. If ballots have multiple pages, they can be split up into separate teams but sorting by pages is also time consuming. When the number of options per ballot is high, the time per ballot increases and there is no way to improve that. There is never any improvement in AHCPB due to technology improvements.

TOBI process t(total TOBI) ~ voters * [ sheets(per ballot) / (num_imagescanners * ppm) + t(recognize votes in one ballot) / (imaging processing computers)].

TOBI leverages technology and time to process an election can be much lower.

Examples

Being as optimistic as possible, hand tallying races proceeds at about 2.5 to 5 seconds per ballot. We can use an estimate of 3.5 seconds. This is based on a quote from this document:

• DevelopmentReport-DigitalImageAudits-WI-audit-proposal.pdf: Proposal for digital image approach for audits in WI

Wisconsin Election Integrity is using a projector - and projecting ballot images - and reporting cutting the hand counting time by more than half. advantage is in terms of speed - one disadvantage is that the chain of custody issues around the ballot images are a significant problem. However, we must note, this is not an AHCPB method, but a proposed audit methodology.

Page 7:
"When two teams of two auditors (four people) counted votes for two candidates in a mayoral primary, they reached agreement on the visual count for both candidates at a rate of 125 ballots every five minutes, so that a reporting unit with 1,000 ballots could be verified in 40 minutes. A hand count of the same precinct would have taken the same number of people at least two hours."

They are claiming 2.4 seconds per ballot per race, in a special case of only two candidates in one race. This is faster than the 3.5 seconds we have chosen to use below. Please note that the quote above is for an AUDIT of races already scanned. So the race need only be hand-counted one time and then compared with the machine count. If you use ONLY hand-counted ballots, then you must count AT LEAST TWICE so you can have the teams compare to make sure their result is reliable. Teams MUST work independently, so the time savings claimed here goes away.

Constants:

Characteristic	TOBI	AHCPB	Comments
AHCPB
Parameter	Value	Unit	Comment
tally time per option	3.5	sec	See above.
tries per ballot	2.2	tries	we assume two teams do every precinct until they match, and 20% of the time they have a do-over
teams per precinct	2	team
worker per team	4	workers
Cost per worker / hour	19.5	$15 * 1.3 = $19.50	Only direct costs per worker, not fully burdened govt style
TOBI
Parameter	Value	Unit	Comment
image scanner rate	250	image/min	Assumes double side ballot
recognition rate	1200	ballot/min	Not sure of this number, highly dependent on machine and software
Cost per worker /hour	60	On per-machine basis	Only direct costs per worker, not fully burdened govt style

Characteristic	TOBI	AHCPB	Comments
Location	Voters	Ballot Sheets	Sides	Options Per Ballot	Precincts	Percent VBM	scanners	Image Recognition Machines	AHCPB Time (hours)	AHCPB Cost per precinct	AHCPBCost Total ($)	AHCPB Cost per Voter	TOBI Scanning Time (hours)	TOBI Recognition Time (hours)	TOBI Total Time (hrs)	TOBI Time Election Night (hrs)	TOBI Cost Total ($)	TOBI Cost Per Voter ($)
Los Angeles	5500000	1	t2	30	4000	60%	20	15	11.03	1720.47	6881875	1.25125	36.67	5.09	41.76	16.70	50111.11	0.0091
San Diego	1500000	2	2	30	1550	60%	10	8	7.76	1210.89	1876875	1.25125	40.00	2.60	42.60	17.04	25562.50	0.0170

Actual Data

Please see the attached detailed manual tally spreadsheet for San Diego in the 2016 primary election.

• 1_Manual_Tally_Spreadsheet_Report_June_2016-_San_Diego_County_Final.pdf: San Diego Manual Tally Report spreadsheet

It lists the number of contests and ballots. This election, a single ballot had two sheet, each double sided (four sides total). The size of the ballot results in very long times for manual tallies and takes into account the fact that they had to re-tally these precincts many times because they did not agree with the computer count. On the other hand, if they reached agreement immediately, then they could hand-tally only one time, whereas if you relied upon ONLY AHCPB, it would take AT LEAST two teams tallying the same precinct so they could check each other's result.

seq	precinct	contests	ballots	time (hrs)	time (minutes)	total ballot-races	time per ballot-race (min)
418	270510	27	281	37.50	2,250	308	7.31
597	376700	27	226	12.42	745	253	2.94
368	240000	26	203	13.33	800	229	3.49
1229	487000	15	191	15.35	921	206	4.47
363	237200	25	182	18.16	1,090	207	5.26
991	442800	16	159	5.75	345	175	1.97
32	110150	26	150	9.58	575	176	3.27
857	420520	12	150	11.42	685	162	4.23
686	405400	13	148	7.67	460	161	2.86
637	403500	12	138	6.50	390	150	2.60
1332	528200	12	137	8.83	530	149	3.56
1418	538500	13	128	7.5	450	141	3.19
1454	549280	14	123	5.67	340	137	2.48
1431	546600	13	92	3.75	225	105	2.14
670	404230	11	70	4.25	255	81	3.15
877	423900	11	47	2.17	130	58	2.24
						AVERAGE:	3.45

We could figure the other races the same way. Based on this ACTUAL DATA, let's be optimistic and say only 1 min per race-ballot total, so no one can claim we are being too conservative.

In the primary, we had about 775K ballots processed (1.5 million in general election).

• 1550 precincts. 775K/1550 ~ 500 ballots (total) per precinct. What you see above is only for one class of ballots.

• 500 ballots x 25 races x 1 min = 12,500 minutes = 208 hours = 26 days (assuming 8 hour days).
• 500 ballots x 15 races x 1 min = 7500 minutes = 125 hours = about 16 days.

Let's say you had to do it on election night, so the results were available the next day. This is not reasonable for other reasons, but let's go with it.

• Number of races at average of 20. How many ballots?
• 8 hours = 480 minutes = 480 race/ballots. For 20 races, you can do 24 ballots. For one race, you can do 480 ballots.

Thus, AHCPB is only viable if you have:

• very simple ballots
• very small precincts
• plenty of time and lots of workers
• no need to ever do recounts
• Trustworthy workers.

San Diego uses the read/tally method which is the fastest and most accurate method. The reason these races took so long were because they had to do them over and over because they could not get the same result as the computer.

Comment on OPEN SOURCE

If all ballots are imaged, COTS (commercial off-the-shelf) scanning equipment can be used to create ballot images. In fact, it is safer to do so because COTS products are likely not hacked for the ballot imaging application, whereas any custom scanner would include possible hacking back-doors. This includes the current generation of ES&S DS850 type scanners that perform two steps at once, first pass being to create ballot images, and the second pass to do the recognition. We prefer that for security, ballot images are created first on COTS document scanner equipment that has no knowledge of the election processing application and do not have the capability to recognize the vote, followed by a pass over OMR (Optical Mark Recognition) software that recognizing voter intent in their marking of the ballot. The ES&S machines do both at once and outstack some ballots if they can't be imaged correctly, and they do not include these in the ballot set.

As long as all ballot images are released to the public and the imaging process includes certain parameters, then it really does not matter if any of their software is open or closed source. This issue goes away.

Open source software does not mean it is safe. Opening the source just means someone who is a programmer might be able to read and understand it and might be able to find back doors. There are very few people who can do this, and even if you have open source and an experienced team of software analysts, it does not mean you will find hidden weaknesses. Some vulnerabilities are due to bugs and defects in the code that are difficult to find through analysis and only are found in functional tests (actual operation).

Open source also allows hackers to understand the code and leverage weaknesses. Open Source is a very limited solution that is very difficult to use in any meaningful way to insure safety. I know many groups champion this but just ask around and see how many people have actually looked and analyzed any open source code for weaknesses. Pretty much zero, and again, just having someone review it does not mean they will understand it enough to catch what it is doing or any hidden bugs and vulnerabilities.

The only truly safe way to deal with this problem is with OPEN BALLOT IMAGES which are released to the public.

Conclusion

TOBI has significant benefits over AHCPB and very few drawbacks. We believe that the only true benefit to AHCPB is that it is a simpler concept. However, we believe that the public will understand the underlying concept of capturing ballot image data and making it available for competing groups to review. OPEN SOURCE is a false solution that does not guarantee anything and may actually make it easier to hack. Thus, open source is not a target of TOBI.