Cops Canvass Report Manual Tally Procedure

This is a section of the Cops Canvass Report. See also San Diego County Canvass Procedure

Manual Tally Procedure

The Ro V maintains the One Percent Manual Tally Procedure to govern this process, which is mandated by the So S. This is also refered to as the Post-Election Manual Tally (PEMT).

Manual tally procedure in a nutshell

1. Select dates for the random draw of precincts and the actual tally, notify the public by placing a notice at the front counter. (COPs: This should also be placed on the website.)
2. Random Selection of Precincts
   1. Calculate one percent and round up.
   2. Have scheduled observers attend the draw
   3. The procedure uses three sets of balls from 0-9 and one set of balls numbered 0 and 1, representing the four digits, (ones, tens, hundreds, and thousands, respectively).
   4. Shake the containers and draw balls manually. (COPs: Since the containers are clear plastic, this may result in trying to choose specific balls.)
   5. Draw ones, tens, hundreds digits.
   6. If the three numbers in step six is 650 or less, the thousands container is then shaken, otherwise if the number is 651 or greater, the selection is complete.
   • Probability of selecting any number 650 or less or from 1000 to 1650 = (0.1)*(0.1)*(0.1)*(.65)*(0.5) = 0.0325%
   • Probability of selecting any number 651 to 999 = (0.1)*(0.1)*(0.1)*(0.35) = 0.0350%
   • The only time this would be correct is when "Cut Off" (i.e. 650 in this case) is 666 so that 666/2 = 1000-666.
     • Probability of selecting any number 666 or less or from 1000 to 1666 = (0.1)*(0.1)*(0.1)*(.666)*(0.5) = 0.0333%
     • Probability of selecting any number 667 to 999 = (0.1)*(0.1)*(0.1)*(0.333) = 0.0333%
   • Therefore, this procedure does not produce exact results and is not generally applicable, however, results are approximately correct because 650 is almost the same as 666.
   • Selecting precincts randomly will not necessarily result in even sampling of the precincts. In the November 2008 election, no precincts were selected in the range from 1 to 453 inclusive while nine precincts were chosen in the highest numbered 450 precincts. The likelihood of that happening is about 1 chance out of 10,000 tries, which is rare indeed.
3. Record contests as selected by the random draw.
4. Select additional precincts for the manual tally to cover all remaining contests on the ballot.
5. Prepare for the manual tally, including:
   • Order summary reports for election night and the related ballots. (COPs note: the staff creating these reports will know which ones are being requested, and therefore, could conceivably prepare known good reports for those precincts while the balance of the election may be substantially modified.)
6. Conduct the manual tally
   • Each precinct as one three-person team, one calls out the vote and two tally on separate sheets.
   • Supervisor compares with summary report.
   • If there is a variance, they re-tally until they agree, but no more than three times.
   • Supervisor records results, variances, and causes (if knoswn) on consolidation log.
7. When the tally is complete, return ballots for secure archival
   • Compile a memo of the results for the Ro V.

Recommendations to Improve the Post-Election Manual Tally

Issue A045 - PEMT Sample Size Insufficient

Statistics of elections should be used to determine the appropriate sample size. ElectionsMathematics.org provides an excellent analysis to determine the sample size necessary to detect corrupt precincts sufficient to change the outcome of the election. See Kathy Dopp, 'Checking election outcome accuracy Post-election audit sampling' for a thorough analysis of the statistics. COPS recommends that a 5% sample be used, because the assumption that all the precincts are the same size is not true, and this provides an additional level of confidence.

See Five Percent Sample for a full discussion of this issue and calculations related to this election.

• Issue A045: Proposal to Ro V
  

COPs recommends that the Ro V use a 5% sample in the PEMT procedure in order to detect corrupt vote counts sufficient to change the outcome of the election, and to account for the fact that not all the precincts are the same size, which is an assumption of the analysis by Kathy Dopp.

• Issue A045: Legislative
  COPs recommends changes in the law to require that at least a 5% sample be used in the PEMT. The 1% sample is too small to detect changes sufficient to change the election, with a confidence level sufficient for a reliable audit.

Issue A033 - PEMT Performed Prematurely

The PEMT is performed too early after election night and does not include a complete set of ballots for the precincts under inspection. The PEMT should be performed after all write-ins, VBM and provisional ballots have been processed. The write-ins that were removed on election night certainly should be completed before starting.

• Issue A033: Question to Ro V
  In upcoming elections, include all ballots in the PEMT manual tally.

• Issue A033: Legislative
  The Law should be refined to define exactly when the PEMT should be performed and what should be included in it. The law should define:
  • that the PEMT should include all ballots completed at polling places on election night, including all write-ins and "damaged" ballots, and it also should include all VBM ballots received by the deadline for those precincts.
  • that the PEMT should also review Ballot Statements and determine irregularities.
  • that it is not allowed to rescan offending precincts and then inappropriately deem the precincts as error free.
  • That comparison with a computer report should use the final canvass report, not an interim report. (See Issue A036)

Issue A034 - PEMT - Notification on Web Site

The Ro V should notify the public on the Ro V website regarding the selection and tally in addition to placing a notice at the front counter. The web site is much more accessible than just the front counter and will incur no significant additional cost for the Ro V but will make coordination by the public much easier.

• Issue A034: Proposal to Ro V
  Provide notification of selection and tally of PEMT precincts on the web site as well as the front counter.

• Issue A034: Legislative
  Include minor revisions to elections code to require that all electing districts provide notices on their web site as well as notices on the front counter of their office.

Issue A035 - PEMT Random Selection Not Technically Correct

The random selection of precincts procedure is incorrect and not generally applicable. COPs recommends that the Registrar Of Voters change this procedure as follows:

• Use four ten-sided die (which are easily available on-line for $3.99 for a set of 10 die).
• Roll the dice perhaps in separate boxes or use different colors to allow them to be treated as ones, tens, hundreds and thousands.
• Declare that on the die representing thousands, even numbers will be considered "0" and odd numbers represent "1".
• If the total number shown on the die is greater than the highest numbered precinct sequence number (1697 in this case), disregard the roll and roll again.
• If the precinct is from a Collection Center that has already been sampled, disregard the roll and roll again (See discussion below).

We noted that the distribution of sampled precincts was not uniform. There were no precincts selected in the first 453 precincts, but nine selected in the last 453 precincts. This can happen in a random selection process, but the likelihood that no precincts would be selected in the first 453 precincts is only one in 10,000 -- very unusual.

It would be preferrable to sample the precincts based on the structure of the collection process. Since there are 78 Collection Centers, each of the sampled precincts should come from a different collection center. To support this constraint, the last step was added to the procedure above to choose again if the collection center had already been sampled. In the November 2008 PEMT, there were two collection centers that were sampled twice.

• Issue A035: Proposal to Ro V
  Change the method of random selection as described above.

• Issue A035: Legislative
  The elections code should specify exactly how to randomly select the precincts because the statistics of random selection is frequently not well understood by elections workers. In this case, the San Diego Ro V used a methodology that was technically incorrect and not generally applicable.

Issue A036 - PEMT Computer Report Manipulation Vulnerability

There is a manipulation vulnerability regarding how the computer reports are generated. Current practice has the Ro V staff produce reports which are, because the PEMT is currently performed early in the canvass cycle, tentative at best. The reports used by the PEMT should be the (draft) final canvass report to eliminate the possibility that a compromised member of Ro V staff might substitute a known correct report for the precincts to be sampled while allowing the other precincts to be modified.

If the election was fraudulently manipulated by Ro V staff using the central tabulator, then it would be possible to cover that up by producing "good" reports for the precincts in the PEMT while allowing many other precincts to remain compromised.

See also Issue Issue A033: PEMT Peformed too early.

• Issue A036: Question to Ro V
  To avoid this vulnerability, the final canvass report should be used, and the actions of the IT staff observed by outsiders during the generation of these reports.

• Issue A036: Legislative
  Elections code should specify that PEMT procedure should use the (draft) final canvass report and not interim reports. The PEMT precincts should be unknown until after the (draft) final report is produced to eliminate the possibility that the report could be generated differently for PEMT precincts than the untested precincts.

Issue A037 - Manual Tally Done Backwards

Current practice treats the output of the computer scanners as the known good result and the tally teams redo their work until they can match it. This is contrary to the principle that the tally teams should be the most reliable result. Therefore, the PEMT procedure should be modified as follows:

1. Tally each precinct by two tally teams, each operating as they are now.
2. Compare the result of the two teams.
3. If they differ, tally again by a third team
4. If possible, determine what causes them to differ and determine the "correct" result.
5. Compare this result with the report as generated by the electronic scanners.
6. If the scanners produce a different result, rescan the ballots to determine if that is the source of the error.
7. If an error is detected, it may require that all ballots be rescanned using different equipment.

• Issue A037: Question to Ro V
  Please modify the PEMT procedure as described above.

• Issue A037: Legislative
  Improve elections code law to clearly define how the PEMT tallying should be done with the assumption that the tally teams should produce an independent result, and not rely on the computer report.

Issue A038 - PEMT Rescanning

There is no provision in the current Post Election Manual Tally procedure to allow that workers rescan precinct ballots if the vote determined by the manual tally differs from the electronic scan. However, the Ro V rescans the ballots. They confirm the error (i.e. additional ballots are discovered in the re-scan) but the Ro V nevertheless declares that the precinct is error free. The rescanning procedure to confirm an error must be included in the written procedure if it is to be performed, and then the consequences of finding the error must also be described, such as performing a complete election rescan.

• Issue A038: Proposal to Ro V
  Your method of handling discrepancies in the PEMT is not documented, and your current practice is absolutely incorrect. If there are offending precincts, a full explanation should be included, and if the precinct can be corrected by rescanning, that would imply that the entire canvass may be faulty, and may need to be completely rescanned after corrections are made to scanner operating procedure.

• Issue A038: Legislative
  Elections code law should be enhanced to clearly specify that the Ro V is not allowed to simply rescan the offending precincts in the PEMT and fix any problems and then declare everything is resolved. The conclusion is quite the opposite. If there are offending precincts, a full explanation should be included, and if the precinct can be corrected by rescanning, that would imply that the entire canvass may be faulty, and may need to be completely rescanned after corrections are made to scanner operating procedure.

Issue A039 - Tally Sheets too big to process automatically

The Post-Election Manual Tally (PEMT) should use paper that is easily processed by automatic document feeding equipment. Currently, the tally sheets are on over-sized paper and are difficult to scan.

See Issue Issue A023: Documents Automatically Processable.

• Issue A039: Question to Ro V
  Please use Tally Sheets that conform to standard sheet sizes so they are easy to automatically scan, and avoid staples, tabs, and other additions that will make scanning difficult.

• Issue A039: Legislative
  See Issue A023: Documents Automatically Processable.

Issue A040 - PEMT Reports Incomplete

The PEMT should produce a report for every race and every precinct, as suggested by the So S. These sheets are easy to prepare if they are prepared when the manual tally is conducted. The resulting number of pages should be at least (Number of Precincts)*(Number of Races). For the November 2008 election, this would have produced about 400 pages for VBM and 400 pages for election day results. (The number of races applicable to any Precinct is less than the total number of races run in the county as many are not county-wide.) The report prepared by the Ro V had only five pages, not 800, and did not include any vote counts. (By comparison, Los Angeles County produced 1,300 pages in their report.)

• Issue A040: Question to Ro V
  The So S suggests a format for the PEMT report but the Ro V ignores those requirements. We have requested compliance in past elections and have yet to see the Ro V comply with the recommended format. We require compliance to help us with our oversight task.

• Issue A040: Legislative
  The So S suggests a format for the PEMT report but the Ro V ignores those requirements. This should be specified in law with penalties to provide a means to force compliance.

Issue A041 - PEMT Should Reconcile to Rosters

The PEMT does not look at the ballot statements and sign-in rosters, and are restricted to the ballots actually run through the scanners on election night. Unfortunately, since there are many (9,000) write-ins removed, the PEMT does not match the number of people who signed the rosters at the polling place. The PEMT should include a complete look at the precincts under consideration and not limit the view to just the operation of the scanners. See also Issue A033: PEMT Peformed too early.

• Issue A041: Question to Ro V
  Please modify the procedure used in the PEMT to include review of sign-in rosters and BallotStatements, and include ALL ballots to match sign-in rosters and ballot statements.

• Issue A041: Legislative
  Law should specify that PEMT should reconcile ballot counts to original sign-in rosters, not just to the computer reports. See Issue A033: PEMT Peformed too early.

Issue A042 - PEMT Errors Not Addressed

In the November 2008 Election, a number of errors were detected by the Post Election Manual Tally (PEMT) procedure. These issues were not addressed and the Ro V simply ignored them. The Ro V said the errors were resolved with the reconciliation procedure (which is not documented and does not have any report) but there is no confidence that this procedure actually resolves these issues.

• Of the original 17 precincts sampled, 5 had errors, (29% error rate).
• The additional 33 precincts combined (50 total) had 10 failing precincts, (20% error rate).
• 5 ballots were lost in the 1% sample. By extension, 1134 ballots were lost (about 8 per each scanner on the floor).
• Apparently, no research into whether the vote was different (Vote Substitution error) unless ballots were missing or total vote count different.
• This issue is in addition to the 9,000 "missing" ballots due to removal of "damaged" and write-in ballots.

• Issue A042: Question to Ro V
  You have refused in the past to answer our question as to why 29% of the precincts in the 17-precinct sample had errors of some kind, with 5 ballots lost. This loss exceeds the two-ballot loss allowed for all precincts.

• Issue A042: Legislative
  The Ro V simply ignores errors revealed by the PEMT by rescanning those precincts, these failures (29% of the precincts tested) actually proves that there are serious problems with the processing performed by the Registrar of Voters in San Diego County. Election law must be improved to clarify the responsibilities of the Ro V with respect to errors resolved by the PEMT procedure.

Next Section: Audit Trail