BOARD OF ADVISORS & STANDARDS BOARD
VVSG TRAINING WORKSHOP
David Flater Part 1, OCTOBER 15, 2007

VVSG Tutorial Narration
Core Requirements Part 1*

[Slide 1]

[NARRATOR:] This is Part One of the next Voluntary Voting System Guidelines Core Requirements [CRT] training modules. Parts One through Four are presented by Dr. David Flater of the National Institute of Standards and Technology's Information Technology Laboratory. These modules present an overview of the material in Part 3 of the Guideline recommendations including Questions and Answers from the members of the Election Assistance Commission's Board of Advisors and Standards Board.

[MR. FLATER:] Okay, good morning, everyone. I'm David Flater. I work with the Core Requirements and Testing Subcommittee.

[Slide 2]

The organization of this presentation begins with prerequisites. As it happens, much of the material is interrelated. In order to get moving, at some point you have to introduce a whole lot of concepts all at the same time, and so towards the beginning, I'll be zigzagging a bit trying to cover all this background material.
Once we get through that, it should become easier. I'll be able to walk through the chapters in order with a few exceptions, and then we can discuss the details of the material as we go through the document.

My goals are to tell you what this document says along with some detail on how exactly it differs from the previous guidelines and what the anticipated impact of those changes is.

[Slide 3]

So first regarding how do we end up with this designation of certain chapters as core chapters and other chapters not? Core requirements are simply those that are neither Security and Transparency requirements nor Human Factors and Privacy requirements. Everything else is considered a core requirement.

This most notably includes your high-level general functional requirements on voting systems that you would expect, such as all voting systems shall enable you to count votes. It also includes things such as reliability, and accuracy, and workmanship.

[Slide 4]

Now as we looked at revising the core requirements that were carried over from VVSG 2005, there were certain global changes that had to be applied throughout the document, the reorganization to reduce redundancy, identifying requirements, which used to simply be mentioned in passing, if you will, in a paragraph of discussion text. We've now pulled them out, given them numbers so that the test lab can say, okay, this is the requirement that you didn't fulfill, and we also tried to clarify language and use terms consistently to reduce confusion.
When we did make substantive changes to requirements, we've tried to follow the, if it isn't broken, don't fix it rule. Not only that but even if it was broken, if we could not think of a solution that everyone thought was better, we didn't fix the problem.

[Slide 5]

Now there have been many clarifications made throughout the document. Many of these came up because previous generations of reviewers would latch on to some text. Sometimes it was old text that's been there for years; sometimes it was text that we wrote and became very confused about it. And when a lot of discussion was required to explain those requirements, we tried to incorporate that discussion into the document as informative text so that other people with the same confusion might benefit.
There were also assorted mandates to be implemented as part of the revisions. One of them was to remove guidance for punch card technology, and there are also two sections of the document, one the process model and the other the public information package, which are responsive to TGDC resolutions.

[Slide 6]

The big changes made by CRT can be summarized as follows: firstly, there are now definitions for voting variations. These were optional features of voting systems that in previous guidelines, there was simply language saying the vendor shall discuss which of the following features are implemented and if they are implemented, how they are implemented.
There are now definitions that give a baseline for what it means to support these features. There are changes to both the benchmarks and the test methods for reliability and accuracy.
There was a major revision to the software workmanship section, previously called coding conventions or coding standards, so that there are fewer prescribed programming roles and more of a focus on the 20 percent of the rules that made 80 percent of the difference, which are the rules having to do with software integrity.

I believe the electromagnetic compatibility quality assurance and configuration management sections have been completely rewritten. My colleague Alan Goldfine will be discussing those.
There were also significant changes having to do with test methods and related procedures used by the test labs. One is logic verification, which I'll be talking about later; one is the addition of a volume test, which I'll be talking about later; and also the tightening of certain ground rules for how the conformity assessment is conducted.

[Slide 7]

Other changes that did not have a huge impact on the document but which are important from the perspective of voting systems in general, the reporting requirements have been clarified, pulled together in one place and harmonized with each other.

The meaning of accuracy for optical scanners has been clarified: essentially you had marks and non-marks and in the real world, there are more categories than just those, and I'll be going into more detail about that.

Early voting is now definitely compatible with the guidelines. The phenomenon of early voting came about after, I think, the previous editions of the guidelines were issued, and certainly interpretations were made to make it consistent, and it's now been established that in the next iteration of guidelines, it is in fact consistent, and what we're talking about are states that, in a formal process, allow voters to come into specific locations before the official election day and cast their votes.

The definitions having to do with software, commercial off-the-shelf software, have been clarified with regards to many different issues that related to that.
And finally, there has been a test added in part three- an operating test for humidity to respond to concerns about paper ballots not scanning properly under humid conditions.

[Slide 8]

There are also some changes made by the Security and Transparency subcommittee to some sections that began as CRT sections, but it turned out that STS had the lion's share of the requirements there. John Wack is going to talk about integratability and data exporter interchange and also the ballot activation section.

[Slide 9]

So before I start in on actually going through the material chapter by chapter, we need to get some background material out of the way, which includes understanding the terms that are used and also addressing the many questions that there will be about voting system and voting place classes.
First and foremost is the distinction between the voting system and voting devices. This was not called out specifically in previous versions of the guidelines, and the result was that in some places, it wasn't clear which we were talking about.

[Slide 10]

The voting system is the whole shebang that you buy from a manufacturer. The voting system is composed of voting devices and some other things, documentations, supply, etc.

[Slide 11]

The impact of this in requirements is that sometimes we have requirements saying voting systems shall do something. What this means is that the voting system as a whole shall do this. It is not necessarily a prescription of which piece of the system is going to do this. It's just that the system as a whole somehow has to accomplish this requirement and satisfy this requirement.

On the other hand, we have requirements saying voting devices shall. This means each and every voting device in the system shall satisfy this requirement in and of itself.
You can easily envision that there are both sorts of requirements, and it's important to keep them separate. Yes.

[Slide 12]

[QUESTIONER:] Okay, and this has been a problem in the past, because, say a vendor has an optical scan system and a DRE system- are those considered devices or systems because in the past, they were tested as individual systems, and for those of us who use both of them in a polling place, a lot of issues would arise in the implementation of both of them. So are they devices or are they one system?

[MR. FLATER:] This does address the issue of what we might call blended systems. In the previous guidelines, it talked about paper-based systems and precinct count systems, and in reality, we had devices of both types being integrated.
What you described is a case where we had some paper-based devices, some DRE devices, but they work together to form a voting system.

[QUESTIONER:] So they have to be tested in conjunction with each other? That might be correct?

[MR. FLATER:] Yes.

[QUESTIONER:] Okay, so no longer will you test this piece on it and then this piece on it and certify those individually. They have to be certified that they work in conjunction with each other?

[MR. FLATER:] Yes, but there may be multiple configurations.

[QUESTIONER:] Right, and I understand that. It's just that there is going to be some way of knowing they interact together at the same-

[MR. FLATER:] Yes.

[QUESTIONER:] I assume that means one vendor's products. But what do you do- are you all setting up interoperability between vendors products? Because you're obviously - Diebold is not going to test ES&S equipment to find out if it works with Diebold equipment.

[MR. FLATER:] What we have here does not include an interoperability testing regime. What we have here is a set of guidelines. Somebody brings a system and applies to have this system- you know, the conformity of the system, assessed.

Although it's unlikely, there is nothing prohibiting the occurrence of- I mean let's say two vendors miraculously shake hands one day and say we want to be interoperable. They are entitled to apply for certification of a blended system including all those products, but I mean that's completely orthogonal to the definitions, if you will. What happens is we have an application to certify a system that includes certain devices. If they go together- if different sorts of devices go together in different sorts of ways, we get a more complicated request where we've got this configuration and that configuration. Ultimately they can all be certified.

[QUESTIONER:] Just to make sure I understand. You have got an optical scan system, and you take that through testing- and it's optical state and you're selling it as optical scan, and you also have a DRE system that is going to get tested and used as a voting system- as a DRE voting system. And some county decides, as in the case of Boone County Missouri, that they use those two devices together as part of an election package in her county, you're not really saying that they then become one system.

[MR. FLATER:] That's a third configuration. You described three configurations. One is a purely optical scan system, one is a purely DRE system, and one is a blended system.

[QUESTIONER:] Ok, just so I understand.

[QUESTIONER:] If, say the vendor decides - If they're going to sell them to any one jurisdiction for use in an election, together that would be a blended system, right? And to conform with this, if they are going to have those being used together to conform to these requirements, they would have to be tested as a blended system?

[MR. FLATER:] That is correct.

[MR. HANCOCK:] David, if I may, this is nothing new. This is the way the conformance and certification is going on now. They can have a certified op scan system. They can have a certified DRE system. When they bring them together, that's a different configuration and would have to be certified separately. That's happening now. There's no difference at this point.

[Slide 13]

[MR. FLATER:] Now besides voting device and voting system, another term you may see throughout the document is voting process. Voting process is used to refer to the big picture where people and processes, election administration concerns, etc., are included. Now it's important to note, the people and processes are outside the scope of the product standard. They're not included in the definition of voting system, and they are not assessed by test labs except indirectly when the manufacturer says that the following procedures shall be followed when using the system, it is assumed by the test lab that those procedures will be followed, but the guidelines themselves very carefully attempt to avoid constraining these processes that are the prerogative of the jurisdictions to determine. Now when there is a requirement on the voting system to play nice with a certain process, the VVSG does refer to the voting process but attempts to do so in such a manner that doesn't step on the toes of the jurisdictions being able to determine their own procedures.

[Slide 14]

I'm now going to move on to the subject of classes, another one of the prerequisites for understanding the requirements.
A class is an identified set of voting systems or voting devices sharing a specified characteristic. There are diagrams in part one, section 2.5.2, which show the classes that are referred to in the document and how they relate to one another. Primarily classes are used to narrow the scope of requirements, using the 'applies to' field.

[Slide 15]

The two class diagrams that you can find in part one, section 2.5.2, form generalization and specialization hierarchies, or if you want to use the formal term that applies, it's a lattice. Generalization specialization refers to- okay, the more general class is pine tree, and the specialization would be long-needle pine and short-needle pine. Pine tree would also have a sibling class referring to leafy trees.

You can describe similar relationships between different kinds of forests. You can have a pine forest or you can have the type of forest that drops its leaves in the fall.
This is analogous to the relationship between these different sorts of trees, but the relationship between the forest and the trees that compose it is a different kind of relationship. This is a partative relationship as opposed to the generic relationship, and that's why we have two separate diagrams, one diagram talking about voting systems, one diagram talking about the voting devices that compose those systems.

[Slide 16]

Now the voting system diagram is the simpler one. In addition to the general class voting system, there are numerous voting variations that are called out underneath it. The sub-class is simply referred to voting systems that support this voting variation. I'm going to talk more about voting variations later. There's one other class net diagram, IVVR independent voter verifiable records, which will be discussed tomorrow in the Security and Transparency Subcommittee's presentation.

[Slide 17]

The more complex diagram is the voting device class diagram. At the top there is, of course, the general class voting device that includes all voting devices. Voting devices are also broken down into sub-classes by the different optional voting variations that they support. For example, straight party voting devices, a voting device that supports straight party voting.
Commonly understood device categories such as DRE or optical scanner appear in this diagram. There are also various generalizations such as vote capture device, all devices that are responsible for collecting votes from the voters- tabulator, paper-based device, program device, generalizations such as that would have requirements that would apply not to a particular device category but to a certain general type of device.

And there are other important concepts as well such as IVVR vote capture device, accessible voting station, and one of our favorites, VVPAT.

[Slide 18]

Now as you look at these diagrams, one frequent misunderstanding that occurs is many people have an implicit assumption that the classes are mutually exclusive. If you see that there's a class accessible voting station and there's a class DRE, many people assume that you have to pick one when you're looking at a particular device, and in fact you don't.

[Slide 19]

We use a notation with the wedge symbol. We have accessible voting station wedge DRE, refers to the intersection of those two classes. These are accessible voting stations that happen to be DREs, or DREs that are also accessible voting stations.

[Slide 20]

The reason this makes sense has to do with the structuring of the requirements in trying to reduce the number of times that we have to repeat a given requirement in different contexts.
Certain requirements are essential to DREs. Certain requirements are essential to accessible voting stations, but it's not the case that all DREs are accessible voting stations. It's not the case that all accessible voting stations are DREs.

If we use the class structure as I've described, then it's clear that the device that is both a DRE and an accessible voting station must satisfy both sets of requirements, but we need not repeat these requirements.

If we didn't have the class structure, you would have DREs, you would have accessible DREs, you would have electronically assisted ballot printers, and then you would have accessible EBPs, and there would be a great deal of repetition of requirements between the accessible DREs and the accessible EBPs, and what we've done here is we've factored those out.
We've pulled out this generalization of accessible voting stations, put these requirements in there once, and then every device that claims to be accessible automatically inherits those requirements.

[Slide 21]

Since we know everything about classes, it's time for a quiz, and every good quiz is a trick question, so this one's no exception. Is the subclass paper-based device wedge tabulator any different from the class optical scanner?

[Slide 22]

Well, the answer is maybe. If we take a completely concrete analysis of the world as it stands, we have a mandate to remove guidance for punch card systems out of the guidelines.
Right now there does not exist a paper-based tabulator within our universe of discourse which is not an optical scanner. So these classes are for now equivalent, if we just look at the members of the classes, but there is a different intent here.

When we talk about paper-based tabulators, we may be thinking about punch card readers or we may be thinking about a technology yet to come which is a paper-based tabulator, but it does not use optics, so by keeping these classes separate, even though punch card reader has been removed from the spec, we're preserving this distinction in the event that an innovative new sort of paper-based tabulator comes along. We will already have a set of requirements that has been tagged for that sort of device separate from the requirements that have been tagged as being specific to optical scanners, whereas if we eliminated this distinction, then this work of separating those requirements would have to be done at the point where the counter example appeared, where someone invented a paper-based tabulator that was not an optical scanner.

[Slide 23]

[NARRATOR:] Additional explanatory presentations on the Voluntary Voting System Guidelines can be accessed from the web site: vote.nist.gov.

Page Created: December 4, 2007
Last Updated: July 17, 2008

Privacy policy / security notice / accessibility statement
Disclaimer / FOIA
NIST is an agency of the U.S. Commerce Department