VVSG TRAINING NARRATION
NIST BOULDER- VVSG Overview Module Part 1
October 15, 2007

VVSG Tutorial Narration*
VVSG Overview Part 1

[Slide 1]

[NARRATOR:] The following presentation is Part One of the next Voluntary Voting System Guidelines Overview Training Module. There are four parts to this VVSG training module. Each presentation reviews a different section of the guidelines document in layman's language. The overview includes question and answer sessions with members of the Election Assistance Commission's Board of Advisors and Standards Board.

[MR. WACK:] Hi, folks. Good morning, I'm John Wack, and what I'm going to do is basically try to give you an overview and help you to essentially get a good foundation on the requirements structures, where they are in the document, and a whole host of things about the document. And primarily this is so the other presenters after me don't have to do that, and you'll be able to better follow their presentations.

[Slide 2]

So basically what I'll try to do here is just, as best as I can, give you the tools to understand how to use the document. And I believe that one of your goals really is to understand the document as well as you can, so that you can explain it to other people.

[Slide 3]

This is what I'm going to talk about essentially, just an overview. Mark already gave you a lot of that. I'll walk through the material somewhat quickly. Then I'm going to talk about the conformance clause, specifically the material in it, because I think that really is the foundational part of the VVSG. You have to understand that. From there, you can then go on to the other presentations.

[Slide 4]

Okay, overview of the VVSG document. Again, these first couple of slides, I'm going to breeze through a little bit.

[Slide 5]

I really put together this slide here to just basically say we're talking about new equipment, but as much as possible, and I think this is definitely true in the core requirements area, the aim was if it wasn't broken, don't try to fix it.

Make things more specific from previous versions of the standards. Don't just throw in a lot of new stuff; that's pie in the sky. If there was a way to do it in software, people tried to do it in software as opposed to changing hardware.

We aren't experts in the area of how much things will cost, what are the cost-benefit ratios of adding new types of hardware and so on and so forth, but we tried as much as possible to not make voting systems more expensive, so we actually did look quite a bit at what things would cost.
In some areas, we possibly even reduced the costs a little bit, perhaps from what's in VVSG 2005. One specific area: basically we had a requirement in 2005 to have a trusted external interface that would in essence tell you that you have the correct version of the voting systems software out there on the voting system.

And we actually reduced that requirement. We took away that requirement as being a mandatory requirement in this version because of other things we added.

[Slide 6]

So a lot of discussion about this: I wanted to emphasize that the primary audience really has to be vendors and test labs. It has to be this crowd; it has to be written for them. They are the ones that actually build voting systems and test them. They need this to be as precise as possible and, at the same time, it's a very different type of standard, in that unlike the standard for your USB drives or what have you, it is out in the public. People want to know about it.

Election officials in particular, who haven't always been as technical as vendors or test labs, need to understand it; states need to go through it, need to write contracts based on the material in it, things of that sort. So there's this tension between making it very precise for the test labs and vendors and plain language, making it very repeatable, and in some cases, the tension is that we've made things very terse in some areas because the more you repeat things, the likelihood is you're going to get it wrong in some spot.

So really, the right thing is to write what's now called the companion document. We really do have to have another document out there that does a better job of giving an overview and explaining the material, so that it's a little bit more understandable.

[Slide 7]

This document, it's structured to be a tool. It really is the primary tool that testers and implementers have to work with.
That's really all it is, and it's really meant to be used electronically as well. We put a lot of hypertext links in there and things of that sort, but we did put a lot of work into just making this thing a standard.

And I'm not saying that the earlier versions of the guidelines were not standards, but we followed typical standard structures and things that are very common in the standards world and things that implementers, test labs, and other groups like that are more accustomed to follow. So that's why it looks the way it does.

We did put a lot of time into that, a lot of time into the organization.

[Slide 8]

We worked with usability experts and things of that sort. As a result, I think it did a lot to improve the precision, but the durability also is an important thing.

This undoubtedly will be updated. It will be updated several times during the public review process as tests are written for the requirements. I think Mark or John may have alluded to this. John Cugini may have alluded to this earlier, but as tests are written, perhaps that might mean we need to tighten up some of the requirements, because we find out that the test is not jiving correctly with it.

So I want to emphasize that it was written in a way so that it could be more easily updated. The class structure goes a long ways towards that, and I'll get into that a little bit.

[Slide 9]

The glossary caused us a lot of problems early on. An earlier version of this document listed five volumes, and one of the volumes, it might have just been called volume two, glossary, something of that sort, and it had a paragraph, although it was a small paragraph, it had a paragraph at the front that said the definitions are specific to the VVSG, they aren't going to be the same definitions that you use, but I think that got lost.

It caused a lot of people problems in that they thought we were redefining terms or defining terms in ways that they don't use them. And again, really it's vital, it's critical to be speaking the same language, and even though it may vary a little bit from local usage, everybody has to be on the same page, and that's one of the reasons why in the PDF and the online versions of this in the HTML, everything is hypertext linked.

It may get in your face a little bit seeing all these blue underlines and seeing vote capture device underlined several million times probably in the document, but it is there to serve as a reminder that these are words with specific meanings, specific to the VVSG.

[Slide 10]

Okay, so what I want to do here is just quickly browse through, and in this browsing of the document, I certainly won't do justice to it, but I thought at this particular point, it was worth it to just go through the document quickly, and then people after me are going to obviously go into a lot more detail.

[Slide 11]

I don't think I'll talk about the introduction necessarily here, but again, we ended up with three parts.

[Slide 12]

They were done specifically in that way to basically segregate or group fundamentally different types of requirements into different parts of the document.

Part one simply has device-related requirements. There are lots of requirements in part two that deal with documentation. You know, basically the technical data package, user documentation, test reports, things of that sort.

And it does require at times some jumping around between parts one and two, but it was important to make clear what requirements go into the technical data package and what requirements do not necessarily. So part two, part three a little bit of testing-related information and requirements.

[Slide 13]

So essentially I want to emphasize that the first chapter of each part basically is about changes from VVSG 2005 and previous versions; VVSG 2005 sort of subsumes VSS 2002.

[Slide 14]

So a number of things are described in here that are meant to be overviews, so I highly recommend that you go through these areas, one of these for each part.

From there we started with usability. In a sense, those are the requirements that deal most directly with the way in which the equipment is used.

[Slide 15]

They came first, followed by two sets of chapters for security. This chapter here really, I think, is heavily influenced by software independence and really talking about an audit-based architecture.

Systems have to be auditable in specific ways in order to be software-independent, and then it goes into electronic records, and paper, IVVR records, independent voter-verified records.

[Slide 16]

David and Alan will go into much more detail about the core requirements in here. I myself have always found these to be the most dense requirements. They deal a lot with some pretty tricky areas, and there's a lot of material in there.

[Slide 17]

People in their presentations will refer to requirements that will be in part two, but essentially we're dealing with all the requirements that go into the technical data package.

[Slide 18]

And I urge you to at least read the introductory paragraphs fairly thoroughly. I did try as much as possible to provide a fairly good overview there.

[Slide 19]

And part three is an interesting part. Part three, I think, may have started out really more as a carryover of existing testing-related requirements that were in volume two of VVSG 2005, and again, getting back into the reasons why we put these three different parts together.

I think there were device-related requirements in the testing area of the VVSG 2005, which made it a little bit more difficult to find all the device-related requirements. So again, part three is purely for testing approaches, and it describes general testing approaches that will be used for testing the requirements in this document.

But again, according to what Mark said, it does not have the actual tests themselves. All the requirements refer to these testing approaches.

[Slide 20 & 21]

Chapter three in the document, part three, really doesn't contain requirements, but it really just contains overviews of the different types of testing, as part of conformance testing.

[Slide 22]

Conformance is conformance to the requirements in a guideline. Interoperability is more, does it work with other implementations?
You can have different vendors basically implementing the USB protocol, and they may all conform to the standard, but they may conform to the standard in ways that are slightly different and as a result, they don't interoperate.

Interoperability is something that will probably happen with voting system components and parts.

[Slide 23]

Also open-ended vulnerability testing, I think, is something that people talked about a lot. I think Nelson will go into more detail on that, but that's included as another testing approach.

[Slide 24]

[QUESTIONER:] You say the audience is the test labs and the manufacturers. Yet, throughout this document, there are requirements on election officials or the underlying -- or a manufacturer can meet the requirement based on passing off the responsibility to an election official.
So I think it's really important that you understand that the election official has to be part of the audience on this if you're going to assume a standard is met by the election official.

[MR. WACK:] True. I couldn't disagree with you at all. Very much agree.

[QUESTIONER:] So you'll put us in the audience?

[MR. WACK:] You are listed in the audience in the document itself. What I'm trying to get across to you is that a problem, I think, with previous guidelines was that they weren't specific, precise, and therefore testable in the same way by different test labs.

So sometimes making language very precise works against making it very understandable, and we try to do our best in that regard to make it as understandable as we can, but what it really comes down to is that the companion document is going to have to go a long way towards other people besides test labs and vendors understanding the document very clearly.

[QUESTIONER:] I understand the document, and I understand in the document when it assumes I'm going to be responsible for this.
My concern is often these things get into the testing lab, and the manufacturer meets the specification by saying that will be handled by the election administrator. We never find out about it until after there is a problem, and the vendor says, well, of course we meet the standards, but you were supposed to do it this way.

And so that's what I mean about throughout the course of this, if there is something in there that is going to be - whether it is the standard or how the manufacturer was able to get through the conformance testing to meet the standard, then the person who is the election official has to know that they are responsible for conformance with this standard, not the vendor.

[MR. WACK:] Well, that's a very good point, and I think we'll take in those words and maybe reuse them. I do know that a lot of the documentation requirements that were put in part two specifically kind of repeat what you just said.

The vendor is required to very clearly document the procedures that would go along with the implementation of a particular control, and I know in the HFP chapter, chapter three of part one, there are a number of requirements in there for the usability of the documentation, specifically for election officials so we know what you're talking about, and it's good you're emphasizing it too, and the companion document will emphasize that too.

[Slide 25]

[QUESTIONER:] I was interested in your reference to costs. You indicated that you thought that in at least several cases in the VVSG that it would result in reduced costs.

Some of us are very curious to hear from vendors on this subject about the impact of the VVSG on costs. Could you elaborate a little bit more on that? Are you saying that you think this is actually going to reduce costs overall, or are there just examples of VVSG that some of the costs will be reduced?

[MR. WACK:] I'm not in any position, and I don't think NIST is, to really go into costs a whole lot, other than my statement was basically intended to say we didn't ignore how much things would cost.

We know that voting systems will cost more because, for one reason, they'll be constructed more precisely and tested much more thoroughly. That will just cost more.

When people talk about costs, there are benefits of course- fewer people complaining about equipment not working correctly and so on and so forth and those have to be taken into account as well. But I don't really want to say a whole lot about costs. Mark may want to jump in.

[MR. SKALL:] Yeah, I think John's point was- you may have misinterpreted what he said- I think he said, in certain instances at least, the security people and presumably everyone doing this, when looking at various alternative requirements, took into account that this requirement would be more costly and perhaps went to another requirement that was less costly. That doesn't mean it's going to be less costly in all systems, just less costly in the alternative requirement.

[Slide 26]

[NARRATOR:] Additional explanatory presentations on the voluntary voting system guidelines can be accessed from the Web site: vote.nist.gov.

Page Created: November 28, 2007
Last Updated: June 26, 2008

Privacy policy / security notice / accessibility statement
Disclaimer / FOIA
NIST is an agency of the U.S. Commerce Department