NIST
and the Help America Vote Act (HAVA)
NIST VVSG Test
Development
Why
Test Suites
Standards
101
Conformance
Testing 101
Test
Suite Review Process
Testing
FAQs HAVA, Certification and Testing
Technical Guidelines Development Committee (TGDC)
National Voluntary Laboratory
Accreditation Program
Contact Us
 U.S.
Election Assistance Commission
|
|
Notes to
Reviewers
Source
Code Analyzer Tool Assessment Guide and Test Suite for the VVSG
2005 and VVSG-NI, Version 1.0
April 1, 2009
The
documents available from this page represent a source code analyzer
tool guide and test suite for use by voting system testing labs
as well as manufacturers of voting systems. Please note that this
is NOT a guide or test suite for determining a voting system's
conformance to the VVSG. It is for tool calibration and assessment
only and is being made available to assist test labs and voting
system software manufacturers in understanding, calibrating and
using automated source code analysis tools against coding requirements
prescribed in VVSG.
Test
suite reviewers are advised to first read and understand the relevant
material in the VVSG 2005 and the VVSG-NI relevant to the test
suites under review, before reviewing the test suites. The following
VVSG sections provided the technical requirements information
needed to create this guide and tool test suite:
VVSG
2005:
- Volume
I Section 5.2, Software Design and Coding Standards
- Volume
II Section 5, Software Testing
VVSG-NI:
- Part
3 Section 4.5.1.A and 4.5.1.B, Source Code Workmanship Requirements
A
complete version of the VVSG 2005 in PDF format can be found at
http://www.eac.gov/program-areas/voting-systems/voting-system-certification/2005-vvsg.
A
complete version of the VVSG-NI in HTML, MS-Word, or PDF formats
can be found at http://www.eac.gov/vvsg.
The source code workmanship requirements of Part 3 Section 4 can
be found at: http://www.eac.gov/vvsg/part3/chapter04.php/.
Source
code analysis is part of the "due diligence" performed
by test labs in compliance with VVSG testing requirements. While
still a human-intensive effort, static source code analysis today
is augmented with automated tools that provide greater confidence
that source code is examined in a thorough, reliable and repeatable
way.
This
tool guide provides a general overview of source code analysis
tools, and is accompanied by tool tests (source code examples)
that labs can use to calibrate those tools against VVSG coding
requirements. The tool tests provided with this tool guide represent
an initial collection written in C, C++ and Java languages. As
this effort moves forward, additional tool tests in these languages
and others will be added to strengthen the tool calibration procedures
of test labs.
Commenting:
Please
send comments on the test suites, by July 1, 2009, to: tools-test@nist.gov.
You
may provide comments directly in your email and/or send attachments
in MS-Word or PDF. If you wish, you may embed your comments within
the PDF documentation using the instructions provided here.
In general, please tell us the features you like and provide us
with comments, corrections, and suggestions on how to improve
the test suites. Please provide the following items:
- Test
suite version number (found in the test suite documentation,
currently Version 1.0)
- Your
name and affiliation (include contact information if desired)
- Identification
of the particular tests for which your comment applies
- If
including suggestions for changes to the tests, a description
of the suggested change including an adequate justification
for the change, or a draft replacement for the test including
the justification and any other necessary documentation or commentary
All
comments will be considered. After all comments have been received
and incorporated into the test suites, a new version of the test
suites will be posted on the NIST web site.
The
source code analyzer tool guide and test suite is available is
available here in a
|
