CHOC'LATE: A framework for specification-based testing

Abstract

In spite of its importance in software reliability, testing is labor intensive and expensive. It has been found that software testing without a good strategy may not be more effective than testing the system with random data. Obviously, the effectiveness of testing relies heavily on how well the test suite-the set of test cases actually used-is generated. This is because the comprehensiveness of the test suite will affect the scope of testing and, hence, the chance of revealing software faults. There are two main approaches to generating test suites: specification-based and code-based. The former generates a test suite from information derived from the specification, without requiring the knowledge of the internal structure of the program. The latter approach, on the other hand, generates a test suite based on the source code of the program. Neither of these approaches is sufficient; they are complementary to one another. In software development, the requirements have to be established before implementation, and the specification should exist prior to coding. In this respect, the specification-based approach to test suite generation is particularly useful because test cases can be generated before coding has been completed. This facilitates software development phases to be performed in parallel, thus allowing time for preparing more thorough test plans and yet shortening the length of the whole process. © 2010 ACM.