Numerous specification techniques have been proposed to improve the development of computer software. The present paper reports results from one experiment to compare the effectiveness of functional testing given three different sets of software specifications. Three groups of experienced programmers independently developed functional test cases for the same automated cruise control software. One group worked solely from a natural language description, one was given a graphical, real-time, structured analysis (RTSA) specification to augment the natural-language description, and a third was given, in addition to the English-language description and the RTSA specification, an executable specification. Two measures of performance were evaluated: 1) the degree of statement coverage achieved by the test cases created by each group and 2) the amount of time taken to create the test cases. No significant difference in performance was found among any of the groups for either of the measures. Individual differences in performance among the subjects were observed. Additional statistical tests were used to evaluate the effect of other factors on these individual differences. Such differences appear to be attributable to variations among the subjects in effort, ability, motivation, and understanding of the assignment. These results suggest that software engineering researchers should investigate people-related, management issues that, perhaps, provide the most significant, short-run influence on the performance of software engineers.
