Automatic Test Generation for Data-Driven Applications

All human beings make mistakes. Programmers are human beings. Thus programmers make mistakes. This seemingly harmless syllogism can have disastrous consequences for the society. As larger parts of our economic and social life rely on computers, software bugs caused by programming mistakes do pose an increasing thread. An example which has left its mark in people's minds is the Ariane 5 rocket explosion in flight, due to an integer overflow in the onboard computer. In current software engineering practice, testing is the primary approach to locate programming mistakes. Before going into production, software is thus tested in a set of real scenarios, in order to detect the bugs at best. Nevertheless, the concrete process enabling software testing in practice is complex, repetitive and costly. This is why researchers and industrialists have been working to automate it for long. The idea is notably that the computer helps the programmer to select automatically the most relevant test scenarios. An interesting technique to do so is symbolic execution, today in use at Nasa and Microsoft. Technically, given a code unit to test, symbolic execution enables the computer to generate test values for the input parameters, making it possible to cover a representative set of execution paths in the code. The present work introduces an extension for symbolic execution, enabling one to use this approach in a new context: testing transactional and data-oriented code units. This kind of code units, frequent in practice, is characterized by a brief and targeted interaction with a relational database, storing large amounts of information in a structured way. Finding bugs in such a kind of code unit is critical, as it can avoid corrupting possibly very important data. In order to achieve this, our work goes back to the theoretical foundations of relational databases and proposes a relational symbolic execution mechanism for SQL. This enables generating automatically input data, including some valid content for the database, making it possible to test the considered code unit with an adequate coverage. The proposed extension is implemented within a tool and evaluated over code samples mixing Java and SQL statements. Our approach is compared with related work. A set of research directions to make it usable at a larger scale are identified and discussed.