Bringing Incremental Builds to Continuous Integration

Maudoux, Guillaume [UCL] Mens, Kim [UCL]

Incremental builds are commonly used to speed up the edit- compile-test loop during program development. By updating only the required parts of a system, build systems can shorten the compilation phase by orders of magnitude. While this technique is commonly used for local builds, it is seldom enabled during continuous integration. Current build system do not offer strong correctness properties on incremental builds. In continuous integration setups, it is therefore difficult to achieve both correctness and efficiency simultaneously. Facing this choice, release engineers tend to favor correct builds over optimized builds. In this article, we show that it is possible to obtain both incremental and correct builds. We start by showing that incremental builds are a desirable optimization in continuous integration environments. Different reasons that prevent release engineers to enable incremental builds in practice are discussed. From these, we derive requirements to be met by future build systems to support incremental continuous integration. Whenever possible, we illustrate shortcomings of build systems with insights from current research and industry efforts in new build systems. We also list existing projects that could be combined into a complete tool supporting efficient and correct incremental compilation in continuous integration environments. Ultimately, this paper defines a new research direction at the intersection of build systems and continuous integration.