cmakemodules/MANUAL.md

cotire manual

Cotire (compile time reducer) is a CMake module that speeds up the build process of CMake based build systems by fully automating techniques as precompiled header usage and single compilation unit builds for C and C++.

motivation

Cotire was born out of a dissatisfaction with the existing CMake solutions for adding precompiled header support and unity build support to CMake based build systems. The design of cotire tries to adhere to the following principles:

as automatic as possible

Precompiled header and unity builds are good ideas in principle, but in reality they do not work if the burdon of maintaining the required additional source files (a prefix header and a unity source file) is put on the developer. A modern build system like CMake provides enough context information to have the build system generate and update these files automatically.

non-intrusive

The configuration of precompiled headers usage and single computation unit builds belongs to the build system and not in the source code. Nobody wants to litter one’s source files with hdrstop pragmas or be forced to add an include directive to every file. The same source code should build properly when a precompiled header isn’t used and should build faster when a precompiled header is used.

minimal interface

Maintaining a build system over time is enough work and the CMake language may often get in your way. Thus the solution should only add few public CMake functions. It should be easy to integrate it into an existing CMake based build system and it should be just as easy to remove it again.

lazy file creation

The additional source files needed for precompiled header support and unity build support should only be created when they are required for the compilation of a target. Thus the solution should not create these files upon configuring the project, but should set up custom build commands for the creation of these files that only kick in when the files are required to exist by the build process.

cross-platform

C/C++ Compilers and IDEs on different platforms vary widely in how the implement precompiled header support. The solution should hide these implementation details and present a uniform interface to the developer on all supported platforms.

cotire basic usage

Cotire consists of a single CMake module file, which can be easily added to an existing CMake project.

The file CMake/cotire.cmake needs to be copied to the module directory of a CMake project. In the top-level CMakeList.txt file, the module directory needs to be added to the CMake module search path:

set (CMAKE_MODULE_PATH "${CMAKE_SOURCE_DIR}/CMake")

To use cotire in a CMake project, one adds the following include directive to the beginning of the top-level CMakeList.txt:

include(cotire)

To speed the build process of a CMake library or executable target, the cotire function is applied to a CMake target. From the example project that ships with cotire:

add_executable(example main.cpp example.cpp log.cpp log.h example.h)
...
cotire(example)

Cotire looks at the properties of the target provided by CMake (e.g., target type, source files, compile flags, preprocessor defines, include directories, ...) and modifies the target’s build process in the following way:

1. cotire adds a custom build rule to produce a unity source file from the target’s sources.
2. cotire adds a custom build rule to produce a prefix header file by tracking the header files included by the target’s sources.
3. cotire adds a custom build rule to produce a precompiled header from the prefix header.
4. cotire modifies the target’s compile flags to make use of the generated precompiled header.
5. cotire adds a couple of new targets.

For makefile based build systems, running make help in the terminal reveals the new targets:

$ make help
...
... all_pch
... all_unity
... clean_cotire
... example
... example_pch
... example_unity

The example_pch target triggers the compilation of the precompiled header and as a side effect the generation of the unity source and the prefix header. The target clean_cotire cleans up all files generated by cotire. The example_unity target produces the same output as the original example target, but does so by performing a unity build. The all_pch and all_unity serve as pool targets for all cotired project targets.

The example_unity target inherits all build settings from the original target example except for linked libraries and target dependencies. To get a linkable unity target, the required libraries have to be added manually to the unity target with target_link_libraries.

cotire generated files

For a target that has been cotired, three files will be generated as part of the build process:

the unity source

The unity source file is generated from the target by querying the target’s SOURCES property. It consists of preprocessor include directives for each of the target source files. The files are included in the same order that is used in the CMake add_executable or add_library call. Header files are omitted.

This is a unity source generated for the example project under Mac OS X:

#ifdef __cplusplus
#include "/Users/sakra/Documents/cotire/src/main.cpp"
#include "/Users/sakra/Documents/cotire/src/example.cpp"
#include "/Users/sakra/Documents/cotire/src/log.cpp"
#endif

The unity source file uses absolute paths to include the target’s source file. The file is not intended to be portable across different build folders or machines. It is an intermediate file tied to the build folder that is automatically recreated by the build system if it is missing.

the prefix header

The prefix header is produced from the unity source file by running the unity file through the preprocessor and keeping track of each header file used (this is done by using option -H with GCC / Clang and /showIncludes with Visual Studio C++). The path of each used header file is compared against an exclude directory list and an include directory list to decide if the header file should be added to the prefix header.

By default the include directory list is empty and the exclude directory list is initialized to "${CMAKE_SOURCE_DIR};${CMAKE_BINARY_DIR}". This default setting guarantees that project headers which are likely to be changed frequently are not added to the prefix header.

Upon generation of the prefix header cotire makes sure that target compile options, include path settings and preprocessor defines (e.g., NDEBUG) that affect the outcome of the preprocessor are correctly set up.

Generating the prefix header from the unity source is much faster than running each individual target source file through the preprocessor, because the coalesced unity source will make the preprocessor process most header files only once.

This is a prefix header produced for the example project with Visual Studio 2008 under Windows:

#ifdef __cplusplus
#include "C:\Program Files\Microsoft Visual Studio 9.0\VC\include\string"
#include "C:\Program Files\Microsoft Visual Studio 9.0\VC\include\algorithm"
#include "C:\Program Files\Microsoft Visual Studio 9.0\VC\include\iostream"
#endif

Generating the prefix file under Ubuntu Linux with GCC 4.6 yields the following result:

#ifdef __cplusplus
#include "/usr/include/c++/4.6/string"
#include "/usr/include/c++/4.6/algorithm"
#include "/usr/include/c++/4.6/iterator"
#include "/usr/include/c++/4.6/iostream"
#endif

Using Clang 3.1 under Mac OS X 10.7 this is the resulting prefix header:

#ifdef __cplusplus
#include "/usr/include/c++/4.2.1/string"
#include "/usr/include/c++/4.2.1/iostream"
#include "/usr/include/c++/4.2.1/iterator"
#endif

Cotire attempts to produce a minimal list of header files by omitting header files indirectly included by a header that is already part of the prefix header. Header files with nonstandard file extensions like .inl, .inc of .ipp are omitted by default.

The generated prefix file includes the selected header files by their absolute paths. This speeds up the precompiling of the prefix header because the compiler does not have to search for header files in include directories again.

The prefix header is tailored to the CMake target that it is generated for and cannot be re-used for a different CMake target. It is tied to the compiler environment of the local machine and is not portable across different compilers or machines. It is automatically recreated by the build system if it goes missing.

the precompiled header

The precompiled header is produced from the generated prefix header by using the proprietary precompiling mechanism depending on the compiler used. For GCC and Clang cotire sets up a custom build rule and generates the precompiled header as described in the documentation for GCC and Clang. Cotire then modifies the COMPILE_FLAGS property of the target to force the inclusion of the prefix header.

Visual Studio C++ uses a different approach to pre-compiling. It requires a host source file to generate the precompiled header as a side effect of producing an object file. Cotire modifies the COMPILE_FLAGS of the first target source file to generate the precompiled header and then modifies the COMPILE_FLAGS of the remaining target source files to include the generated precompiled header.

For Xcode projects generated with CMake, cotire completely hands off the pre-compilation of the prefix header and the inclusion of the precompiled header to the IDE. Cotire attaches a pre-build action to the target which generates the unity source file and the prefix header. Cotire then modifies Xcode attributes of the generated Xcode project to have Xcode precompile the the generated prefix header with the Xcode build steps ProcessPCH for C sources and ProcessPCH++ for C++ sources.

For precompiled headers creation flags must match use flags exactly. Cotire uses the same flags, include directories and preprocessor defines that are used for the compilation of source files for the generation of the precompiled header. Thus the resulting precompiled header binary is only usable for the target and cannot be re-used for a different CMake target.

cotire advanced usage

mixed-language targets

Cotire is able to speed up the build process of mixed language targets, consisting of both C and C++ sources. It generates a separate set of unity source files, prefix headers and precompiled headers for both languages and modifies the COMPILE_FLAGS of each target source file to include the correct precompiled header depending on the compilation language of the source file.

obtaining the names of the generated files and targets

For a cotired target the target properties COTIRE_<LANG>_UNITY_SOURCE, COTIRE_<LANG>_PREFIX_HEADER, COTIRE_<LANG>_PRECOMPILED_HEADER will be set to the paths of the generated files (<LANG> can be set to CXX or C). The target property COTIRE_UNITY_TARGET_NAME will be set to the name of the generated unity target.

If a source file’s COMPILE_FLAGS are modified by cotire, it sets the source file property COTIRE_TARGET to the name of the target, that the source file’s build command has been altered for.

restricting cotire to certain build configurations

To restrict the cotire related modifications to the build process to certain build configurations, the CONFIGURATIONS parameter can be added to the cotire call.

cotire(example CONFIGURATIONS Release MinSizeRel)

For single build type builds the selected configuration will be checked at configure time, for multi-configuration builds the check will be done at build time.

It is recommended to have at least one build configuration that does not make use of cotire to ensure that the project builds properly without cotire.

disabling precompiled headers and unity builds

If the target’s build process should not be modified to make us of the generated precompiled header, the target property COTIRE_ENABLE_PRECOMPILED_HEADER can be set to FALSE:

set_target_properties(example PROPERTIES COTIRE_ENABLE_PRECOMPILED_HEADER FALSE)
cotire(example)

If a unity build target should not be added by cotire, the target property COTIRE_ADD_UNITY_BUILD can be set to FALSE:

set_target_properties(example PROPERTIES COTIRE_ADD_UNITY_BUILD FALSE)
cotire(example)

Both properties default to TRUE. If both are set to FALSE, cotire will only set up custom build rules for the generation of the unity source and the prefix header.

The properties COTIRE_ENABLE_PRECOMPILED_HEADER and COTIRE_ADD_UNITY_BUILD can also be set on directories. A target inherits the property value from its enclosing directory.

disabling precompiled headers for small targets

The cache variable COTIRE_MINIMUM_NUMBER_OF_TARGET_SOURCES can be set to the minimum number of source files required to enable the use of precompiled header. It defaults to 3.

configuring the generation of the prefix header

There are multiple target properties which affect the generation of the prefix header:

• COTIRE_PREFIX_HEADER_IGNORE_PATH can be set to a semicolon separated list of directories. If a header file is found in one of these directories or sub-directories, it will be excluded from the generated prefix header.

• COTIRE_PREFIX_HEADER_INCLUDE_PATH can be set to a semicolon separated list of directories. If a header file is included from one of these directories or sub-directories, it will be included in the generated prefix header.

If a header file is matched by both COTIRE_PREFIX_HEADER_IGNORE_PATH and COTIRE_PREFIX_HEADER_INCLUDE_PATH, the option which yields the closer relative path match wins. For example, if third-party libraries are part of the source tree in a directory called Libs, the following setting will make cotire select header files from the third-party directory, but ignore other project related headers in CMAKE_SOURCE_DIR:

set_target_properties(example PROPERTIES
    COTIRE_PREFIX_HEADER_IGNORE_PATH "${CMAKE_SOURCE_DIR}"
    COTIRE_PREFIX_HEADER_INCLUDE_PATH "${CMAKE_SOURCE_DIR}/Libs")

The properties COTIRE_PREFIX_HEADER_IGNORE_PATH and COTIRE_PREFIX_HEADER_INCLUDE_PATH can also be set on directories.

The following cache variables also affect the selection of prefix headers.

• Directory paths in COTIRE_ADDITIONAL_PREFIX_HEADER_IGNORE_PATH will be added to the list of ignored directories when the prefix header file is created.

• COTIRE_ADDITIONAL_PREFIX_HEADER_IGNORE_EXTENSIONS can be used to ignore header files by file extension. It defaults to the CMake list inc;inl;ipp.

During development changes to the project source files may affect the list of header files that should be selected for inclusion in the prefix header (e.g., a standard include may be added or removed from a target source file). Cotire does not automatically recreate the prefix header, when a target source file is changed, because this would always trigger a re-compilation of the precompiled header and would result in a rebuild of the whole target. To make the prefix header creation dependent on changes to certain target source files, the source file property COTIRE_DEPENDENCY can be set to TRUE for those files.

configuring the generation of the unity source

By default cotire adds all target source file to the generated unity source. In most cases a unity build will not work out of the box, because unity builds break the use of some C and C++ language features. Unity build problems can be tackled in the following way:

• Change the order of the source files in the add_executable or add_library calls. Problematic source files should be moved towards the end.

• Set the source file property COTIRE_EXCLUDED on problematic source files. The source file will not be included in the unity source file and will be compiled separately when the unity build is performed.

• If the unity source file is too large and the compilation process runs into a compiler limit, the target property COTIRE_UNITY_SOURCE_MAXIMUM_NUMBER_OF_INCLUDES can be set. If the target contains more than that number of source files, cotire will create multiple unity source files for it. Each unity source file is compiled separately when the unity build is performed. The property is initialized by value of the cache variable COTIRE_MAXIMUM_NUMBER_OF_UNITY_INCLUDES.

• Another way to break up a large unity source file is to set the source file property COTIRE_START_NEW_UNITY_SOURCE to TRUE on selected target source files. If cotire encounters this property, it will complete the current unity file and start a new one. The new unity source file will include the source file as the first one. This property essentially works as a separator for unity source files.

fixing macro definition clashes

Many unity build problems stem from macro definitions leaking into other target source files, where they may conflict with other definitions of the same name. Cotire adds the properties COTIRE_UNITY_SOURCE_PRE_UNDEFS and COTIRE_UNITY_SOURCE_POST_UNDEFS to fix macro definition clashes.

As an example, if these properties are set on a source file of the example project:

set_source_files_properties (example.cpp PROPERTIES
    COTIRE_UNITY_SOURCE_PRE_UNDEFS "max;min"
    COTIRE_UNITY_SOURCE_POST_UNDEFS "DEBUG_TYPE")

This will make cotire add #undefs to the generated unity source file.

#ifdef __cplusplus
#include "/Users/sakra/Documents/cotire/src/main.cpp"
#undef min
#undef max
#include "/Users/sakra/Documents/cotire/src/example.cpp"
#undef DEBUG_TYPE
#include "/Users/sakra/Documents/cotire/src/log.cpp"
#endif

The properties COTIRE_UNITY_SOURCE_PRE_UNDEFS and COTIRE_UNITY_SOURCE_POST_UNDEFS can also be set on targets. Cotire will add #undefs for each source file in the unity source then.

enabling verbose builds

The cache variable COTIRE_VERBOSE can be set to TRUE to see all compile commands used when generating the cotire related files. Cotire will also print the contents of the generated unity source and the prefix header for verbose builds. COTIRE_VERBOSE defaults to FALSE. When using a Makefile generator COTIRE_VERBOSE defaults to the value of the makefile variable VERBOSE (i.e., make VERBOSE=1).

cotire usage restrictions

using source files for multiple targets

When the same set of source files is used for different targets (e.g., for producing a static and a shared library variant from the same sources), using a precompiled header may not work. Under certain circumstances, cotire cannot enable the precompiled header usage by changing the COMPILE_FLAGS property of the whole target, but must set the COMPILE_FLAGS properties of individual target source files instead. This will break the usage of the source file for multiple targets.

multi-architecture builds under Mac OS X

Neither GCC nor Clang support the use of precompiled headers when performing a Mac OS X multi-architecture build (e.g., using option -DCMAKE_OSX_ARCHITECTURES=i386;x86_64).