1=============================== 2Building a Distribution of LLVM 3=============================== 4 5.. contents:: 6 :local: 7 8Introduction 9============ 10 11This document is geared toward people who want to build and package LLVM and any 12combination of LLVM sub-project tools for distribution. This document covers 13useful features of the LLVM build system as well as best practices and general 14information about packaging LLVM. 15 16If you are new to CMake you may find the :doc:`CMake` or :doc:`CMakePrimer` 17documentation useful. Some of the things covered in this document are the inner 18workings of the builds described in the :doc:`AdvancedBuilds` document. 19 20General Distribution Guidance 21============================= 22 23When building a distribution of a compiler it is generally advised to perform a 24bootstrap build of the compiler. That means building a "stage 1" compiler with 25your host toolchain, then building the "stage 2" compiler using the "stage 1" 26compiler. This is done so that the compiler you distribute benefits from all the 27bug fixes, performance optimizations and general improvements provided by the 28new compiler. 29 30In deciding how to build your distribution there are a few trade-offs that you 31will need to evaluate. The big two are: 32 33#. Compile time of the distribution against performance of the built compiler 34 35#. Binary size of the distribution against performance of the built compiler 36 37The guidance for maximizing performance of the generated compiler is to use LTO, 38PGO, and statically link everything. This will result in an overall larger 39distribution, and it will take longer to generate, but it provides the most 40opportunity for the compiler to optimize. 41 42The guidance for minimizing distribution size is to dynamically link LLVM and 43Clang libraries into the tools to reduce code duplication. This will come at a 44substantial performance penalty to the generated binary both because it reduces 45optimization opportunity, and because dynamic linking requires resolving symbols 46at process launch time, which can be very slow for C++ code. 47 48.. _shared_libs: 49 50.. warning:: 51 One very important note: Distributions should never be built using the 52 *BUILD_SHARED_LIBS* CMake option. That option exists for optimizing developer 53 workflow only. Due to design and implementation decisions, LLVM relies on 54 global data which can end up being duplicated across shared libraries 55 resulting in bugs. As such this is not a safe way to distribute LLVM or 56 LLVM-based tools. 57 58The simplest example of building a distribution with reasonable performance is 59captured in the DistributionExample CMake cache file located at 60clang/cmake/caches/DistributionExample.cmake. The following command will perform 61and install the distribution build: 62 63.. code-block:: console 64 65 $ cmake -G Ninja -C <path to clang>/cmake/caches/DistributionExample.cmake <path to LLVM source> 66 $ ninja stage2-distribution 67 $ ninja stage2-install-distribution 68 69Difference between ``install`` and ``install-distribution`` 70----------------------------------------------------------- 71 72One subtle but important thing to note is the difference between the ``install`` 73and ``install-distribution`` targets. The ``install`` target is expected to 74install every part of LLVM that your build is configured to generate except the 75LLVM testing tools. Alternatively the ``install-distribution`` target, which is 76recommended for building distributions, only installs specific parts of LLVM as 77specified at configuration time by *LLVM_DISTRIBUTION_COMPONENTS*. 78 79Additionally by default the ``install`` target will install the LLVM testing 80tools as the public tools. This can be changed well by setting 81*LLVM_INSTALL_TOOLCHAIN_ONLY* to ``On``. The LLVM tools are intended for 82development and testing of LLVM, and should only be included in distributions 83that support LLVM development. 84 85When building with *LLVM_DISTRIBUTION_COMPONENTS* the build system also 86generates a ``distribution`` target which builds all the components specified in 87the list. This is a convenience build target to allow building just the 88distributed pieces without needing to build all configured targets. 89 90.. _Multi-distribution configurations: 91 92Multi-distribution configurations 93--------------------------------- 94 95The ``install-distribution`` target described above is for building a single 96distribution. LLVM's build system also supports building multiple distributions, 97which can be used to e.g. have one distribution containing just tools and 98another for libraries (to enable development). These are configured by setting 99the *LLVM_DISTRIBUTIONS* variable to hold a list of all distribution names 100(which conventionally start with an uppercase letter, e.g. "Development"), and 101then setting the *LLVM_<distribution>_DISTRIBUTION_COMPONENTS* variable to the 102list of targets for that distribution. For each distribution, the build system 103generates an ``install-${distribution}-distribution`` target, where 104``${distribution}`` is the name of the distribution in lowercase, to install 105that distribution. Each target can only be in one distribution. 106 107Each distribution creates its own set of CMake exports, and the target to 108install the CMake exports for a particular distribution for a project is named 109``${project}-${distribution}-cmake-exports``, where ``${project}`` is the name 110of the project in lowercase and ``${distribution}`` is the name of the 111distribution in lowercase, unless the project is LLVM, in which case the target 112is just named ``${distribution}-cmake-exports``. These targets need to be 113explicitly included in the *LLVM_<distribution>_DISTRIBUTION_COMPONENTS* 114variable in order to be included as part of the distribution. 115 116Unlike with the single distribution setup, when building multiple distributions, 117any components specified in *LLVM_RUNTIME_DISTRIBUTION_COMPONENTS* are not 118automatically added to any distribution. Instead, you must include the targets 119explicitly in some *LLVM_<distribution>_DISTRIBUTION_COMPONENTS* list. 120 121We strongly encourage looking at ``clang/cmake/caches/MultiDistributionExample.cmake`` 122as an example of configuring multiple distributions. 123 124Special Notes for Library-only Distributions 125-------------------------------------------- 126 127One of the most powerful features of LLVM is its library-first design mentality 128and the way you can compose a wide variety of tools using different portions of 129LLVM. Even in this situation using *BUILD_SHARED_LIBS* is not supported. If you 130want to distribute LLVM as a shared library for use in a tool, the recommended 131method is using *LLVM_BUILD_LLVM_DYLIB*, and you can use *LLVM_DYLIB_COMPONENTS* 132to configure which LLVM components are part of libLLVM. 133Note: *LLVM_BUILD_LLVM_DYLIB* is not available on Windows. 134 135Options for Optimizing LLVM 136=========================== 137 138There are four main build optimizations that our CMake build system supports. 139When performing a bootstrap build it is not beneficial to do anything other than 140setting *CMAKE_BUILD_TYPE* to ``Release`` for the stage-1 compiler. This is 141because the more intensive optimizations are expensive to perform and the 142stage-1 compiler is thrown away. All of the further options described should be 143set on the stage-2 compiler either using a CMake cache file, or by prefixing the 144option with *BOOTSTRAP_*. 145 146The first and simplest to use is the compiler optimization level by setting the 147*CMAKE_BUILD_TYPE* option. The main values of interest are ``Release`` or 148``RelWithDebInfo``. By default the ``Release`` option uses the ``-O3`` 149optimization level, and ``RelWithDebInfo`` uses ``-O2``. If you want to generate 150debug information and use ``-O3`` you can override the 151*CMAKE_<LANG>_FLAGS_RELWITHDEBINFO* option for C and CXX. 152DistributionExample.cmake does this. 153 154Another easy to use option is Link-Time-Optimization. You can set the 155*LLVM_ENABLE_LTO* option on your stage-2 build to ``Thin`` or ``Full`` to enable 156building LLVM with LTO. These options will significantly increase link time of 157the binaries in the distribution, but it will create much faster binaries. This 158option should not be used if your distribution includes static archives, as the 159objects inside the archive will be LLVM bitcode, which is not portable. 160 161The :doc:`AdvancedBuilds` documentation describes the built-in tooling for 162generating LLVM profiling information to drive Profile-Guided-Optimization. The 163in-tree profiling tests are very limited, and generating the profile takes a 164significant amount of time, but it can result in a significant improvement in 165the performance of the generated binaries. 166 167In addition to PGO profiling we also have limited support in-tree for generating 168linker order files. These files provide the linker with a suggested ordering for 169functions in the final binary layout. This can measurably speed up clang by 170physically grouping functions that are called temporally close to each other. 171The current tooling is only available on Darwin systems with ``dtrace(1)``. It 172is worth noting that dtrace is non-deterministic, and so the order file 173generation using dtrace is also non-deterministic. 174 175Options for Reducing Size 176========================= 177 178.. warning:: 179 Any steps taken to reduce the binary size will come at a cost of runtime 180 performance in the generated binaries. 181 182The simplest and least significant way to reduce binary size is to set the 183*CMAKE_BUILD_TYPE* variable to ``MinSizeRel``, which will set the compiler 184optimization level to ``-Os`` which optimizes for binary size. This will have 185both the least benefit to size and the least impact on performance. 186 187The most impactful way to reduce binary size is to dynamically link LLVM into 188all the tools. This reduces code size by decreasing duplication of common code 189between the LLVM-based tools. This can be done by setting the following two 190CMake options to ``On``: *LLVM_BUILD_LLVM_DYLIB* and *LLVM_LINK_LLVM_DYLIB*. 191 192.. warning:: 193 Distributions should never be built using the *BUILD_SHARED_LIBS* CMake 194 option. (:ref:`See the warning above for more explanation <shared_libs>`.). 195 196Relevant CMake Options 197====================== 198 199This section provides documentation of the CMake options that are intended to 200help construct distributions. This is not an exhaustive list, and many 201additional options are documented in the :doc:`CMake` page. Some key options 202that are already documented include: *LLVM_TARGETS_TO_BUILD*, 203*LLVM_ENABLE_PROJECTS*, *LLVM_BUILD_LLVM_DYLIB*, and *LLVM_LINK_LLVM_DYLIB*. 204 205**LLVM_ENABLE_RUNTIMES**:STRING 206 When building a distribution that includes LLVM runtime projects (i.e. libcxx, 207 compiler-rt, libcxxabi, libunwind...), it is important to build those projects 208 with the just-built compiler. 209 210**LLVM_DISTRIBUTION_COMPONENTS**:STRING 211 This variable can be set to a semi-colon separated list of LLVM build system 212 components to install. All LLVM-based tools are components, as well as most 213 of the libraries and runtimes. Component names match the names of the build 214 system targets. 215 216**LLVM_DISTRIBUTIONS**:STRING 217 This variable can be set to a semi-colon separated list of distributions. See 218 the :ref:`Multi-distribution configurations` section above for details on this 219 and other CMake variables to configure multiple distributions. 220 221**LLVM_RUNTIME_DISTRIBUTION_COMPONENTS**:STRING 222 This variable can be set to a semi-colon separated list of runtime library 223 components. This is used in conjunction with *LLVM_ENABLE_RUNTIMES* to specify 224 components of runtime libraries that you want to include in your distribution. 225 Just like with *LLVM_DISTRIBUTION_COMPONENTS*, component names match the names 226 of the build system targets. 227 228**LLVM_DYLIB_COMPONENTS**:STRING 229 This variable can be set to a semi-colon separated name of LLVM library 230 components. LLVM library components are either library names with the LLVM 231 prefix removed (i.e. Support, Demangle...), LLVM target names, or special 232 purpose component names. The special purpose component names are: 233 234 #. ``all`` - All LLVM available component libraries 235 #. ``Native`` - The LLVM target for the Native system 236 #. ``AllTargetsAsmParsers`` - All the included target ASM parsers libraries 237 #. ``AllTargetsDescs`` - All the included target descriptions libraries 238 #. ``AllTargetsDisassemblers`` - All the included target dissassemblers libraries 239 #. ``AllTargetsInfos`` - All the included target info libraries 240 241**LLVM_INSTALL_TOOLCHAIN_ONLY**:BOOL 242 This option defaults to ``Off``: when set to ``On`` it removes many of the 243 LLVM development and testing tools as well as component libraries from the 244 default ``install`` target. Including the development tools is not recommended 245 for distributions as many of the LLVM tools are only intended for development 246 and testing use. 247