1======= 2ThinLTO 3======= 4 5.. contents:: 6 :local: 7 8Introduction 9============ 10 11*ThinLTO* compilation is a new type of LTO that is both scalable and 12incremental. *LTO* (Link Time Optimization) achieves better 13runtime performance through whole-program analysis and cross-module 14optimization. However, monolithic LTO implements this by merging all 15input into a single module, which is not scalable 16in time or memory, and also prevents fast incremental compiles. 17 18In ThinLTO mode, as with regular LTO, clang emits LLVM bitcode after the 19compile phase. The ThinLTO bitcode is augmented with a compact summary 20of the module. During the link step, only the summaries are read and 21merged into a combined summary index, which includes an index of function 22locations for later cross-module function importing. Fast and efficient 23whole-program analysis is then performed on the combined summary index. 24 25However, all transformations, including function importing, occur 26later when the modules are optimized in fully parallel backends. 27By default, linkers_ that support ThinLTO are set up to launch 28the ThinLTO backends in threads. So the usage model is not affected 29as the distinction between the fast serial thin link step and the backends 30is transparent to the user. 31 32For more information on the ThinLTO design and current performance, 33see the LLVM blog post `ThinLTO: Scalable and Incremental LTO 34<http://blog.llvm.org/2016/06/thinlto-scalable-and-incremental-lto.html>`_. 35While tuning is still in progress, results in the blog post show that 36ThinLTO already performs well compared to LTO, in many cases matching 37the performance improvement. 38 39Current Status 40============== 41 42Clang/LLVM 43---------- 44.. _compiler: 45 46The 3.9 release of clang includes ThinLTO support. However, ThinLTO 47is under active development, and new features, improvements and bugfixes 48are being added for the next release. For the latest ThinLTO support, 49`build a recent version of clang and LLVM 50<https://llvm.org/docs/CMake.html>`_. 51 52Linkers 53------- 54.. _linkers: 55.. _linker: 56 57ThinLTO is currently supported for the following linkers: 58 59- **gold (via the gold-plugin)**: 60 Similar to monolithic LTO, this requires using 61 a `gold linker configured with plugins enabled 62 <https://llvm.org/docs/GoldPlugin.html>`_. 63- **ld64**: 64 Starting with `Xcode 8 <https://developer.apple.com/xcode/>`_. 65- **lld**: 66 Starting with r284050 for ELF, r298942 for COFF. 67 68Usage 69===== 70 71Basic 72----- 73 74To utilize ThinLTO, simply add the -flto=thin option to compile and link. E.g. 75 76.. code-block:: console 77 78 % clang -flto=thin -O2 file1.c file2.c -c 79 % clang -flto=thin -O2 file1.o file2.o -o a.out 80 81When using lld-link, the -flto option need only be added to the compile step: 82 83.. code-block:: console 84 85 % clang-cl -flto=thin -O2 -c file1.c file2.c 86 % lld-link /out:a.exe file1.obj file2.obj 87 88As mentioned earlier, by default the linkers will launch the ThinLTO backend 89threads in parallel, passing the resulting native object files back to the 90linker for the final native link. As such, the usage model is the same as 91non-LTO. 92 93With gold, if you see an error during the link of the form: 94 95.. code-block:: console 96 97 /usr/bin/ld: error: /path/to/clang/bin/../lib/LLVMgold.so: could not load plugin library: /path/to/clang/bin/../lib/LLVMgold.so: cannot open shared object file: No such file or directory 98 99Then either gold was not configured with plugins enabled, or clang 100was not built with ``-DLLVM_BINUTILS_INCDIR`` set properly. See 101the instructions for the 102`LLVM gold plugin <https://llvm.org/docs/GoldPlugin.html#how-to-build-it>`_. 103 104Controlling Backend Parallelism 105------------------------------- 106.. _parallelism: 107 108By default, the ThinLTO link step will launch as many 109threads in parallel as there are cores. If the number of 110cores can't be computed for the architecture, then it will launch 111``std::thread::hardware_concurrency`` number of threads in parallel. 112For machines with hyper-threading, this is the total number of 113virtual cores. For some applications and machine configurations this 114may be too aggressive, in which case the amount of parallelism can 115be reduced to ``N`` via: 116 117- gold: 118 ``-Wl,-plugin-opt,jobs=N`` 119- ld64: 120 ``-Wl,-mllvm,-threads=N`` 121- ld.lld, ld64.lld: 122 ``-Wl,--thinlto-jobs=N`` 123- lld-link: 124 ``/opt:lldltojobs=N`` 125 126Other possible values for ``N`` are: 127 128- 0: 129 Use one thread per physical core (default) 130- 1: 131 Use a single thread only (disable multi-threading) 132- all: 133 Use one thread per logical core (uses all hyper-threads) 134 135Incremental 136----------- 137.. _incremental: 138 139ThinLTO supports fast incremental builds through the use of a cache, 140which currently must be enabled through a linker option. 141 142- gold (as of LLVM 4.0): 143 ``-Wl,-plugin-opt,cache-dir=/path/to/cache`` 144- ld64 (supported since clang 3.9 and Xcode 8) and Mach-O ld64.lld (as of LLVM 145 15.0): 146 ``-Wl,-cache_path_lto,/path/to/cache`` 147- ELF ld.lld (as of LLVM 5.0): 148 ``-Wl,--thinlto-cache-dir=/path/to/cache`` 149- COFF lld-link (as of LLVM 6.0): 150 ``/lldltocache:/path/to/cache`` 151 152Cache Pruning 153------------- 154 155To help keep the size of the cache under control, ThinLTO supports cache 156pruning. Cache pruning is supported with gold, ld64, and lld, but currently only 157gold and lld allow you to control the policy with a policy string. The cache 158policy must be specified with a linker option. 159 160- gold (as of LLVM 6.0): 161 ``-Wl,-plugin-opt,cache-policy=POLICY`` 162- ELF ld.lld (as of LLVM 5.0), Mach-O ld64.lld (as of LLVM 15.0): 163 ``-Wl,--thinlto-cache-policy=POLICY`` 164- COFF lld-link (as of LLVM 6.0): 165 ``/lldltocachepolicy:POLICY`` 166 167A policy string is a series of key-value pairs separated by ``:`` characters. 168Possible key-value pairs are: 169 170- ``cache_size=X%``: The maximum size for the cache directory is ``X`` percent 171 of the available space on the disk. Set to 100 to indicate no limit, 172 50 to indicate that the cache size will not be left over half the available 173 disk space. A value over 100 is invalid. A value of 0 disables the percentage 174 size-based pruning. The default is 75%. 175 176- ``cache_size_bytes=X``, ``cache_size_bytes=Xk``, ``cache_size_bytes=Xm``, 177 ``cache_size_bytes=Xg``: 178 Sets the maximum size for the cache directory to ``X`` bytes (or KB, MB, 179 GB respectively). A value over the amount of available space on the disk 180 will be reduced to the amount of available space. A value of 0 disables 181 the byte size-based pruning. The default is no byte size-based pruning. 182 183 Note that ThinLTO will apply both size-based pruning policies simultaneously, 184 and changing one does not affect the other. For example, a policy of 185 ``cache_size_bytes=1g`` on its own will cause both the 1GB and default 75% 186 policies to be applied unless the default ``cache_size`` is overridden. 187 188- ``cache_size_files=X``: 189 Set the maximum number of files in the cache directory. Set to 0 to indicate 190 no limit. The default is 1000000 files. 191 192- ``prune_after=Xs``, ``prune_after=Xm``, ``prune_after=Xh``: Sets the 193 expiration time for cache files to ``X`` seconds (or minutes, hours 194 respectively). When a file hasn't been accessed for ``prune_after`` seconds, 195 it is removed from the cache. A value of 0 disables the expiration-based 196 pruning. The default is 1 week. 197 198- ``prune_interval=Xs``, ``prune_interval=Xm``, ``prune_interval=Xh``: 199 Sets the pruning interval to ``X`` seconds (or minutes, hours 200 respectively). This is intended to be used to avoid scanning the directory 201 too often. It does not impact the decision of which files to prune. A 202 value of 0 forces the scan to occur. The default is every 20 minutes. 203 204Clang Bootstrap 205--------------- 206 207To `bootstrap clang/LLVM <https://llvm.org/docs/AdvancedBuilds.html#bootstrap-builds>`_ 208with ThinLTO, follow these steps: 209 2101. The host compiler_ must be a version of clang that supports ThinLTO. 211#. The host linker_ must support ThinLTO (and in the case of gold, must be 212 `configured with plugins enabled <https://llvm.org/docs/GoldPlugin.html>`_). 213#. Use the following additional `CMake variables 214 <https://llvm.org/docs/CMake.html#options-and-variables>`_ 215 when configuring the bootstrap compiler build: 216 217 * ``-DLLVM_ENABLE_LTO=Thin`` 218 * ``-DCMAKE_C_COMPILER=/path/to/host/clang`` 219 * ``-DCMAKE_CXX_COMPILER=/path/to/host/clang++`` 220 * ``-DCMAKE_RANLIB=/path/to/host/llvm-ranlib`` 221 * ``-DCMAKE_AR=/path/to/host/llvm-ar`` 222 223 Or, on Windows: 224 225 * ``-DLLVM_ENABLE_LTO=Thin`` 226 * ``-DCMAKE_C_COMPILER=/path/to/host/clang-cl.exe`` 227 * ``-DCMAKE_CXX_COMPILER=/path/to/host/clang-cl.exe`` 228 * ``-DCMAKE_LINKER=/path/to/host/lld-link.exe`` 229 * ``-DCMAKE_RANLIB=/path/to/host/llvm-ranlib.exe`` 230 * ``-DCMAKE_AR=/path/to/host/llvm-ar.exe`` 231 232#. To use additional linker arguments for controlling the backend 233 parallelism_ or enabling incremental_ builds of the bootstrap compiler, 234 after configuring the build, modify the resulting CMakeCache.txt file in the 235 build directory. Specify any additional linker options after 236 ``CMAKE_EXE_LINKER_FLAGS:STRING=``. Note the configure may fail if 237 linker plugin options are instead specified directly in the previous step. 238 239The ``BOOTSTRAP_LLVM_ENABLE_LTO=Thin`` will enable ThinLTO for stage 2 and 240stage 3 in case the compiler used for stage 1 does not support the ThinLTO 241option. 242 243More Information 244================ 245 246* From LLVM project blog: 247 `ThinLTO: Scalable and Incremental LTO 248 <http://blog.llvm.org/2016/06/thinlto-scalable-and-incremental-lto.html>`_ 249