Lines Matching +full:llvm +full:- +full:build
2 How To Add Your Build Configuration To LLVM Buildbot Infrastructure
8 This document contains information about adding a build configuration and
9 buildbot worker to the LLVM Buildbot Infrastructure.
22 * The main buildmaster at `<https://lab.llvm.org/buildbot>`_. All builders
24 the build.
25 * The staging buildmaster at `<https://lab.llvm.org/staging>`_. All builders
26 attached to this machine will be completely silent by default when the build
28 commits from the llvm-zorg repository.
51 maintains the build configuration. Generally, operational responsibility lies
52 with the "config owner". We do expect "resource owners" - who are generally
53 the contact listed in a workers attributes - to proxy requests to the relevant
59 Steps To Add Builder To LLVM Buildbot
61 Volunteers can provide their build machines to work as build workers to
62 public LLVM Buildbot.
66 #. Check the existing build configurations to make sure the one you are
71 #. The computer you will be registering with the LLVM buildbot
73 build your configuration successfully. Please check what degree
74 of parallelism (-j param) would give the fastest build. You can build
77 #. Install buildbot-worker (currently we are using buildbot version 2.8.4).
79 as ``pip3 install buildbot-worker==2.8.4``.
81 #. Create a designated user account, your buildbot-worker will be running under,
84 #. Choose the buildbot-worker root directory (all builds will be placed under
85 it), buildbot-worker access name and password the build master will be using
86 to authenticate your buildbot-worker.
88 #. Create a buildbot-worker in context of that buildbot-worker account. Point it
89 to the **lab.llvm.org** port **9994** (see `Buildbot documentation,
91 <http://docs.buildbot.net/current/tutorial/firstrun.html#creating-a-worker>`_
94 .. code-block:: bash
96 $ buildbot-worker create-worker <buildbot-worker-root-directory> \
97 lab.llvm.org:9994 \
98 <buildbot-worker-access-name> \
99 <buildbot-worker-access-password>
107 .. code-block:: bash
109 $ buildbot-worker start <buildbot-worker-root-directory>
115 ``<buildbot-worker-root-directory>/worker/twistd.log``. If your settings
120 #. Fill the buildbot-worker description and admin name/e-mail. Here is an
121 example of the buildbot-worker description::
126 g++.exe (TDM-1 mingw32) 4.4.0
129 Microsoft(R) 32-bit C/C++ Optimizing Compiler Version 16.00.40219.01 for 80x86
134 #. Send a patch which adds your build worker and your builder to
135 `zorg <https://github.com/llvm/llvm-zorg>`_. Use the typical LLVM
136 `workflow <https://llvm.org/docs/Contributing.html#how-to-submit-a-patch>`_.
145 configuration. This causes the builder to build each commit individually
146 and not merge build requests. To maximize quality of feedback to developers,
149 exhausted to improve build times such that the builder can keep up with
153 notifications on build failure; for this you'll need to add an
158 #. Send the buildbot-worker access name and the access password directly to
163 #. Make sure you can start the buildbot-worker and successfully connect
164 to the silent buildmaster. Then set up your buildbot-worker to start
168 #. Check the status of your buildbot-worker on the `Waterfall Display (Staging)
169 <http://lab.llvm.org/staging/#/waterfall>`_ to make sure it is
171 <http://lab.llvm.org/staging/#/workers>`_ to see if administrator
176 up with the build queue. No notifications will be sent, so you can
191 It is possible to test a builder running against a local version of LLVM's
203 * Within a checkout of `llvm-zorg <https://github.com/llvm/llvm-zorg>`_,
208 .. code-block:: bash
210 python -m venv bbenv
212 pip install buildbot{,-console-view,-grid-view,-waterfall-view,-worker,-www}==3.11.7 urllib3
215 ``llvm-zorg`` and ask ``buildbot`` to check the configuration. This step can
218 .. code-block:: bash
220 buildbot create-master llvm-testbbmaster
221 cd llvm-testbbmaster
222 ln -s /path/to/checkout/of/llvm-zorg/buildbot/osuosl/master/master.cfg .
223 ln -s /path/to/checkout/of/llvm-zorg/buildbot/osuosl/master/config/ .
224 ln -s /path/to/checkout/of/llvm-zorg/zorg/ .
229 .. code-block:: bash
231 BUILDBOT_TEST=1 buildbot start --nodaemon .
239 name for the worker associated with the build configuration you want to test
242 .. code-block:: bash
244 buildbot-worker create-worker <buildbot-worker-root-directory> \
246 <buildbot-worker-name> \
248 buildbot-worker start --nodaemon <buildbot-worker-root-directory>
250 * Either wait until the poller sets off a build, or alternatively force a
251 build to start in the web UI.
253 * Review the progress and results of the build in the web UI.
265 .. code-block:: bash
267 ssh -N -L 8011:localhost:8011 -L 9990:localhost:9990 username@buildmaster_server_address
274 builders which can build every commit as they come in. This section
287 complete ~10-15 builds an hour.
290 At 10-15 builds per hour, we need to complete a new build on average every
291 4 to 6 minutes. For anything except the fastest of hardware/build configs,
293 terms, we likely going to need multiple workers to build requests in parallel
295 numbers, if your build config takes e.g. 30 minutes, you will need something
296 on the order of 5-8 workers. If your build config takes ~2 hours, you'll
297 need something on the order of 20-30 workers. The rest of this section
300 Restrict what you build and test
301 Think hard about why you're setting up a bot, and restrict your build
305 of the configuration. (e.g. For a multi-stage clang builder, you probably
310 if you want to both a) confirm that all of LLVM builds with your host
311 compiler, and b) want to do a multi-stage clang build on your target, you
317 In general, we recommend Release build types with Assertions enabled. This
318 generally provides a good balance between build times and bug detection for
320 `-gmlt`), but in general the balance between debug info quality and build
324 Ninja really does help build times over Make, particularly for highly
326 during linking significantly. With a build machine with sufficient
327 parallelism, link times tend to dominate critical path of the build, and are
331 Using ccache materially improves average build times. Incremental builds
332 can be slightly faster, but introduce the risk of build corruption due to
337 One of the non-obvious benefits of using ccache is that it makes the
339 If a change triggers a build request, but doesn't change the build output
340 (e.g. doc changes, python utility changes, etc..), the build will entirely
341 hit in cache and the build request will complete in just the testing time.
345 well, and that having local per-worker caches gets most of the benefit
349 the cache with reasonable access times - i.e. a fast disk, or enough memory
355 As a last resort, you can configure your builder to batch build requests.
356 This makes the build failure notifications markedly less actionable, and