1================================= 2How To Release LLVM To The Public 3================================= 4 5Introduction 6============ 7 8This document contains information about successfully releasing LLVM --- 9including sub-projects: e.g., ``clang`` and ``compiler-rt`` --- to the public. 10It is the Release Manager's responsibility to ensure that a high quality build 11of LLVM is released. 12 13If you're looking for the document on how to test the release candidates and 14create the binary packages, please refer to the :doc:`ReleaseProcess` instead. 15 16.. _timeline: 17 18Release Timeline 19================ 20 21LLVM is released on a time based schedule --- with major releases roughly 22every 6 months. In between major releases there may be dot releases. 23The release manager will determine if and when to make a dot release based 24on feedback from the community. Typically, dot releases should be made if 25there are large number of bug-fixes in the stable branch or a critical bug 26has been discovered that affects a large number of users. 27 28Unless otherwise stated, dot releases will follow the same procedure as 29major releases. 30 31Annual Release Schedule 32----------------------- 33 34Here is the annual release schedule for LLVM. This is meant to be a 35guide, and release managers are not required to follow this exactly. 36Releases should be tagged on Tuesdays. 37 38=============================== ========================= 39Release Approx. Date 40=============================== ========================= 41*release branch: even releases* *4th Tue in January* 42*release branch: odd releases* *4th Tue in July* 43X.1.0-rc1 3 days after branch. 44X.1.0-rc2 2 weeks after branch. 45X.1.0-rc3 4 weeks after branch 46**X.1.0-final** **6 weeks after branch** 47**X.1.1** **8 weeks after branch** 48**X.1.2** **10 weeks after branch** 49**X.1.3** **12 weeks after branch** 50**X.1.4** **14 weeks after branch** 51**X.1.5** **16 weeks after branch** 52**X.1.6 (if necessary)** **18 weeks after branch** 53=============================== ========================= 54 55Release Process Summary 56----------------------- 57 58* Announce release schedule to the LLVM community and update the website. Do 59 this at least 3 weeks before the -rc1 release. 60 61* Create release branch and begin release process. 62 63* Send out release candidate sources for first round of testing. Testing lasts 64 6 weeks. During the first round of testing, any regressions found should be 65 fixed. Patches are merged from mainline into the release branch. Also, all 66 features need to be completed during this time. Any features not completed at 67 the end of the first round of testing will be removed or disabled for the 68 release. 69 70* Generate and send out the second release candidate sources. Only *critical* 71 bugs found during this testing phase will be fixed. Any bugs introduced by 72 merged patches will be fixed. If so a third round of testing is needed. 73 74* The release notes are updated. 75 76* Finally, release! 77 78* Announce bug fix release schedule to the LLVM community and update the website. 79 80* Do bug-fix releases every two weeks until X.1.5 or X.1.6 (if necessary). 81 82Release Process 83=============== 84 85.. contents:: 86 :local: 87 88Release Administrative Tasks 89---------------------------- 90 91This section describes a few administrative tasks that need to be done for the 92release process to begin. Specifically, it involves: 93 94* Updating version numbers, 95 96* Creating the release branch, and 97 98* Tagging release candidates for the release team to begin testing. 99 100Create Release Branch 101^^^^^^^^^^^^^^^^^^^^^ 102 103Branch the Git trunk using the following procedure: 104 105#. Remind developers that the release branching is imminent and to refrain from 106 committing patches that might break the build. E.g., new features, large 107 patches for works in progress, an overhaul of the type system, an exciting 108 new TableGen feature, etc. 109 110#. Verify that the current git trunk is in decent shape by 111 examining nightly tester and buildbot results. 112 113#. Bump the version in trunk to N.0.0git and tag the commit with llvmorg-N-init. 114 If ``X`` is the version to be released, then ``N`` is ``X + 1``. 115 116:: 117 118 $ git tag -sa llvmorg-N-init 119 120#. Clear the release notes in trunk. 121 122#. Create the release branch from the last known good revision from before the 123 version bump. The branch's name is release/X.x where ``X`` is the major version 124 number and ``x`` is just the letter ``x``. 125 126#. On the newly-created release branch, immediately bump the version 127 to X.1.0git (where ``X`` is the major version of the branch.) 128 129#. All tags and branches need to be created in both the llvm/llvm-project and 130 llvm/llvm-test-suite repos. 131 132Update LLVM Version 133^^^^^^^^^^^^^^^^^^^ 134 135After creating the LLVM release branch, update the release branches' 136version with the script in ``llvm/utils/release/bump-version.py``. 137 138Tagging the LLVM Release Candidates 139^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 140 141Tag release candidates: 142 143:: 144 145 $ git tag -sa llvmorg-X.Y.Z-rcN 146 147The pre-packaged source tarballs will be automatically generated via the 148"Release Sources" workflow on GitHub. This workflow will create an artifact 149containing all the release tarballs and the artifact attestation. The 150Release Manager should download the artifact, verify the tarballs, sign them, 151and then upload them to the release page. 152 153:: 154 155 $ unzip artifact.zip 156 $ gh auth login 157 $ for f in *.xz; do gh attestation verify --owner llvm $f && gpg -b $f; done 158 159Tarballs, release binaries, or any other release artifacts must be uploaded to 160GitHub. This can be done using the github-upload-release.py script in utils/release. 161 162:: 163 164 $ github-upload-release.py upload --token <github-token> --release X.Y.Z-rcN --files <release_files> 165 166 167Build The Binary Distribution 168^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 169 170Creating the binary distribution requires following the instructions 171:doc:`here <ReleaseProcess>`. 172 173That process will perform both Release+Asserts and Release builds but only 174pack the Release build for upload. You should use the Release+Asserts sysroot, 175normally under ``final/Phase3/Release+Asserts/llvmCore-3.8.1-RCn.install/``, 176for test-suite and run-time benchmarks, to make sure nothing serious has 177passed through the net. For compile-time benchmarks, use the Release version. 178 179The minimum required version of the tools you'll need are :doc:`here <GettingStarted>` 180 181Release Qualification Criteria 182------------------------------ 183 184There are no official release qualification criteria. It is up to the 185the release manager to determine when a release is ready. The release manager 186should pay attention to the results of community testing, the number of outstanding 187bugs, and then number of regressions when determining whether or not to make a 188release. 189 190The community values time based releases, so releases should not be delayed for 191too long unless there are critical issues remaining. In most cases, the only 192kind of bugs that are critical enough to block a release would be a major regression 193from a previous release. 194 195Official Testing 196---------------- 197 198A few developers in the community have dedicated time to validate the release 199candidates and volunteered to be the official release testers for each 200architecture. 201 202These will be the ones testing, generating and uploading the official binaries 203to the server, and will be the minimum tests *necessary* for the release to 204proceed. 205 206This will obviously not cover all OSs and distributions, so additional community 207validation is important. However, if community input is not reached before the 208release is out, all bugs reported will have to go on the next stable release. 209 210The official release managers are: 211 212* Even releases: Tom Stellard (tstellar@redhat.com) 213* Odd releases: Tobias Hieta (tobias@hieta.se) 214 215The official release testers are volunteered from the community and have 216consistently validated and released binaries for their targets/OSs. To contact 217them, you should post on the `Discourse forums (Project 218Infrastructure - Release Testers). <https://discourse.llvm.org/c/infrastructure/release-testers/66>`_ 219 220The official testers list is in the file ``RELEASE_TESTERS.TXT``, in the ``LLVM`` 221repository. 222 223Community Testing 224----------------- 225 226Once all testing has been completed and appropriate bugs filed, the release 227candidate tarballs are put on the website and the LLVM community is notified. 228 229We ask that all LLVM developers test the release in any the following ways: 230 231#. Download ``llvm-X.Y``, ``llvm-test-X.Y``, and the appropriate ``clang`` 232 binary. Build LLVM. Run ``make check`` and the full LLVM test suite (``make 233 TEST=nightly report``). 234 235#. Download ``llvm-X.Y``, ``llvm-test-X.Y``, and the ``clang`` sources. Compile 236 everything. Run ``make check`` and the full LLVM test suite (``make 237 TEST=nightly report``). 238 239#. Download ``llvm-X.Y``, ``llvm-test-X.Y``, and the appropriate ``clang`` 240 binary. Build whole programs with it (ex. Chromium, Firefox, Apache) for 241 your platform. 242 243#. Download ``llvm-X.Y``, ``llvm-test-X.Y``, and the appropriate ``clang`` 244 binary. Build *your* programs with it and check for conformance and 245 performance regressions. 246 247#. Run the :doc:`release process <ReleaseProcess>`, if your platform is 248 *different* than that which is officially supported, and report back errors 249 only if they were not reported by the official release tester for that 250 architecture. 251 252We also ask that the OS distribution release managers test their packages with 253the first candidate of every release, and report any *new* errors in GitHub. 254If the bug can be reproduced with an unpatched upstream version of the release 255candidate (as opposed to the distribution's own build), the priority should be 256release blocker. 257 258During the first round of testing, all regressions must be fixed before the 259second release candidate is tagged. 260 261In the subsequent stages, the testing is only to ensure that bug 262fixes previously merged in have not created new major problems. *This is not 263the time to solve additional and unrelated bugs!* If no patches are merged in, 264the release is determined to be ready and the release manager may move onto the 265next stage. 266 267Reporting Regressions 268--------------------- 269 270Every regression that is found during the tests (as per the criteria above), 271should be filled in a bug in GitHub and added to the release milestone. 272 273If a bug can't be reproduced, or stops being a blocker, it should be removed 274from the Milestone. Debugging can continue, but on trunk. 275 276Backport Requests 277----------------- 278 279Instructions for requesting a backport to a stable branch can be found :doc:`here <GitHub>`. 280 281Triaging Bug Reports for Releases 282--------------------------------- 283 284This section describes how to triage bug reports: 285 286#. Search for bugs with a Release Milestone that have not been added to the 287 "Release Status" github project: 288 289 https://github.com/llvm/llvm-project/issues?q=is%3Aissue+milestone%3A%22LLVM+14.0.5+Release%22+no%3Aproject+ 290 291 Replace 14.0.5 in this query with the version from the Release Milestone being 292 targeted. 293 294 Add these bugs to the "Release Status" project. 295 296#. Navigate to the `Release Status project <https://github.com/orgs/llvm/projects/3>`_ 297 to see the list of bugs that are being considered for the release. 298 299#. Review each bug and first check if it has been fixed in main. If it has, update 300 its status to "Needs Pull Request", and create a pull request for the fix 301 using the /cherry-pick or /branch comments if this has not been done already. 302 303#. If a bug has been fixed and has a pull request created for backporting it, 304 then update its status to "Needs Review" and notify a knowledgeable reviewer. 305 Usually you will want to notify the person who approved the patch in Phabricator, 306 but you may use your best judgement on who a good reviewer would be. Once 307 you have identified the reviewer(s), assign the issue to them and mention 308 them (i.e @username) in a comment and ask them if the patch is safe to backport. 309 You should also review the bug yourself to ensure that it meets the requirements 310 for committing to the release branch. 311 312#. Once a bug has been reviewed, add the release:reviewed label and update the 313 issue's status to "Needs Merge". Check the pull request associated with the 314 issue. If all the tests pass, then the pull request can be merged. If not, 315 then add a comment on the issue asking someone to take a look at the failures. 316 317#. Once the pull request has been merged push it to the official release branch 318 with the script ``llvm/utils/git/sync-release-repo.sh``. 319 320 Then add a comment to the issue stating that the fix has been merged along with 321 the git hashes from the release branch. Add the release:merged label to the issue 322 and close it. 323 324 325Release Patch Rules 326------------------- 327 328Below are the rules regarding patching the release branch: 329 330#. Patches applied to the release branch may only be applied by the release 331 manager, the official release testers or the maintainers with approval from 332 the release manager. 333 334#. Release managers are encouraged, but not required, to get approval from a 335 maintainer before approving patches. If there are no reachable maintainers 336 then release managers can ask approval from patch reviewers or other 337 developers active in that area. 338 339#. *Before RC1* Patches should be limited to bug fixes, important optimization 340 improvements, or completion of features that were started before the branch 341 was created. As with all phases, release managers and maintainers can reject 342 patches that are deemed too invasive. 343 344#. *Before RC2* Patches should be limited to bug fixes or backend specific 345 improvements that are determined to be very safe. 346 347#. *Before RC3/Final Major Release* Patches should be limited to critical 348 bugs or regressions. 349 350#. *Bug fix releases* Patches should be limited to bug fixes or very safe 351 and critical performance improvements. Patches must maintain both API and 352 ABI compatibility with the previous major release. 353 354 355Release Final Tasks 356------------------- 357 358The final stages of the release process involves tagging the "final" release 359branch, updating documentation that refers to the release, and updating the 360demo page. 361 362Update Documentation 363^^^^^^^^^^^^^^^^^^^^ 364 365Review the documentation in the release branch and ensure that it is up 366to date. The "Release Notes" must be updated to reflect new features, bug 367fixes, new known issues, and changes in the list of supported platforms. 368The "Getting Started Guide" should be updated to reflect the new release 369version number tag available from Subversion and changes in basic system 370requirements. 371 372.. _tag: 373 374Tag the LLVM Final Release 375^^^^^^^^^^^^^^^^^^^^^^^^^^ 376 377Tag the final release sources: 378 379:: 380 381 $ git tag -sa llvmorg-X.Y.Z 382 $ git push https://github.com/llvm/llvm-project.git llvmorg-X.Y.Z 383 384Update the LLVM Website 385^^^^^^^^^^^^^^^^^^^^^^^ 386 387The website must be updated before the release announcement is sent out. Here 388is what to do: 389 390#. Check out the ``www-releases`` module from GitHub. 391 392#. Create a new sub-directory ``X.Y.Z`` in the releases directory. 393 394#. Copy and commit the ``llvm/docs`` and ``LICENSE.txt`` files into this new 395 directory. 396 397#. Update the ``releases/download.html`` file with links to the release 398 binaries on GitHub. 399 400#. Update the ``releases/index.html`` with the new release and link to release 401 documentation. 402 403#. After you push the changes to the www-releases repo, someone with admin 404 access must login to prereleases-origin.llvm.org and manually pull the new 405 changes into /data/www-releases/. This is where the website is served from. 406 407#. Finally checkout the llvm-www repo and update the main page 408 (``index.html`` and sidebar) to point to the new release and release 409 announcement. 410 411Announce the Release 412^^^^^^^^^^^^^^^^^^^^ 413 414Create a new post in the `Announce Category <https://discourse.llvm.org/c/announce>`_ 415once all the release tasks are complete. For X.1.0 releases, make sure to include a 416link to the release notes in the post. For X.1.1+ releases, generate a changelog 417using this command and add it to the post. 418 419:: 420 421 $ git log --format="- %aN: [%s (%h)](https://github.com/llvm/llvm-project/commit/%H)" llvmorg-X.1.N-1..llvmorg-X.1.N 422 423Once the release has been announced add a link to the announcement on the llvm 424homepage (from the llvm-www repo) in the "Release Emails" section. 425