Lines Matching +full:docs +full:- +full:polly +full:- +full:html
9 the GitHub migration `status page <https://llvm.org/GitHubMigrationStatus.html>`_
22 infrastructure) will continue to work with a Git-based LLVM.
29 This proposal relates only to moving the hosting of our source-code repository
31 using GitHub's issue tracker, pull-requests, or code-review.
35 username/password-hash.
41 ----------------
57 --------
66 through the Git-SVN integration.
82 -----------
85 projects. Any of these could replace the code-hosting infrastructure that we
92 BitBucket: it offers read-write **SVN** access to the repository
93 (https://github.com/blog/626-announcing-svn-support).
94 This would enable people to continue working post-migration as though our code
101 -------------------------------------
103 The current SVN repository hosts all the LLVM sub-projects alongside each other.
105 all LLVM sub-projects.
113 - "The 'branch' I most care about is mainline, and losing the ability to say
116 - "I like those results sorted by time and the chronology should be obvious, but
119 - "There is still the major regression with unreadable version numbers.
121 non-trivial issue." [JSonnRevNum]_
122 - "Sequential IDs are important for LNT and llvmlab bisection tool." [MatthewsRevNum]_.
125 ``git rev-list --count <commit-hash>``. This identifier is unique only
126 within a single branch, but this means the tuple `(num, branch-name)` uniquely
129 We can thus use this revision number to ensure that e.g. `clang -v` reports a
130 user-friendly revision number (e.g. `main-12345` or `4.0-5321`), addressing
134 -------------------------------
145 We could supply a pre-push hook on the client side that would run and check the
152 -------------------------
161 -------------------------
163 1. Update docs to mention the move, so people are aware of what is going on.
164 2. Set up a read-only version of the GitHub project, mirroring our current SVN
167 umbrella repository update (if the multirepo is selected) or the read-only
168 Git views for the sub-projects (if the monorepo is selected).
171 ------------------
191 --------------------------
194 10. Switch the SVN repository to read-only and allow pushes to the GitHub repository.
199 -------------------
209 ----------------
211 The LLVM git repository hosted at https://github.com/llvm/llvm-project contains all
212 sub-projects in a single source tree. It is often referred to as a monorepo and
213 mimics an export of the current SVN repository, with each sub-project having its
214 own top-level directory. Not all sub-projects are used for building toolchains.
215 For example, www/ and test-suite/ are not part of the monorepo.
217 Putting all sub-projects in a single checkout makes cross-project refactoring
220 * New sub-projects can be trivially split out for better reuse and/or layering
223 * Changing an API in LLVM and upgrading the sub-projects will always be done in
225 * Moving code across sub-project (during refactoring for instance) in a single
227 * Tooling based on `git grep` works natively across sub-projects, allowing to
230 * Having all the sources present encourages maintaining the other sub-projects
234 the sub-projects move synchronously, and a single revision number (or commit
239 Building a single sub-project
243 all sub-projects together. It is trivial to configure builds for a single
244 sub-project.
252 cmake path/to/monorepo -DLLVM_ENABLE_PROJECTS=lld
254 cmake path/to/monorepo -DLLVM_ENABLE_PROJECTS=clang
256 .. _git-svn-mirror:
259 ---------------------
261 Read-only sub-project mirrors
264 With the Monorepo, it is undecided whether the existing single-subproject
265 mirrors (e.g. https://git.llvm.org/git/compiler-rt.git) will continue to
276 ------------------
279 standalone sub-project, particularly on runtimes like libcxx and compiler-rt
283 use the SVN bridge or the single-subproject Git mirrors. However, it's
286 standalone sub-project, even if they aren't contributing to it, due to the
288 sub-project Git mirrors would addresses this.
289 * Preservation of the existing read/write SVN-based workflows relies on the
296 * :ref:`Checkout/Clone a Single Project, without Commit Access <workflow-checkout-commit>`.
297 * :ref:`Checkout/Clone Multiple Projects, with Commit Access <workflow-monocheckout-multicommit>`.
298 * :ref:`Commit an API Change in LLVM and Update the Sub-projects <workflow-cross-repo-commit>`.
299 …* :ref:`Branching/Stashing/Updating for Local Development or Experiments <workflow-mono-branching>…
300 * :ref:`Bisecting <workflow-mono-bisecting>`.
306 how end-users or developers would interact with the repository for
307 various use-cases.
309 .. _workflow-checkout-commit:
312 ---------------------------------------------------
320 svn co https://user@llvm.org/svn/llvm-project/llvm/trunk llvm
321 # or using the read-only Git view, with git-svn
324 git svn init https://llvm.org/svn/llvm-project/llvm/trunk --username=<username>
325 git config svn-remote.svn.fetch :refs/remotes/origin/main
326 git svn rebase -l # -l avoids fetching ahead of the git mirror.
331 .. _workflow-multicheckout-nocommit:
339 git clone https://github.com/llvm/llvm-project.git
341 At this point you have every sub-project (llvm, clang, lld, lldb, ...), which
343 can still build only compiler-rt for instance. In this way it's not different
350 echo /compiler-rt > .git/info/sparse-checkout
351 git read-tree -mu HEAD
353 The data for all sub-projects is still in your `.git` directory, but in your
354 checkout, you only see `compiler-rt`.
355 Before you push, you'll need to fetch and rebase (`git pull --rebase`) as
358 Note that when you fetch you'll likely pull in changes to sub-projects you don't
365 git log origin/main@{1}..origin/main -- libcxx
371 Note that today with SVN or git-svn, this step is not possible since the
375 ----------------------------------------------------
384 svn co https://llvm.org/svn/llvm-project/llvm/trunk llvm -r $REVISION
386 svn co https://llvm.org/svn/llvm-project/clang/trunk clang -r $REVISION
388 svn co https://llvm.org/svn/llvm-project/libcxx/trunk libcxx -r $REVISION
390 Or using git-svn::
394 git svn init https://llvm.org/svn/llvm-project/llvm/trunk --username=<username>
395 git config svn-remote.svn.fetch :refs/remotes/origin/main
396 git svn rebase -l
397 git checkout `git svn find-rev -B r258109`
401 git svn init https://llvm.org/svn/llvm-project/clang/trunk --username=<username>
402 git config svn-remote.svn.fetch :refs/remotes/origin/main
403 git svn rebase -l
404 git checkout `git svn find-rev -B r258109`
408 git svn init https://llvm.org/svn/llvm-project/libcxx/trunk --username=<username>
409 git config svn-remote.svn.fetch :refs/remotes/origin/main
410 git svn rebase -l
411 git checkout `git svn find-rev -B r258109`
413 Note that the list would be longer with more sub-projects.
415 .. _workflow-monocheckout-multicommit:
420 The repository contains natively the source for every sub-projects at the right
423 git clone https://github.com/llvm/llvm-project.git
424 cd llvm-projects
430 .. _workflow-cross-repo-commit:
432 Commit an API Change in LLVM and Update the Sub-projects
433 --------------------------------------------------------
436 subversion users and for git-svn users. For example, few Git users try to update
447 ----------------------------------------------------------------
453 git-svn can do it. Let's look in practice what it means when dealing with
454 multiple sub-projects.
466 git checkout -b MyBranch
468 git checkout -b MyBranch
470 git checkout -b MyBranch
480 .. _workflow-mono-branching:
494 git checkout -b MyBranch
501 ---------
509 sub-projects makes it possible to script around.
511 Using the existing Git read-only view of the repositories, it is possible to use
515 .. _workflow-mono-bisecting:
524 The same example, finding which commit introduces a regression where clang-3.9
525 crashes but not clang-3.8 passes, will look like::
552 ``migrate-downstream-fork.py`` tool at
553 https://github.com/jyknight/llvm-git-migration.
556 ---------------
558 Basic instructions for ``migrate-downstream-fork.py`` are in the
563 mkdir my-monorepo
564 git -C my-monorepo init
567 git -C my-monorepo remote add upstream/monorepo https://github.com/llvm/llvm-project.git
573 clang-tools-extra
574 compiler-rt
575 debuginfo-tests
583 polly )
585 git -C my-monorepo remote add upstream/split/${p} https://github.com/llvm-mirror/${p}.git
586 git -C my-monorepo remote add local/split/${p} https://my.local.mirror.org/${p}.git
590 git -C my-monorepo fetch --all
592 # Run migrate-downstream-fork to rewrite local branches on top of
595 cd my-monorepo
596 migrate-downstream-fork.py \
599 --new-repo-prefix=refs/remotes/upstream/monorepo \
600 --old-repo-prefix=refs/remotes/upstream/split \
601 --source-kind=split \
602 --revmap-out=monorepo-map.txt
605 # Octopus-merge the resulting local split histories to unify them.
612 git -C my-monorepo branch --no-track local/octopus/main \
613 $(git -C my-monorepo merge-base refs/remotes/upstream/monorepo/main \
615 git -C my-monorepo checkout local/octopus/${my_local_branch}
620 git -C my-monorepo branch ${subproject_branch} \
627 git -C my-monorepo merge ${subproject_branches[@]}
631 git -C my-monorepo branch -d ${subproject_branch}
635 for ref in $(git -C my-monorepo for-each-ref --format="%(refname)" \
638 git -C my-monorepo branch upstream/${upstream_branch} ${ref}
643 U1 - U2 - U3 <- upstream/main
645 \ \ - Llld1 - Llld2 -
647 \ - Lclang1 - Lclang2-- Lmerge <- local/octopus/main
649 - Lllvm1 - Lllvm2-----
659 ``local/octopus/<local branch>`` need not use ``git-merge-base`` to
664 ---------------------
675 UM1 ---- UM2 -- UM3 -- UM4 ---- UM5 ---- UM6 ---- UM7 ---- UM8 <- main
685 The tool ``zip-downstream-fork.py`` at
686 https://github.com/greened/llvm-git-migration/tree/zip can be used to
687 convert the umbrella history into a monorepo-based history with
690 U1 - U2 - U3 <- upstream/main
692 \ -----\--------------- local/zip--.
694 - Lllvm1 - Llld1 - UM3 - Lclang1 - Lclang2 - Lllvm2 - Llld2 - Lmyproj1 <-'
709 history and so the edge ``U2 -> Lclang1`` is a visual reminder of what
712 Even so, the edge ``U3 -> Llld1`` could be problematic for future
719 running ``zip-downstream-fork.py``. If downstream merged each project
739 top-level directory in the umbrella, just as they do in the monorepo .
743 Given the above run of ``migrate-downstream-fork.py``, a recipe to
746 # Import any non-LLVM repositories the umbrella references.
747 git -C my-monorepo remote add localrepo \
751 subprojects=( clang clang-tools-extra compiler-rt debuginfo-tests libclc
753 parallel-libs polly pstl )
756 # already done for the ``migrate-downstream-fork.py`` run).
759 https://github.com/llvm-mirror/${subproject}.git
764 # already done for the ``migrate-downstream-fork.py`` run).
772 git -C my-monorepo remote add umbrella \
777 echo "myproj local/myproj" > my-monorepo/submodule-map.txt
781 cd my-monorepo
782 zip-downstream-fork.py \
784 --new-repo-prefix=refs/remotes/upstream/monorepo \
785 --old-repo-prefix=refs/remotes/upstream/split \
786 --revmap-in=monorepo-map.txt \
787 --revmap-out=zip-map.txt \
788 --subdir=local \
789 --submodule-map=submodule-map.txt \
790 --update-tags
794 git -C my-monorepo branch --no-track local/zip/main refs/remotes/umbrella/main
796 Note that if the umbrella has submodules to non-LLVM repositories,
797 ``zip-downstream-fork.py`` needs to know about them to be able to
801 With ``--update-tags`` the tool will migrate annotated tags pointing
806 commit after rewriting, or the ``--update-tags`` option may be
813 ``submodule-map.txt`` is a list of pairs, one per line. The first
826 tools/clang/tools/extra clang-tools-extra
827 projects/compiler-rt compiler-rt
828 projects/debuginfo-tests debuginfo-tests
836 tools/polly polly
846 Because llvm is itself the umbrella, we use --subdir to write its
849 # Import any non-LLVM repositories the umbrella references.
850 git -C my-monorepo remote add localrepo \
854 subprojects=( clang clang-tools-extra compiler-rt debuginfo-tests libclc
856 parallel-libs polly pstl )
859 # already done for the ``migrate-downstream-fork.py`` run).
862 https://github.com/llvm-mirror/${subproject}.git
867 # already done for the ``migrate-downstream-fork.py`` run).
875 # so zip-downstream-fork.py knows what it is.
876 git -C my-monorepo remote add umbrella \
881 echo "tools/clang clang" > my-monorepo/submodule-map.txt
882 echo "tools/clang/tools/extra clang-tools-extra" >> my-monorepo/submodule-map.txt
883 echo "projects/compiler-rt compiler-rt" >> my-monorepo/submodule-map.txt
884 echo "projects/debuginfo-tests debuginfo-tests" >> my-monorepo/submodule-map.txt
885 echo "projects/libclc libclc" >> my-monorepo/submodule-map.txt
886 echo "projects/libcxx libcxx" >> my-monorepo/submodule-map.txt
887 echo "projects/libcxxabi libcxxabi" >> my-monorepo/submodule-map.txt
888 echo "projects/libunwind libunwind" >> my-monorepo/submodule-map.txt
889 echo "tools/lld lld" >> my-monorepo/submodule-map.txt
890 echo "tools/lldb lldb" >> my-monorepo/submodule-map.txt
891 echo "projects/openmp openmp" >> my-monorepo/submodule-map.txt
892 echo "tools/polly polly" >> my-monorepo/submodule-map.txt
893 echo "projects/myproj local/myproj" >> my-monorepo/submodule-map.txt
897 cd my-monorepo
898 zip-downstream-fork.py \
900 --new-repo-prefix=refs/remotes/upstream/monorepo \
901 --old-repo-prefix=refs/remotes/upstream/split \
902 --revmap-in=monorepo-map.txt \
903 --revmap-out=zip-map.txt \
904 --subdir=llvm \
905 --submodule-map=submodule-map.txt \
906 --update-tags
910 git -C my-monorepo branch --no-track local/zip/main refs/remotes/umbrella/main
913 Comments at the top of ``zip-downstream-fork.py`` describe in more
917 ----------------------------
927 an umbrella setup, the ``import-downstream-repo.py`` tool at
928 https://github.com/greened/llvm-git-migration/tree/import can help with
932 git -C my-monorepo remote add myrepo https://my.local.mirror.org/myrepo.git
939 cd my-monorepo
940 import-downstream-repo.py \
943 --new-repo-prefix=refs/remotes/upstream/monorepo \
944 --subdir=myrepo \
945 --tag-prefix="myrepo-"
949 for ref in $(git -C my-monorepo for-each-ref --format="%(refname)" \
952 git -C my-monorepo branch --no-track myrepo/${branch} ${ref}
956 git -C my-monorepo branch --no-track myrepo/main refs/remotes/myrepo/main
959 git -C my-monorepo checkout local/zip/main # Or local/octopus/main
960 git -C my-monorepo merge myrepo/main
966 ``--tag-prefix`` tells ``import-downstream-repo.py`` to rename
970 the upstream monorepo had its tags rewritten with an "llvmorg-"
971 prefix, name conflicts should not be an issue. ``--tag-prefix`` can
977 R1 - R2 - R3 <- main
984 U1 - U2 - U3 <- upstream/main
986 \ -----\--------------- local/zip--.
988 - Lllvm1 - Llld1 - UM3 - Lclang1 - Lclang2 - Lllvm2 - Llld2 - Lmyproj1 - M1 <-'
990 R1 - R2 - R3 <-.
993 myrepo-release/1 |
995 myrepo/main--'
1008 commits based on some criteria (``zip-downstream-fork.py`` uses the
1018 ----------------------------
1021 clean up. The python tools use ``git-fast-import`` which leaves a lot
1025 git -C my-monorepo checkout main
1029 git -C my-monorepo branch -D local/zip/main || true
1030 git -C my-monorepo branch -D local/octopus/main || true
1033 git -C my-monorepo remote remove upstream/monorepo
1036 git -C my-monorepo remote remove upstream/split/${p}
1037 git -C my-monorepo remote remove local/split/${p}
1040 git -C my-monorepo remote remove localrepo
1041 git -C my-monorepo remote remove umbrella
1042 git -C my-monorepo remote remove myrepo
1054 git -C my-monorepo for-each-ref --format="%(refname)" ${refs_to_clean[@]} |
1055 xargs -n1 --no-run-if-empty git -C my-monorepo update-ref -d
1057 git -C my-monorepo reflog expire --all --expire=now
1060 while ! git -C my-monorepo \
1061 -c gc.reflogExpire=0 \
1062 -c gc.reflogExpireUnreachable=0 \
1063 -c gc.rerereresolved=0 \
1064 -c gc.rerereunresolved=0 \
1065 -c gc.pruneExpire=now \
1066 gc --prune=now; do
1071 git -C my-monorepo repack -A -d -f --depth=250 --window=250
1073 git -C my-monorepo prune-packed
1074 git -C my-monorepo prune
1082 .. [LattnerRevNum] Chris Lattner, http://lists.llvm.org/pipermail/llvm-dev/2011-July/041739.html
1083 .. [TrickRevNum] Andrew Trick, http://lists.llvm.org/pipermail/llvm-dev/2011-July/041721.html
1084 .. [JSonnRevNum] Joerg Sonnenberger, http://lists.llvm.org/pipermail/llvm-dev/2011-July/041688.html
1085 .. [MatthewsRevNum] Chris Matthews, http://lists.llvm.org/pipermail/cfe-dev/2016-July/049886.html
1086 .. [statuschecks] GitHub status-checks, https://help.github.com/articles/about-required-status-chec…