Lines Matching +full:check +full:- +full:clang +full:- +full:python
26 software required to build LLVM, as well as `Python <http://python.org>`_ 3.8 or
35 and ``llvm/test`` respectively and are expected to always pass -- they should be
39 "test-suite") and are in the ``test-suite``
40 `repository on GitHub <https://github.com/llvm/llvm-test-suite.git>`_.
42 tests" in places, which is less ambiguous than "test-suite" and remains
46 ----------
56 ----------------
70 ----------------
77 See `llvm/test/Analysis/BranchProbabilityInfo/loop.ll <https://github.com/llvm/llvm-project/blob/main/llvm/test/Analysis/BranchProbabilityInfo/loop.ll>`_
80 ``test-suite``
81 --------------
84 can be compiled and linked into a stand-alone program that can be
98 The test-suite is located in the ``test-suite``
99 `repository on GitHub <https://github.com/llvm/llvm-test-suite.git>`_.
104 ---------------------------
106 The test suite contains tests to check quality of debugging information.
112 ``cross-project-tests/debuginfo-tests`` directory.
120 main LLVM tree). Use ``make check-all`` to run the unit and regression tests
123 The ``test-suite`` module contains more comprehensive tests including whole C
127 -------------------------
129 To run all of the LLVM unit tests use the check-llvm-unit target:
131 .. code-block:: bash
133 % make check-llvm-unit
135 To run all of the LLVM regression tests use the check-llvm target:
137 .. code-block:: bash
139 % make check-llvm
144 .. code-block:: bash
146 % cmake -DCMAKE_BUILD_TYPE="Release" -DLLVM_ENABLE_ASSERTIONS=On
148 If you have `Clang <https://clang.llvm.org/>`_ checked out and built, you
149 can run the LLVM and Clang tests simultaneously using:
151 .. code-block:: bash
153 % make check-all
158 .. code-block:: bash
160 % make check LIT_ARGS="-v --vg --vg-leak"
164 To run individual tests or subsets of tests, you can use the ``llvm-lit``
168 .. code-block:: bash
170 % llvm-lit <path to llvm-project>/llvm/test/Integer/BitPacked.ll
173 The test files are in the ``llvm-project`` directory, not the directory you
178 .. code-block:: bash
180 % llvm-lit <path to llvm-project>/llvm/test/CodeGen/ARM
182 The regression tests will use the Python psutil module only if installed in a
183 **non-user** location. Under Linux, install with sudo or within a virtual
184 environment. Under Windows, install Python for all users and then run
187 For more information on using the :program:`lit` tool, see ``llvm-lit --help``
191 ---------------------------
193 To run debugging information tests simply add the ``cross-project-tests``
195 command-line.
205 The directory is broken into several sub-directories, each focused on a
209 ----------------------------
218 how to run the tests. This file is just Python code and thus is very
257 .. code-block:: llvm
259 ; RUN: llvm-as < %s | llvm-dis > %t1
260 ; RUN: llvm-dis < %s.bc-13 > %t2
276 lines is deprecated - please do not send or commit patches that use it.]*
279 test. Check if there are files already covering your feature and consider
283 -----------------------------------------
289 For example to generate assertions in an :program:`llc`-based test, after
292 .. code-block:: bash
294 % llvm/utils/update_llc_test_checks.py --llc-binary build/bin/llc test.ll
299 If you want to update assertions in an existing test case, pass the `-u` option
302 Sometimes a test absolutely depends on hand-written assertions and should not
311 .. code-block:: none
314 opt -passes='print<cost-model>'
317 C/C++, or clang/clang++ (IR checks)
323 llvm-mca
332 ----------------------------
335 baseline check-lines first. That is, the test will show a miscompile or
339 A follow-up patch with code changes to the compiler will then show check-line
343 Baseline tests (no-functional-change or NFC patch) may be pushed to main
344 without pre-commit review if you have commit access.
347 -----------------------------------
349 - Use auto-generated check lines (produced by the scripts mentioned above)
351 - Include comments about what is tested/expected in a particular test. If there
354 - Avoid undefined behavior and poison/undef values unless necessary. For
357 - Minimize tests by removing unnecessary instructions, metadata, attributes,
358 etc. Tools like ``llvm-reduce`` can help automate this.
359 - Outside PhaseOrdering tests, only run a minimal set of passes. For example,
360 prefer ``opt -S -passes=instcombine`` over ``opt -S -O3``.
361 - Avoid unnamed instructions/blocks (such as ``%0`` or ``1:``), because they may
363 running the test through ``opt -S -passes=instnamer``.
364 - Try to give values (including variables, blocks and functions) meaningful
369 -----------
373 using ``split-file`` to extract them. For example,
375 .. code-block:: llvm
377 ; RUN: split-file %s %t
378 ; RUN: llvm-link -S %t/a.ll %t/b.ll | FileCheck %s
380 ; CHECK: ...
382 ;--- a.ll
384 ;--- b.ll
387 The parts are separated by the regex ``^(.|//)--- <part>``.
390 ``--leading-lines`` to add leading empty lines to preserve line numbers.
407 .. code-block:: llvm
411 ; RUN: llvm-link %S/Inputs/ident.a.ll %S/Inputs/ident.b.ll -S | FileCheck %s
415 ; CHECK-DAG: !llvm.ident = !{!0, !1, !2}
416 ; CHECK-DAG: "Compiler V1"
417 ; CHECK-DAG: "Compiler V2"
418 ; CHECK-DAG: "Compiler V3"
441 ----------------
446 output from Clang), you can include generation instructions within
447 ``split-file`` part called ``gen``. Then, run
451 .. code-block:: none
453 ; RUN: rm -rf %t && split-file %s %t && cd %t
454 ; RUN: opt -S a.ll ... | FileCheck %s
456 ; CHECK: hello
458 ;--- a.cc
460 ;--- gen
461 clang --target=x86_64-linux -S -emit-llvm -g a.cc -o -
463 ;--- a.ll
466 .. code-block:: bash
470 The script will prepare extra files with ``split-file``, invoke ``gen``, and
475 skip ``split-file`` in RUN lines.
477 .. code-block:: none
479 # RUN: llvm-mc -filetype=obj -triple=x86_64 %s -o a.o
482 # CHECK: hello
485 #--- a.cc
487 #--- gen
488 clang --target=x86_64-linux -S -g a.cc -o -
497 ``gen`` is invoked with ``PWD`` set to ``/proc/self/cwd``. Clang commands
498 don't need ``-fdebug-compilation-dir=`` since its default value is ``PWD``.
500 Check prefixes should be placed before ``.endif`` since the part after
503 If the test body contains multiple files, you can print ``---`` separators and
504 utilize ``split-file`` in ``RUN`` lines.
506 .. code-block:: none
508 # RUN: rm -rf %t && split-file %s %t && cd %t
511 #--- a.cc
513 #--- b.cc
515 #--- gen
516 clang --target=x86_64-linux -S -O1 -g a.cc -o -
517 echo '#--- b.s'
518 clang --target=x86_64-linux -S -O1 -g b.cc -o -
519 #--- a.s
522 -------------
528 .. code-block:: console
535 $ opt -S /path/to/example.ll
542 ``ModuleID`` can unexpectedly match against ``CHECK`` lines. For example:
544 .. code-block:: llvm
546 ; RUN: opt -S %s | FileCheck
549 ; CHECK-NOT: load
558 Platform-Specific Tests
559 -----------------------
562 either related to code generated, specific output or back-end features,
564 run on different architectures (and don't even compile all back-ends),
567 The first problem is to check for target-specific output, for example sizes
570 * Tests containing Windows paths will fail on Linux and vice-versa.
571 * Tests that check for ``x86_64`` somewhere in the text will fail anywhere else.
574 Also, if the test rely on any behaviour that is coded in any back-end, it must
578 only run if a specific back-end is compiled and available.
582 .. code-block:: python
588 Other platform-specific tests are those that depend on a specific feature
589 of a specific sub-architecture, for example only to Intel chips that support ``AVX2``.
591 For instance, ``test/CodeGen/X86/psubus.ll`` tests three sub-architecture
594 .. code-block:: llvm
596 ; RUN: llc -mcpu=core2 < %s | FileCheck %s -check-prefix=SSE2
597 ; RUN: llc -mcpu=corei7-avx < %s | FileCheck %s -check-prefix=AVX1
598 ; RUN: llc -mcpu=core-avx2 < %s | FileCheck %s -check-prefix=AVX2
602 .. code-block:: llvm
611 So, if you're testing for a behaviour that you know is platform-specific or
612 depends on special features of sub-architectures, you must add the specific
618 ---------------------------
629 .. code-block:: llvm
634 ; UNSUPPORTED: system-linux
638 ``REQUIRES`` and ``UNSUPPORTED`` and ``XFAIL`` all accept a comma-separated
641 - Features added to ``config.available_features`` by configuration files such as ``lit.cfg``.
642 String comparison of features is case-sensitive. Furthermore, a boolean expression can
643 contain any Python regular expression enclosed in ``{{ }}``, in which case the boolean
647 - The default target triple, preceded by the string ``target=`` (for example,
648 ``target=x86_64-pc-windows-msvc``). Typically regular expressions are used
649 to match parts of the triple (for example, ``target={{.*}}-windows{{.*}}``
659 .. code-block:: llvm
663 ; UNSUPPORTED: system-windows, target={{.*linux.*}} && !target={{.*android.*}}
665 ; XFAIL: target=powerpc{{.*}}, system-darwin
669 ----------------------------
692 Checking the target triple can be tricky; it's easy to mis-specify. For
694 mips64, and mips64el. ``target={{.*}}-linux-gnu`` will match
695 x86_64-unknown-linux-gnu, but not armv8l-unknown-linux-gnueabihf.
696 Prefer to use hyphens to delimit triple components (``target=mips-{{.*}}``)
705 ``target={{.+-freebsd.*}} || target={{.+-netbsd.*}}``
709 -------------
740 Use ``rm -rf %t && mkdir %t`` instead if a temporary directory is necessary.
748 ``%{fs-src-root}``
752 ``%{fs-tmp-root}``
757 ``%{fs-sep}``
808 ``%(line)``, ``%(line+<number>)``, ``%(line-<number>)``
817 **LLVM-specific substitutions:**
831 **Clang-specific substitutions:**
833 ``%clang``
834 Invokes the Clang driver.
837 Invokes the Clang driver for C++.
840 Invokes the CL-compatible Clang driver.
843 Invokes the G++-compatible Clang driver.
846 Invokes the Clang frontend.
851 ``i686-pc-win32`` target, ``%itanium_abi_triple`` will expand to
852 ``i686-pc-mingw32``. This allows a test to run with a specific ABI without
855 **FileCheck-specific substitutions:**
862 .. _Test-specific substitutions:
864 **Test-specific substitutions:**
868 - Lit configuration files (e.g., ``lit.cfg`` or ``lit.local.cfg``) can define
872 (even if it contains sequences that python's ``re.sub`` considers to be
874 - To define substitutions within a single test file, lit supports the
881 serve as the parameters of another newly defined ``%{check}`` substitution:
883 .. code-block:: llvm
888 ; DEFINE: %{check} = \
889 ; DEFINE: %clang_cc1 -verify -fopenmp -fopenmp-version=51 %{cflags} \
890 ; DEFINE: -emit-llvm -o - %s | \
896 desired before each use of ``%{check}`` in a ``RUN:`` line:
898 .. code-block:: llvm
900 ; REDEFINE: %{cflags} = -triple x86_64-apple-darwin10.6.0 -fopenmp-simd
901 ; REDEFINE: %{fcflags} = -check-prefix=SIMD
902 ; RUN: %{check}
904 ; REDEFINE: %{cflags} = -triple x86_64-unknown-linux-gnu -fopenmp-simd
905 ; REDEFINE: %{fcflags} = -check-prefix=SIMD
906 ; RUN: %{check}
908 ; REDEFINE: %{cflags} = -triple x86_64-apple-darwin10.6.0
909 ; REDEFINE: %{fcflags} = -check-prefix=NO-SIMD
910 ; RUN: %{check}
912 ; REDEFINE: %{cflags} = -triple x86_64-unknown-linux-gnu
913 ; REDEFINE: %{fcflags} = -check-prefix=NO-SIMD
914 ; RUN: %{check}
918 establish the substitution order so that both ``%{check}`` and its parameters
925 - Upon arriving at each ``RUN:`` line, lit expands all substitutions in that
929 ``%(line-<number>)``.
930 - When expanding substitutions in a ``RUN:`` line, lit makes only one pass
936 - While lit configuration files can insert anywhere in the substitution list,
940 - Defining a substitution in terms of itself, whether directly or via other
949 - ``DEFINE: %{name} = value``
960 - ``REDEFINE: %{name} = value``
972 - **Substitution name**: In the directive, whitespace immediately before or
978 - It is impossible for ``%{name}`` to contain sequences that are special in
979 python's ``re.sub`` patterns. Otherwise, attempting to specify
982 - The braces help avoid the possibility that another substitution's pattern
983 will match part of ``%{name}`` or vice-versa, producing confusing
988 - **Substitution value**: The value includes all text from the first
989 non-whitespace character after ``=`` to the last non-whitespace character. If
990 there is no non-whitespace character after ``=``, the value is the empty
991 string. Escape sequences that can appear in python ``re.sub`` replacement
993 - **Line continuations**: If the last non-whitespace character on the line after
1014 .. code-block:: llvm
1028 .. code-block:: shell
1038 ``config.recursiveExpansionLimit``, which can be set to a non-negative integer
1044 .. code-block:: shell
1058 -------
1067 % llvm-lit "-Dllc=llc -verify-machineinstrs"
1077 --------------
1085 Zero result codes become 1. Non-zero result codes become 0.
1089 ``PR[0-9]+``. This is the syntax for specifying a PR (Problem Report) number