Lines Matching +full:check +full:- +full:clang +full:- +full:python
5 --------------------
12 driver or through ``lldb-test`` which is a tool that exposes the internal
13 data structures in an easy-to-parse way for testing. Most people will know
19 SB API. These are written in Python and use LLDB's ``dotest.py`` testing
20 framework on top of Python's `unittest
21 <https://docs.python.org/3/library/unittest.html>`_.
44 ``lldb-test`` using ``FileCheck``. Shell tests are generally small and fast to
47 ``lldb-test`` is a relatively new addition to the test suite. It was the first
54 the subcomponents already exposed by lldb-test. But when it comes to LLDB's
56 as the Python API. For example, to test setting a breakpoint, you could do it
62 means that you have to have a well-defined test scenario that you can easily
76 of (some) aspects of handling of binaries with non-native architectures or
91 ``dotest.py``. Tests are written in Python and test binaries (inferiors) are
92 compiled with Make. The majority of API tests are end-to-end tests that compile
96 implementation is located under ``lldb/packages/Python/lldbsuite``. We have
98 `unittest <https://docs.python.org/3/library/unittest.html>`_. Those can be
100 `lldbtest.py <https://github.com/llvm/llvm-project/blob/main/lldb/packages/Python/lldbsuite/test/lldbtest.py>`_.
103 <https://github.com/llvm/llvm-project/tree/main/lldb/test/API/sample_test>`_.
104 The test directory will always contain a python file, starting with ``Test``.
115 Let's start with the Python test file. Every test is its own class and can have
128 <https://github.com/llvm/llvm-project/blob/main/lldb/packages/Python/lldbsuite/test/lldbutil.py>`_
134 <https://ftp.gnu.org/old-gnu/Manuals/dejagnu-1.3/html_node/dejagnu_6.html>`_
164 CFLAGS_EXTRAS := -std=c99
169 `Makefile.rules <https://github.com/llvm/llvm-project/blob/main/lldb/packages/Python/lldbsuite/test/make/Makefile.rules>`_
198 arguments -- without any modifications to the code. You can create a focused
228 the `SBType`-related parts of the API. With those languages it's also
235 Languages such as Objective-C that have a dependency on a runtime
243 **Avoid specifying test-specific compiler flags when including system headers.**
249 in your test (e.g., adding ``-DFOO`` to the ``Makefile`` or ``self.build``
262 should always be deterministic (i.e., do not generate and check against
286 Avoid writing your tests on top of ``self.expect(...)`` calls that check
293 check will unexpectedly pass if it's ran as the first expression in a test:
317 last option as they give non-descriptive error messages. The test class has
320 ones. Check the documentation of Python's ``unittest`` module to see what
324 +-----------------------------------------------+-----------------------------------------------------------------+
326 +-----------------------------------------------+-----------------------------------------------------------------+
328 +-----------------------------------------------+-----------------------------------------------------------------+
330 +-----------------------------------------------+-----------------------------------------------------------------+
332 +-----------------------------------------------+-----------------------------------------------------------------+
345 **Do not use hard-coded line numbers in your test case.**
353 .. code-block:: c
359 .. code-block:: c
363 The Python test case TestBreakpointConditions.py uses the comment strings to find the line numbers during setUp(self) and use them
369 These features can be use to properly mark your test class or method for platform-specific tests, compiler specific, version specific.
373 .. code-block:: python
377 @skipIf(compiler=no_match("clang"))
383 self.do_test(dict(CFLAGS_EXTRAS="-gdwarf-5 -gpubnames"))
385 This tells the test harness that unless we are running "linux" and clang version equal & above 8.0, the test should be skipped.
387 **Class-wise cleanup after yourself.**
389 TestBase.tearDownClass(cls) provides a mechanism to invoke the platform-specific cleanup after finishing with a test class. A test
393 The default cleanup action performed by the packages/Python/lldbsuite/test/lldbtest.py module invokes the "make clean" os command.
398 .. code-block:: python
411 --
426 -----------------
430 On Windows any invocations of python should be replaced with python_d, the
442 The easiest way to run the LLDB test suite is to use the ``check-lldb`` build
447 $ ninja check-lldb
452 By default, the ``check-lldb`` target builds the test programs with the same
461 used by appending ``-A`` and ``-C`` options respectively. For example, to test
462 LLDB against 32-bit binaries built with a custom version of clang, do:
466 $ cmake -DLLDB_TEST_USER_ARGS="-A;i386;-C;/path/to/custom/clang" -G Ninja
467 $ ninja check-lldb
469 Note that multiple ``-A`` and ``-C`` flags can be specified to
473 the ``--setting`` option of the test runner via this same variable. For example
474 ``--setting;target.disable-aslr=true``.
477 ``<build-dir>/bin/lldb-dotest --help``.
483 with ``check-lldb``.
485 * Use ``check-lldb-unit`` to run just the unit tests.
486 * Use ``check-lldb-api`` to run just the SB API tests.
487 * Use ``check-lldb-shell`` to run just the shell tests.
491 target ``check-lldb-shell-objectfile``. However, because the unit tests and API
504 $ ./bin/llvm-lit -sv <llvm-project-root>/lldb/test --filter <test>
512 $ ./bin/llvm-lit -sv <llvm-project-root>/lldb/test/Shell/Commands/CommandScriptImmediateOutput
516 passing ``--param`` to lit and setting a value for ``dotest-args``.
520 $ ./bin/llvm-lit -sv <llvm-project-root>/lldb/test --param dotest-args='-C gcc'
529 In addition to running all the LLDB test suites with the ``check-lldb`` CMake
531 build you can use the ``lldb-dotest`` binary, which is a wrapper around
535 one-off with a different configuration.
541 $ ./bin/lldb-dotest -p TestInferiorCrashing.py
546 $ python dotest.py --executable <path-to-lldb> -p TestInferiorCrashing.py ../packages/Python/lldbsuite/test
548 If the test is not specified by name (e.g. if you leave the ``-p`` argument
554 $ ./bin/lldb-dotest functionalities/data-formatter
558 $ python dotest.py --executable <path-to-lldb> functionalities/data-formatter
564 $ python dotest.py -h
583 $ ./tools/lldb/unittests/Host/HostTests --gtest_filter=SocketTest.DomainListenConnectAccept
589 Running the test-suite remotely is similar to the process of running a local
592 1. You must have the lldb-server running on the remote system, ready to accept
595 2. You must tell the test-suite how to connect to the remote system. This is
597 flags to cmake, and ``--platform-name`` parameter to ``dotest.py``.
605 being used is other than Clang.
612 actual hardware. :doc:`/resources/qemu-testing` describes how to setup an
614 ``llvm-project/lldb/scripts/lldb-test-qemu``. These scripts currently
618 -----------------------
620 On non-Windows platforms, you can use the ``-d`` option to ``dotest.py`` which
622 until a debugger is attached. Then run ``lldb -p <pid>`` to attach.
624 To instead debug a test's python source, edit the test and insert ``import pdb; pdb.set_trace()`` or ``breakpoint()`` (Python 3 only) at the point you want to start debugging. The ``breakpoint()`` command can be used for any LLDB Python script, not just for API tests.
637 On Windows, it is strongly recommended to use Python Tools for Visual Studio
645 #. Create a Visual Studio Project for the Python code.
646 #. Go to File -> New -> Project -> Python -> From Existing Python Code.
647 #. Choose llvm/tools/lldb as the directory containing the Python code.
650 #. Make sure there is a Python Environment installed for your distribution. For example, if you installed Python to ``C:\Python35``, PTVS needs to know that this is the interpreter you want to use for running the test suite.
651 #. Go to Tools -> Options -> Python Tools -> Environment Options
652 #. Click Add Environment, and enter Python 3.5 Debug for the name. Fill out the values correctly.
655 #. In the General tab, Make sure Python 3.5 Debug is the selected Interpreter.
656 #. In Debug/Search Paths, enter the path to your ninja/lib/site-packages directory.
658 #. If you want to enabled mixed mode debugging, check Enable native code debugging (this slows down debugging, so enable it only on an as-needed basis.)
666 --arch=i686
668 --executable D:/src/llvmbuild/ninja/bin/lldb.exe
670 -s D:/src/llvmbuild/ninja/lldb-test-traces
671 -u CXXFLAGS -u CFLAGS
673 --enable-crash-dialog
674 # Path to release clang.exe
675 -C d:\src\llvmbuild\ninja_release\bin\clang.exe
677 -p TestPaths.py
679 D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test
683 --arch=i686 --executable D:/src/llvmbuild/ninja/bin/lldb.exe -s D:/src/llvmbuild/ninja/lldb-test-traces -u CXXFLAGS -u CFLAGS --enable-crash-dialog -C d:\src\llvmbuild\ninja_release\bin\clang.exe -p TestPaths.py D:\src\llvm\tools\lldb\packages\Python\lldbsuite\test --no-multiprocess
685 .. [#] `https://lldb.llvm.org/python_reference/lldb.SBTarget-class.html#BreakpointCreateByName <https://lldb.llvm.org/python_reference/lldb.SBTarget-class.html#BreakpointCreateByName>`_