[lldb-dap] Improve `stackTrace` and `exceptionInfo` DAP request handlers (#105905)

Refactoring `stackTrace` to perform frame look ups in a more on-demand
fashion to improve overall performance.

[lldb-dap] Improve `stackTrace` and `exceptionInfo` DAP request handlers (#105905)

Refactoring `stackTrace` to perform frame look ups in a more on-demand
fashion to improve overall performance.

Additionally adding additional information to the `exceptionInfo`
request to report exception stacks there instead of merging the
exception stack into the stack trace. The `exceptionInfo` request is
only called if a stop event occurs with `reason='exception'`, which
should mitigate the performance of `SBThread::GetCurrentException`
calls.

Adding unit tests for exception handling and stack trace supporting.

show more ...

[lldb] Remove python helper getCompilerBinary() (#100660)

This causes a number of tests be `UNRESOLVED` on Windows if
`getCompiler()` has a space in the name, because `getCompilerBinary()`
uncondi

[lldb] Remove python helper getCompilerBinary() (#100660)

This causes a number of tests be `UNRESOLVED` on Windows if
`getCompiler()` has a space in the name, because `getCompilerBinary()`
unconditionally splits on whitespace and returns the first result, which
might just be`"C:\Program"` if using a compiler such as `clang-cl` `cl`
from the absolute path to Visual studio's installation directory.

Co-authored-by: kendal <kendal@thebrowser.company>

show more ...

lldb: android: fix missing Python import of urlparse in lldb test utilities (#99934)

## Issue
Attempting to run the lldb API tests against a remote-android target
fails with the error `NameError:

lldb: android: fix missing Python import of urlparse in lldb test utilities (#99934)

## Issue
Attempting to run the lldb API tests against a remote-android target
fails with the error `NameError: name 'urlparse' is not defined`.

## Root Cause
It looks the Python import of `urlparse` was removed by mistake in
22ea97d7bfd65abf68a68b13bf96ad69be23df54. This import is only used when
running the lldb API tests against a remote-android target so it went
unnoticed.

## Fix
This change simply puts back the missing import. It is a one line
change.

fixes #99931

## Validation
Tested on Fedora 39 with an attached Android device:

`cd llvm-project`
`cmake -S llvm -B build -G Ninja -DLLVM_ENABLE_PROJECTS='clang;lldb'
-DCMAKE_BUILD_TYPE=Release -DLLDB_ENABLE_PYTHON=On`
`ninja -C build`
`./build/bin/lldb-dotest --arch aarch64 --out-of-tree-debugserver
--platform-name=remote-android
--platform-working-dir=/data/local/tmp/ds2
--platform-url=connect://localhost:5432 --compiler
~/Android/Sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/linux-x86_64/bin/clang`

show more ...

[lldb] add RISCV target specific info in API tests (#99039)

Add information about RISCV first register in python API testsuite, that
is used to check register readability in tests.

Fixed tests o

[lldb] add RISCV target specific info in API tests (#99039)

Add information about RISCV first register in python API testsuite, that
is used to check register readability in tests.

Fixed tests on RISCV target:

TestBreakpointByFileColonLine.BreakpointByLineAndColumnTestCase
TestAddressBreakpoints.AddressBreakpointTestCase
TestBreakpointAutoContinue.BreakpointAutoContinue
TestInterruptBacktrace.TestInterruptingBacktrace
TestBadAddressBreakpoints.BadAddressBreakpointTestCase
TestScriptedResolver.TestScriptedResolver
TestStopHookScripted.TestStopHooks
TestBreakpointConditions.BreakpointConditionsTestCase
TestLocalVariables.LocalVariablesTestCase
TestFindLineEntry.FindLineEntry
TestScriptedResolver.TestScriptedResolver
TestInlineSourceFiles.InlineSourceFilesTestCase
TestModuleAndSection.ModuleAndSectionAPIsTestCase
TestFrameVar.TestFrameVar
TestInferiorAssert.AssertingInferiorTestCase
TestInferiorCrashing.CrashingInferiorTestCase
TestInferiorCrashingStep.CrashingInferiorStepTestCase
TestRegistersIterator.RegistersIteratorTestCase
TestCoroutineHandle.TestCoroutineHandle
TestWithLimitDebugInfo.TestWithLimitDebugInfo
TestLLDBIterator.LLDBIteratorTestCase
TestMemoryWrite.MemoryWriteTestCase
TestNestedTemplate.NestedTemplateTestCase
TestParrayVrsCharArrayChild.TestParrayVrsCharArrayChild
TestRecursiveInferior.CrashingRecursiveInferiorTestCase
TestRecursiveInferiorStep.CrashingRecursiveInferiorStepTestCase
TestRunLocker.TestRunLocker
TestSampleTest.RenameThisSampleTestTestCase
TestUniqueTypes3.UniqueTypesTestCase3
TestPrintStackTraces.ThreadsStackTracesTestCase
TestUnicodeSymbols.TestUnicodeSymbols
TestUnusedInlinedParameters.TestUnusedInlinedParameters
TestValueVarUpdate.ValueVarUpdateTestCase
TestPtrRef2Typedef.PtrRef2TypedefTestCase
TestDataFormatterStdIterator.StdIteratorDataFormatterTestCase
TestDataFormatterStdString.StdStringDataFormatterTestCase
TestDataFormatterStdVBool.StdVBoolDataFormatterTestCase

show more ...

[lldb][test] Set target and host OS for API tests in case of remote testing

Makefile.rules uses HOST_OS and OS variables for determining host and target
OSes for API tests compilation.

This comm

[lldb][test] Set target and host OS for API tests in case of remote testing

Makefile.rules uses HOST_OS and OS variables for determining host and target
OSes for API tests compilation.

This commit moves the platform detection logic from Makefile to Python lldb
test suite.

This is useful for the case of Windows-to-Linux cross-testing.

show more ...

[lldb] Use packaging module instead of pkg_resources (#93712)

Use the packaging [1] module for parsing version numbers, instead of
pkg_resources which is distributed with setuptools. I recently swi

[lldb] Use packaging module instead of pkg_resources (#93712)

Use the packaging [1] module for parsing version numbers, instead of
pkg_resources which is distributed with setuptools. I recently switched
over to using the latter, knowing it was deprecated (in favor of the
packaging module) because it comes with Python out of the box. Newer
versions of setuptools have removed `pkg_resources` so we have to use
packaging.

[1] https://pypi.org/project/packaging/

show more ...

[lldb] Add ability to detect darwin host linker version to xfail tests (#83941)

When Apple released its new linker, it had a subtle bug that caused
LLDB's TLS tests to fail. Unfortunately this mean

[lldb] Add ability to detect darwin host linker version to xfail tests (#83941)

When Apple released its new linker, it had a subtle bug that caused
LLDB's TLS tests to fail. Unfortunately this means that TLS tests are
not going to work on machines that have affected versions of the linker,
so we should annotate the tests so that they only work when we are
confident the linker has the required fix.

I'm not completely satisfied with this implementation. That being said,
I believe that adding suport for linker versions in general is a
non-trivial change that would require far more thought. There are a few
challenges involved:
- LLDB's testing infra takes an argument to change the compiler, but
there's no way to switch out the linker.
- There's no standard way to ask a compiler what linker it will use.
- There's no standard way to ask a linker what its version is. Many
platforms have the same name for their linker (ld).
- Some platforms automatically switch out the linker underneath you. We
do this for Windows tests (where we use LLD no matter what).

Given that this is affecting the tests on our CI, I think this is an
acceptable solution in the interim.

show more ...

[lldb] Migrate distutils.version.LooseVersion to packaging (#82066)

The distutils package has been deprecated and was removed from Python
3.12. The migration page [1] advises to use the packaging mo

[lldb] Migrate distutils.version.LooseVersion to packaging (#82066)

The distutils package has been deprecated and was removed from Python
3.12. The migration page [1] advises to use the packaging module
instead. Since Python 3.6 that's vendored into pkg_resources.

[1] https://peps.python.org/pep-0632/#migration-advice

show more ...

Revert "[lldb] Migrate distutils.version.LooseVersion to packaging" (#82297)

Reverts llvm/llvm-project#82066 because the following tests started
failing after:


[lldb-api.commands/expression/im

Revert "[lldb] Migrate distutils.version.LooseVersion to packaging" (#82297)

Reverts llvm/llvm-project#82066 because the following tests started
failing after:


[lldb-api.commands/expression/import-std-module/deque-basic.TestDequeFromStdModule.py](https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/66513/testReport/junit/lldb-api/commands_expression_import-std-module_deque-basic/TestDequeFromStdModule_py/)

[lldb-api.commands/expression/import-std-module/deque-dbg-info-content.TestDbgInfoContentDequeFromStdModule.py](https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/66513/testReport/junit/lldb-api/commands_expression_import-std-module_deque-dbg-info-content/TestDbgInfoContentDequeFromStdModule_py/)

[lldb-api.commands/expression/import-std-module/forward_list.TestForwardListFromStdModule.py](https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/66513/testReport/junit/lldb-api/commands_expression_import-std-module_forward_list/TestForwardListFromStdModule_py/)

[lldb-api.commands/expression/import-std-module/forward_list-dbg-info-content.TestDbgInfoContentForwardListFromStdModule.py](https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/66513/testReport/junit/lldb-api/commands_expression_import-std-module_forward_list-dbg-info-content/TestDbgInfoContentForwardListFromStdModule_py/)

[lldb-api.commands/expression/import-std-module/list.TestListFromStdModule.py](https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/66513/testReport/junit/lldb-api/commands_expression_import-std-module_list/TestListFromStdModule_py/)

[lldb-api.commands/expression/import-std-module/non-module-type-separation.TestNonModuleTypeSeparation.py](https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/66513/testReport/junit/lldb-api/commands_expression_import-std-module_non-module-type-separation/TestNonModuleTypeSeparation_py/)

[lldb-api.commands/expression/import-std-module/queue.TestQueueFromStdModule.py](https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/66513/testReport/junit/lldb-api/commands_expression_import-std-module_queue/TestQueueFromStdModule_py/)

[lldb-api.commands/expression/import-std-module/retry-with-std-module.TestRetryWithStdModule.py](https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/66513/testReport/junit/lldb-api/commands_expression_import-std-module_retry-with-std-module/TestRetryWithStdModule_py/)

[lldb-api.commands/expression/import-std-module/unique_ptr.TestUniquePtrFromStdModule.py](https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/66513/testReport/junit/lldb-api/commands_expression_import-std-module_unique_ptr/TestUniquePtrFromStdModule_py/)

[lldb-api.commands/expression/import-std-module/unique_ptr-dbg-info-content.TestUniquePtrDbgInfoContent.py](https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/66513/testReport/junit/lldb-api/commands_expression_import-std-module_unique_ptr-dbg-info-content/TestUniquePtrDbgInfoContent_py/)

[lldb-api.commands/expression/import-std-module/vector-of-vectors.TestVectorOfVectorsFromStdModule.py](https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/66513/testReport/junit/lldb-api/commands_expression_import-std-module_vector-of-vectors/TestVectorOfVectorsFromStdModule_py/)

[lldb-api.functionalities/data-formatter/data-formatter-stl/libcxx/shared_ptr.TestDataFormatterLibcxxSharedPtr.py](https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/66513/testReport/junit/lldb-api/functionalities_data-formatter_data-formatter-stl_libcxx_shared_ptr/TestDataFormatterLibcxxSharedPtr_py/)

[lldb-api.functionalities/data-formatter/data-formatter-stl/libcxx/unique_ptr.TestDataFormatterLibcxxUniquePtr.py](https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/66513/testReport/junit/lldb-api/functionalities_data-formatter_data-formatter-stl_libcxx_unique_ptr/TestDataFormatterLibcxxUniquePtr_py/)

show more ...

[lldb] Migrate distutils.version.LooseVersion to packaging (#82066)

The distutils package has been deprecated and was removed from Python
3.12. The migration page [1] advises to use the packaging m

[lldb] Migrate distutils.version.LooseVersion to packaging (#82066)

The distutils package has been deprecated and was removed from Python
3.12. The migration page [1] advises to use the packaging module
instead. Since Python 3.6 that's vendored into pkg_resources.

[1] https://peps.python.org/pep-0632/#migration-advice

show more ...

[lldb] Make use of LD_LIBRARY_PATH on OpenBSD (#74017)

[lldb/test] Add OpenBSD to _get_platform_os (#74036)

[lldb][test] Remove `self` references from decorators (#72416)

This is partial step toward removing the vendored `unittest2` dep in
favor of the `unittest` library in standard python. One of the lar

[lldb][test] Remove `self` references from decorators (#72416)

This is partial step toward removing the vendored `unittest2` dep in
favor of the `unittest` library in standard python. One of the large
differences is when xfail decorators are evaluated. With the `unittest2`
vendored dep, this can happen at the moment of calling the test case,
and with LLDB's decorator wrappers, we are passed the test class in the
decorator arg. With the `unittest` framework, this is determined much
earlier; we cannot decide when the test is about to start that we need
to xfail.

Fortunately, almost none of these checks require any state that can't be
determined statically. For this patch, I moved the impl for all the
checks to `lldbplatformutil` and pointed the decorators to that,
removing as many `self` (i.e. test class object) references as possible.
I left wrappers within `TestBase` that forward to `lldbplatformutil` for
convenience, but we should probably remove those later.

The remaining check that can't be moved statically is the check for the
debug info type (e.g. to xfail only for dwarf). Fixing that requires a
different approach, so I will postpone that to the next patch.

show more ...

[lldb] Remove use of __future__ in python

These were useful primarily for the Python 2 to 3 transition. Python 2
is no longer supported so these are no longer necessary.

Differential Revision: http

[lldb] Remove use of __future__ in python

These were useful primarily for the Python 2 to 3 transition. Python 2
is no longer supported so these are no longer necessary.

Differential Revision: https://reviews.llvm.org/D157759

show more ...

[NFC][Py Reformat] Reformat python files in lldb

This is an ongoing series of commits that are reformatting our Python
code. Reformatting is done with `black` (23.1.0).

If you end up having problem

[NFC][Py Reformat] Reformat python files in lldb

This is an ongoing series of commits that are reformatting our Python
code. Reformatting is done with `black` (23.1.0).

If you end up having problems merging this commit because you have made
changes to a python file, the best way to handle that is to run `git
checkout --ours <yourfile>` and then reformat it with black.

RFC: https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style

Differential revision: https://reviews.llvm.org/D151460

show more ...

[lldb/test] Use SBPlatform info for lldbplatformutil.getPlatform()

Previously, we just used the platform name. This worked mostly OK, but
it required adding special handling for any unusual (and pot

[lldb/test] Use SBPlatform info for lldbplatformutil.getPlatform()

Previously, we just used the platform name. This worked mostly OK, but
it required adding special handling for any unusual (and potentially
downstream) platform plugins, as evidenced by the hardcoding of the
qemu-user platform.

The current implementation was added in
D121605/21c5bb0a636c23ec75b13681c0a6fdb03ecd9c0d, which this essentially
reverts and goes back to the previous method of retrieving the platform
name from the platform triple (the "OS" field).

The motivation for D121605 was the ability to retrieve the process
without constructing an SBDebugger object (which would be necessary in a
world where SBPlatforms are managed by SBDebuggers). However, this world
did not arrive (mainly due to other commitments on my part), and I now
think that if we do want to go in that direction, that we should just
create a dummy/empty SBDebugger object for holding the initial
SBPlatform.

One benefit of D121605 was the unification of getPlatform and
getHostPlatform code paths, and I preserve that benefit by unifying them
in the other direction -- using the host SBPlatform for getHostPlatform.

Differential Revision: https://reviews.llvm.org/D138430

show more ...

[lldb] Remove uses of six module (NFC)

With lldb (& llvm) requiring Python 3.6+, use of the `six` module can be removed.

Differential Revision: https://reviews.llvm.org/D131304

Recommit "[lldb/test] Don't use preexec_fn for launching inferiors"

This recommits b15b1421, which reverted in was reverted in f51c47d98 due to
failures on apple systems. The problem was that the pa

Recommit "[lldb/test] Don't use preexec_fn for launching inferiors"

This recommits b15b1421, which reverted in was reverted in f51c47d98 due to
failures on apple systems. The problem was that the patch introduced a race
where the debug server could start the attach process before the first process
(which isn't supposed to be attached to) was set up. This caused us to attach
to the wrong process.

The new version introduces additional synchronization to ensure that does not
happen.

Original commit message was:
As the documentation states, using this is not safe in multithreaded
programs, and I have traced it to a rare deadlock in some of the tests.

The reason this was introduced was to be able to attach to a program
from the very first instruction, where our usual mechanism of
synchronization -- waiting for a file to appear -- does not work.

However, this is only needed for a single test
(TestGdbRemoteAttachWait) so instead of doing this everywhere, I create
a bespoke solution for that single test. The solution basically
consists of outsourcing the preexec_fn code to a separate (and
single-threaded) shim process, which enables attaching and then executes
the real program.

This pattern could be generalized in case we needed to use it for other
tests, but I suspect that we will not be having many tests like this.

This effectively reverts commit
a997a1d7fbe229433fb458bb0035b32424ecf3bd.

show more ...

Revert "[lldb/test] Don't use preexec_fn for launching inferiors"

This reverts commit b15b1421bc9a11b318b65b489e5fd58dd917db1f because it
breaks GreenDragon [1]. The bot has been red for several day

Revert "[lldb/test] Don't use preexec_fn for launching inferiors"

This reverts commit b15b1421bc9a11b318b65b489e5fd58dd917db1f because it
breaks GreenDragon [1]. The bot has been red for several days, so
reverting to green while I take a look.

[1] https://green.lab.llvm.org/green/view/LLDB/job/lldb-cmake/45012/

show more ...

[lldb/test] Don't use preexec_fn for launching inferiors

As the documentation states, using this is not safe in multithreaded
programs, and I have traced it to a rare deadlock in some of the tests.

[lldb/test] Don't use preexec_fn for launching inferiors

As the documentation states, using this is not safe in multithreaded
programs, and I have traced it to a rare deadlock in some of the tests.

The reason this was introduced was to be able to attach to a program
from the very first instruction, where our usual mechanism of
synchronization -- waiting for a file to appear -- does not work.

However, this is only needed for a single test
(TestGdbRemoteAttachWait) so instead of doing this everywhere, I create
a bespoke solution for that single test. The solution basically
consists of outsourcing the preexec_fn code to a separate (and
single-threaded) shim process, which enables attaching and then executes
the real program.

This pattern could be generalized in case we needed to use it for other
tests, but I suspect that we will not be having many tests like this.

This effectively reverts commit
a997a1d7fbe229433fb458bb0035b32424ecf3bd.

show more ...

Recommit [lldb/test] Make category-skipping logic "platform"-independent

This recommits dddf4ce03, which was reverted because of a couple of test
failures on macos. The reason behind the failures wa

Recommit [lldb/test] Make category-skipping logic "platform"-independent

This recommits dddf4ce03, which was reverted because of a couple of test
failures on macos. The reason behind the failures was that the patch
inadvertenly changed the value returned by the host platform from
"macosx" to "darwin". The new version fixes that.

Original commit message was:

The decision which categories are relevant for a particular test run
happen very early in the test setup process. They use the SBPlatform
object to determine which categories should be skipped. The platform
object created for this purpose transcends individual test runs.

This setup is not compatible with the direction discussed in
<https://discourse.llvm.org/t/multiple-platforms-with-the-same-name/59594>
-- when platform objects are tied to a specific (SB)Debugger, they need
to be created alongside it, which currently happens in the test setUp
method.

This patch is the first step in that direction -- it rewrites the
category skipping logic to avoid depending on a global SBPlatform
object. Fortunately, the skipping logic is fairly simple (and I believe
it outght to stay that way) and mainly consists of comparing the
platform name against some hardcoded lists. This patch bases this
comparison on the platform name instead of the os part of the triple (as
reported by the platform).

Differential Revision: https://reviews.llvm.org/D121605

show more ...

Revert "[lldb/test] Make category-skipping logic "platform"-independent"

This reverts commit dddf4ce034a8e06cc1351492dceece3fa2344c14. It breaks
a couple of tests on macos.

[lldb/test] Make category-skipping logic "platform"-independent

The decision which categories are relevant for a particular test run
happen very early in the test setup process. They use the SBPlatf

[lldb/test] Make category-skipping logic "platform"-independent

The decision which categories are relevant for a particular test run
happen very early in the test setup process. They use the SBPlatform
object to determine which categories should be skipped. The platform
object created for this purpose transcends individual test runs.

This setup is not compatible with the direction discussed in
<https://discourse.llvm.org/t/multiple-platforms-with-the-same-name/59594>
-- when platform objects are tied to a specific (SB)Debugger, they need
to be created alongside it, which currently happens in the test setUp
method.

This patch is the first step in that direction -- it rewrites the
category skipping logic to avoid depending on a global SBPlatform
object. Fortunately, the skipping logic is fairly simple (and I believe
it outght to stay that way) and mainly consists of comparing the
platform name against some hardcoded lists. This patch bases this
comparison on the platform name instead of the os part of the triple (as
reported by the platform).

Differential Revision: https://reviews.llvm.org/D121605

show more ...

[lldb/test] Ensure launched processes are ready to be attached

Linux systems can be configured (and most of them are configured that
way) to disable attaching to unrelated processes, /unless/ those

[lldb/test] Ensure launched processes are ready to be attached

Linux systems can be configured (and most of them are configured that
way) to disable attaching to unrelated processes, /unless/ those
processes explicitly allow that.

Our test inferiors do that by explicitly calling prctl(PR_SET_PTRACER,
PR_SET_PTRACER_ANY) (a.k.a., lldb_enable_attach). This requires
additional synchronization to ensure that the test does not attempt
attach before that statement is executed.

This is working fine (albeit cumbersome) for most tests but
TestGdbRemoteAttachWait is special in that it wants to start the
inferior _after_ issuing the attach request. This means that the usual
synchronization method does not work.

This patch introduces a different solution -- enable attaching in the
test harness, before the process is launched. Besides fixing this
problem, this is also better because it avoids the need to add special
code to each attach test (which is a common error).

One gotcha here is that it won't work for remote test suites, as we
don't control launching there. However, we could add a similar option to
lldb-platform, or require that lldb-platform itself is started with
attaching enabled. At that point we could delete all lldb_enable_attach
logic.

show more ...

[lldb] Add Apple simulator platforms to lldbplatform.py

This just adds the simulator platforms to the lldbplatform enumerations
and the respective test decorator.

The platform names for the simulat

[lldb] Add Apple simulator platforms to lldbplatform.py

This just adds the simulator platforms to the lldbplatform enumerations
and the respective test decorator.

The platform names for the simulator are just the SDK names since D85537, so
that's why we are not using LLDB's usual platform names here (e.g., SDK =
"iphonesimulator" vs LLDB platform ="ios-simulator").

Also removes the duplicate platform enumaration in lldbplatformutil.py.

Reviewed By: JDevlieghere

Differential Revision: https://reviews.llvm.org/D89694

show more ...