Re-merge `A few updates around "transcript"` (#92843) (#94067)

Problematic PR: https://github.com/llvm/llvm-project/pull/92843
Reverted by: https://github.com/llvm/llvm-project/pull/94088

The fi

Re-merge `A few updates around "transcript"` (#92843) (#94067)

Problematic PR: https://github.com/llvm/llvm-project/pull/92843
Reverted by: https://github.com/llvm/llvm-project/pull/94088

The first PR added a test which fails in Linux builds (see the last few
comments there).
This PR contains all the changes in the first PR, plus the fix to the
said test.

---------

Co-authored-by: Roy Shi <royshi@meta.com>

show more ...

Revert "A few updates around "transcript"" (#94088)

Reverts llvm/llvm-project#92843 because it broke some lldb tests:

https://luci-milo.appspot.com/ui/p/fuchsia/builders/toolchain.ci/clang-linux-

Revert "A few updates around "transcript"" (#94088)

Reverts llvm/llvm-project#92843 because it broke some lldb tests:

https://luci-milo.appspot.com/ui/p/fuchsia/builders/toolchain.ci/clang-linux-x64/b8746385730949743489/overview

show more ...

A few updates around "transcript" (#92843)

# Changes

1. Changes to the structured transcript.
1. Add fields `commandName` and `commandArguments`. They will hold the
name and the arguments strin

A few updates around "transcript" (#92843)

# Changes

1. Changes to the structured transcript.
1. Add fields `commandName` and `commandArguments`. They will hold the
name and the arguments string of the expanded/executed command (e.g.
`breakpoint set` and `-f main.cpp -l 4`). This is not to be confused
with the `command` field, which holds the user input (e.g. `br s -f
main.cpp -l 4`).
2. Add field `timestampInEpochSeconds`. It will hold the timestamp when
the command is executed.
3. Rename field `seconds` to `durationInSeconds`, to improve
readability, especially since `timestampInEpochSeconds` is added.
2. When transcript is available and the newly added option
`--transcript` is present, add the transcript to the output of
`statistics dump`, as a JSON array under a new field `transcript`.
3. A few test name and comment changes.

show more ...

Add new Python API `SBCommandInterpreter::GetTranscript()` (#90703)

# Motivation

Currently, the user can already get the "transcript" (for "what is the
transcript", see `CommandInterpreter::Save

Add new Python API `SBCommandInterpreter::GetTranscript()` (#90703)

# Motivation

Currently, the user can already get the "transcript" (for "what is the
transcript", see `CommandInterpreter::SaveTranscript`). However, the
only way to obtain the transcript data as a user is to first destroy the
debugger, then read the save directory. Note that destroy-callbacks
cannot be used, because 1\ transcript data is private to the command
interpreter (see `CommandInterpreter.h`), and 2\ the writing of the
transcript is *after* the invocation of destory-callbacks (see
`Debugger::Destroy`).

So basically, there is no way to obtain the transcript:
* during the lifetime of a debugger (including the destroy-callbacks,
which often performs logging tasks, where the transcript can be useful)
* without relying on external storage

In theory, there are other ways for user to obtain transcript data
during a debugger's life cycle:
* Use Python API and intercept commands and results.
* Use CLI and record console input/output.

However, such ways rely on the client's setup and are not supported
natively by LLDB.


# Proposal

Add a new Python API `SBCommandInterpreter::GetTranscript()`.

Goals:
* It can be called at any time during the debugger's life cycle,
including in destroy-callbacks.
* It returns data in-memory.

Structured data:
* To make data processing easier, the return type is `SBStructuredData`.
See comments in code for how the data is organized.
* In the future, `SaveTranscript` can be updated to write different
formats using such data (e.g. JSON). This is probably accompanied by a
new setting (e.g. `interpreter.save-session-format`).

# Alternatives

The return type can also be `std::vector<std::pair<std::string,
SBCommandReturnObject>>`. This will make implementation easier, without
having to translate it to `SBStructuredData`. On the other hand,
`SBStructuredData` can convert to JSON easily, so it's more convenient
for user to process.

# Privacy

Both user commands and output/error in the transcript can contain
privacy data. However, as mentioned, the transcript is already available
to the user. The addition of the new API doesn't increase the level of
risk. In fact, it _lowers_ the risk of privacy data being leaked later
on, by avoiding writing such data to external storage.

Once the user (or their code) gets the transcript, it will be their
responsibility to make sure that any required privacy policies are
guaranteed.

# Tests

```
bin/llvm-lit -sv ../external/llvm-project/lldb/test/API/python_api/interpreter/TestCommandInterpreterAPI.py
```

```
bin/llvm-lit -sv ../external/llvm-project/lldb/test/API/commands/session/save/TestSessionSave.py
```

---------

Co-authored-by: Roy Shi <royshi@meta.com>
Co-authored-by: Med Ismail Bennani <ismail@bennani.ma>

show more ...

[lldb] Replace assertEquals with assertEqual (NFC) (#82073)

assertEquals is a deprecated alias for assertEqual and has been removed
in Python 3.12. This wasn't an issue previously because we used a

[lldb] Replace assertEquals with assertEqual (NFC) (#82073)

assertEquals is a deprecated alias for assertEqual and has been removed
in Python 3.12. This wasn't an issue previously because we used a
vendored version of the unittest module. Now that we use the built-in
version this gets updated together with the Python version used to run
the test suite.

show more ...

[NFC][Py Reformat] Reformat python files in lldb

This is an ongoing series of commits that are reformatting our Python
code. Reformatting is done with `black` (23.1.0).

If you end up having problem

[NFC][Py Reformat] Reformat python files in lldb

This is an ongoing series of commits that are reformatting our Python
code. Reformatting is done with `black` (23.1.0).

If you end up having problems merging this commit because you have made
changes to a python file, the best way to handle that is to run `git
checkout --ours <yourfile>` and then reformat it with black.

RFC: https://discourse.llvm.org/t/rfc-document-and-standardize-python-code-style

Differential revision: https://reviews.llvm.org/D151460

show more ...

[LLDB] Remove __future__ imports from tests

Not needed now that we require python 3.

Reviewed By: kastiglione, JDevlieghere

Differential Revision: https://reviews.llvm.org/D131761

[lldb][tests] Automatically call compute_mydir (NFC)

Eliminate boilerplate of having each test manually assign to `mydir` by calling
`compute_mydir` in lldbtest.py.

Differential Revision: https://r

[lldb][tests] Automatically call compute_mydir (NFC)

Eliminate boilerplate of having each test manually assign to `mydir` by calling
`compute_mydir` in lldbtest.py.

Differential Revision: https://reviews.llvm.org/D128077

show more ...

[lldb/tests] Removed add_test_categories decorator for python API tests, NFC

There is a .categories file in the python_api directory that makes all nested tests
belong to the category "pyapi". The d

[lldb/tests] Removed add_test_categories decorator for python API tests, NFC

There is a .categories file in the python_api directory that makes all nested tests
belong to the category "pyapi". The decorator is unnecessary for these tests.

show more ...

[lldb][test] Remove symlink for API tests.

Summary: Moves lldbsuite tests to lldb/test/API.

This is a largely mechanical change, moved with the following steps:

```
rm lldb/test/API/testcases
mkdi

[lldb][test] Remove symlink for API tests.

Summary: Moves lldbsuite tests to lldb/test/API.

This is a largely mechanical change, moved with the following steps:

```
rm lldb/test/API/testcases
mkdir -p lldb/test/API/{test_runner/test,tools/lldb-{server,vscode}}
mv lldb/packages/Python/lldbsuite/test/test_runner/test lldb/test/API/test_runner
for d in $(find lldb/packages/Python/lldbsuite/test/* -maxdepth 0 -type d | egrep -v "make|plugins|test_runner|tools"); do mv $d lldb/test/API; done
for d in $(find lldb/packages/Python/lldbsuite/test/tools/lldb-vscode -maxdepth 1 -mindepth 1 | grep -v ".py"); do mv $d lldb/test/API/tools/lldb-vscode; done
for d in $(find lldb/packages/Python/lldbsuite/test/tools/lldb-server -maxdepth 1 -mindepth 1 | egrep -v "gdbremote_testcase.py|lldbgdbserverutils.py|socket_packet_pump.py"); do mv $d lldb/test/API/tools/lldb-server; done
```

lldb/packages/Python/lldbsuite/__init__.py and lldb/test/API/lit.cfg.py were also updated with the new directory structure.

Reviewers: labath, JDevlieghere

Tags: #lldb

Differential Revision: https://reviews.llvm.org/D71151

show more ...