History log of /spdk/scripts/bpf/trace.py (Results 1 – 24 of 24)
Revision (<<< Hide revision tags) (Show revision tags >>>) Date Author Comments
Revision tags: v24.05, v24.09-pre, v24.05-rc1
# a6e5d032 02-Apr-2024 Jim Harris <jim.harris@samsung.com>

trace: rename trace_event's poller_id to owner_id

"poller_id" was sort of a very early and crude attempt at associating
trace events with who generated them. The new "owner" concept is replacing
tha

trace: rename trace_event's poller_id to owner_id

"poller_id" was sort of a very early and crude attempt at associating
trace events with who generated them. The new "owner" concept is replacing
that, so just rename the existing "poller_id" to "owner_id".

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: Icd61e39c5225030690ef0238cd0b3a90846fcc6b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22647
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...


# 26d44a12 29-Mar-2024 Jim Harris <jim.harris@samsung.com>

trace: rename owner to owner_type

We will be adding the real concept of an "owner" in upcoming patches. So change
the existing "owner" to "owner_type" which more accurate reflects its meaning.
For e

trace: rename owner to owner_type

We will be adding the real concept of an "owner" in upcoming patches. So change
the existing "owner" to "owner_type" which more accurate reflects its meaning.
For example, an "owner_type" can be OWNER_BDEV, but an actual "owner" would
be some specific bdev identified by its name.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: Ieb25eb5a574a88b3812291f2345796e5bb66a54f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22645
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...


# fc04b134 20-Mar-2024 Jim Harris <jim.harris@samsung.com>

trace: increase SPDK_TRACE_MAX_LCORE to 1024

We limited this to 128 originally since that was aligned with the 128 max
lcores supported by DPDK on x86.

But 128 isn't a suitable maximum anymore. SPD

trace: increase SPDK_TRACE_MAX_LCORE to 1024

We limited this to 128 originally since that was aligned with the 128 max
lcores supported by DPDK on x86.

But 128 isn't a suitable maximum anymore. SPDK trace files are also dynamic
now, meaning we don't allocate trace buffers for unused lcores. There is a
small amount of space that we allocate per-lcore for thread names, but this
pales in comparison to the size of a typical trace buffer.

So bump the number to 1024. This will allocate an extra 14KB of data per
trace file ((1024 - 128) * 16B) for thread names, but each core's trace
buffer by default is 1MB so this seems like a minor amount of additional
overhead.

Fixes issue #3301.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I87e656fc7c8b2002e90d71f6eb11fb4699582f63
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22418
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>

show more ...


# 6da17199 09-Feb-2024 Jim Harris <jim.harris@samsung.com>

trace: add size field to spdk_trace_file

This is more straightforward than using an extra entry in the
lcore_history_offsets array to store the file size. This will also make
more sense if/when we a

trace: add size field to spdk_trace_file

This is more straightforward than using an extra entry in the
lcore_history_offsets array to store the file size. This will also make
more sense if/when we add other types of variable sized data to the trace
files in the future.

The bulk of the changes here are in the trace_record app. We now need to
write out the file_size explicitly to the out_fd.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I3f22359b23ae5915146142b76e38924568782334
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21791
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...


# 23dbcec5 09-Feb-2024 Jim Harris <jim.harris@samsung.com>

trace: merge spdk_trace_flags into spdk_trace_file

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I8a070f986825de2470d5873eaeed3849d36bf1af
Reviewed-on: https://review.spdk.io/gerrit/

trace: merge spdk_trace_flags into spdk_trace_file

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I8a070f986825de2470d5873eaeed3849d36bf1af
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21790
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...


Revision tags: LTS, v24.01, v24.05-pre, v24.01-rc1, v23.09, v24.01-pre, v23.09-rc1, v23.05, v23.09-pre, v23.01.1
# 7c30df4e 21-Apr-2023 Jim Harris <james.r.harris@intel.com>

usdt: add SPDK_DTRACE_PROBE variants that don't collect ticks

While userspace probes have a high overhead when enabled due
to the trap, it is still cleaner and slightly more efficient
to not have al

usdt: add SPDK_DTRACE_PROBE variants that don't collect ticks

While userspace probes have a high overhead when enabled due
to the trap, it is still cleaner and slightly more efficient
to not have all of the SPDK_DTRACE_PROBE macros implicitly
capture the tsc counter as an argument.

So rename the existing SPDK_DTRACE_PROBE macros to
SPDK_DTRACE_PROBE_TICKS, and create new SPDK_DTRACE_PROBE
macros without the implicit ticks argument.

Note this does cause slight breakage if there is any
out-of-tree code that using SPDK_DTRACE_PROBE previously,
and programs written against those probes would need to
adjust their arguments. But the likelihood of such code
existing is practically nil, so I'm just renaming the
macros to their ideal state.

All of the nvmf SPDK_DTRACE_PROBE calls are changed to
use the new _TICKS variants. The event one is left
without _TICKS - we have no in-tree scripts that use
the tsc for that event.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Icb965b7b8f13c23d671263326029acb88c82d9df
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17669
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...


Revision tags: v23.01, v23.05-pre, v23.01-rc1
# 555ca7ad 17-Dec-2022 Mike Gerdts <mgerdts@nvidia.com>

rpc: fix python style

A bunch of python files in rpc and scripts directories were missing two
blank lines after the opening comments. python3-pycodestyle-2.5.0-2.el8
was unhappy with this

Signed-of

rpc: fix python style

A bunch of python files in rpc and scripts directories were missing two
blank lines after the opening comments. python3-pycodestyle-2.5.0-2.el8
was unhappy with this

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Iee7f2bd4f0ddd96b2be89949a3aa324b2f9ea43a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16003
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...


Revision tags: v22.01.2
# 17538bdc 02-Nov-2022 paul luse <paul.e.luse@intel.com>

add (c) and SPDX header to python files as needed

per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --for

add (c) and SPDX header to python files as needed

per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date default <file> | tail -1

and then pull just the year from the result.

Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc)

Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line. Simply there for
consistency.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I6cd3f18d1b469d5ef249d26ddb2923ca6b970bd4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15208
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...


Revision tags: v22.09, v23.01-pre, v22.09-rc1
# e36f0d36 10-Aug-2022 Jim Harris <james.r.harris@intel.com>

nvme/pcie, nvme/tcp: add cb_arg context tracepoint argument

This allows mapping an nvme_request back to the
nvme_bdev_io.

This requires bumping up the max number of arguments per
tracepoint. 5 was

nvme/pcie, nvme/tcp: add cb_arg context tracepoint argument

This allows mapping an nvme_request back to the
nvme_bdev_io.

This requires bumping up the max number of arguments per
tracepoint. 5 was previously chosen as max since it
exactly fit in 64 bytes (1 cacheline) when all
arguments were stored as uint64_t, but now that we
support uint32_t arguments we can afford extra
arguments when some of them are uint32_t. I've
bumped it to 8 so we can avoid having to touch
this value multiple times if we find some cases
where we need 7 or 8 args.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie2ef5e59d10549860b47542e68c1c34efa63047f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13995
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>

show more ...


Revision tags: v22.05, v22.09-pre, v22.05-rc1, v22.01.1, v22.01, v22.01-rc1
# 1ff3715d 25-Nov-2021 Josh Soref <jsoref@gmail.com>

spelling: scripts

Part of #2256

* available
* classes
* coalesce
* configuration
* controller
* currently
* dependencies
* different
* displaying
* does not
* environment
* explicitly
* hugepages
*

spelling: scripts

Part of #2256

* available
* classes
* coalesce
* configuration
* controller
* currently
* dependencies
* different
* displaying
* does not
* environment
* explicitly
* hugepages
* ignoring
* inflight
* initialize
* parameters
* priority
* properties
* recovery
* subsystem
* subsystems
* template
* the same
* underlying
* usable
* values

Change-Id: Ibc8567af288c9f4641563835e16bf88949ba6a71
Signed-off-by: Josh Soref <jsoref@gmail.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10408
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...


Revision tags: v21.10, v21.10-rc1
# 3c271d5e 18-Oct-2021 Krzysztof Karas <krzysztof.karas@intel.com>

scripts/bpf/trace.py: enable printing relations for NativeProvider

Currently the only way to see the relations, while using trace.py script,
is to feed it a JSON formatted file. This patch allows tr

scripts/bpf/trace.py: enable printing relations for NativeProvider

Currently the only way to see the relations, while using trace.py script,
is to feed it a JSON formatted file. This patch allows trace.py
to do the same for the for the binary trace files.

Change-Id: I7237896ede608080d3bec896a43586e34c297b04
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9906
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...


# 5594c7c8 27-Sep-2021 Krzysztof Karas <krzysztof.karas@intel.com>

scripts/bpf/trace.py: print relation information

Make use of the changes introduced in previous patches while
printing trace information.

Change-Id: I69edadfb4a7d34fa9c8c1c520b083372317cabbb
Signed

scripts/bpf/trace.py: print relation information

Make use of the changes introduced in previous patches while
printing trace information.

Change-Id: I69edadfb4a7d34fa9c8c1c520b083372317cabbb
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9638
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>

show more ...


# 70c17160 19-Oct-2021 Krzysztof Karas <krzysztof.karas@intel.com>

trace: enable adding relations between traces

Currently we do not have any way to connect traces from different
modules in SPDK. This change modifies our trace library
and app/trace to handle adding

trace: enable adding relations between traces

Currently we do not have any way to connect traces from different
modules in SPDK. This change modifies our trace library
and app/trace to handle adding relations between trace points
and a trace object.

Additionally this patch adds classes and fields to structs
inside trace.py to prepare it for future patches implementing
printing relation information.

Change-Id: Ia09d01244d923957d589fd37e6d4c98f9f7bbd07
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9620
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>

show more ...


# 6d4ad9a2 25-Aug-2021 Krzysztof Karas <krzysztof.karas@intel.com>

scripts/bpf/trace.py: add tcp events

Add tcp events changed in previous patch to the trace.py for
enhanced event information.

Change-Id: I8f176905a8283ba1b588d95bdbc075c38677acbe
Signed-off-by: Krz

scripts/bpf/trace.py: add tcp events

Add tcp events changed in previous patch to the trace.py for
enhanced event information.

Change-Id: I8f176905a8283ba1b588d95bdbc075c38677acbe
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9307
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@gmail.com>

show more ...


# e89224d1 03-Sep-2021 Konrad Sztyber <konrad.sztyber@intel.com>

scripts/trace: improve line formatting when printing

Changed the way a line is formatted to avoid unnecessary operations.
This slightly (~5-10%) decreases the runtime when the script is executed
on

scripts/trace: improve line formatting when printing

Changed the way a line is formatted to avoid unnecessary operations.
This slightly (~5-10%) decreases the runtime when the script is executed
on large trace files.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5549462d56f866bea99609f746aa53890b98d622
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9442
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>

show more ...


# 364dbc8f 02-Sep-2021 Konrad Sztyber <konrad.sztyber@intel.com>

scripts/trace: suppress KeyboardInterrupt and BrokenPipe exceptions

The stack trace is no longer printed if a user presses <C-c> while the
script is running or pipes its output to tools such as head

scripts/trace: suppress KeyboardInterrupt and BrokenPipe exceptions

The stack trace is no longer printed if a user presses <C-c> while the
script is running or pipes its output to tools such as head or less.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I0951c38d914986b5c9bf2ee98bcd046f0e957ff0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9441
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>

show more ...


# bc71c2e6 02-Sep-2021 Konrad Sztyber <konrad.sztyber@intel.com>

scripts/trace: add build/lib to LD_LIBRARY_PATH before starting

It makes it more convinient for a user to start the script, as it's no
longer necessary to update the library path manually.

Signed-o

scripts/trace: add build/lib to LD_LIBRARY_PATH before starting

It makes it more convinient for a user to start the script, as it's no
longer necessary to update the library path manually.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I0dbeb54cad2585a50162c91f48c5ef199fa133b9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9440
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>

show more ...


# 5d5d9cbb 02-Sep-2021 Konrad Sztyber <konrad.sztyber@intel.com>

scripts/trace: use trace library for parsing traces

Added trace provider based on the trace library. It uses ctypes to
define the trace objects and execute library calls. It significantly
(~2x) sp

scripts/trace: use trace library for parsing traces

Added trace provider based on the trace library. It uses ctypes to
define the trace objects and execute library calls. It significantly
(~2x) speeds up the execution time on large trace files.

The script automatically detects the type of the trace file and will use
the appropriate engine (either JSON or the trace library) using
python-magic for file detection. When piping a file through stdin, JSON
format will be assumed.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Icfa216edee4662714266968eeb72f0097b3779b4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9439
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...


# b14196fa 02-Sep-2021 Konrad Sztyber <konrad.sztyber@intel.com>

scripts/trace: add owner/object type to tracepoint definition

These fields aren't needed by the JSON provider, as the trace entries in
this format already has the owner and object fields filled, but

scripts/trace: add owner/object type to tracepoint definition

These fields aren't needed by the JSON provider, as the trace entries in
this format already has the owner and object fields filled, but it'll be
needed for the provider using the trace library.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I9db974793193fe59bfffc8dd3b975d5343036be9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9438
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>

show more ...


# d1732fce 31-Aug-2021 Konrad Sztyber <konrad.sztyber@intel.com>

scripts/trace: create trace provider abstraction

This abstraction will make it possible to support different sources for
the traces. The current JSON implementation is moved to a provdider and
a ne

scripts/trace: create trace provider abstraction

This abstraction will make it possible to support different sources for
the traces. The current JSON implementation is moved to a provdider and
a new one using the trace_parse library will be added in the following
patches.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I322e7984ffe19ce8b8e1bb551e8339655383623f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9437
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>

show more ...


Revision tags: v21.07, v21.07-rc1
# 597688b2 10-Jun-2021 Konrad Sztyber <konrad.sztyber@intel.com>

scripts/trace: use ijson to parse the traces

Since the trace files can get very large (several GBs), parsing them
using python's json module might require an unfeasible amount of memory,
as it needs

scripts/trace: use ijson to parse the traces

Since the trace files can get very large (several GBs), parsing them
using python's json module might require an unfeasible amount of memory,
as it needs to load the whole file first. The ijson [1] library
provides interfaces for parsing files iteratively, only loading a small
portion of a file at a time.

It requires the input JSON to have the tsc_rate and the definitions of
the tracepoints listed before the tracepoint entries. It's not a big
deal, as this is the way `spdk_trace` generates it, but it's worth
noting, as passing that file through something like `jq -S` might make
it unreadable to the trace script.

[1] https://pypi.org/project/ijson

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I03c0c3fb47091da615a3978b8d63edf4d876b811
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8275
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>

show more ...


# e61fbe91 27-May-2021 Konrad Sztyber <konrad.sztyber@intel.com>

scripts/trace: qpair annotations

This patch adds support for annotating qpairs. The raw pointer values
are replaced with a list of various properties of a qpair collected by
the bpftrace script.

A

scripts/trace: qpair annotations

This patch adds support for annotating qpairs. The raw pointer values
are replaced with a list of various properties of a qpair collected by
the bpftrace script.

A line like this:

`0 2856282624.000 RDMA_REQ_NEW id: r3256 qpair: 0x26ba550`

becomes (the line was only broken here):

`0 2856282502.669 RDMA_REQ_NEW id: r3256 qpair(ptr=0x26ba550, thread=2, qid=0, \
subnqn=nqn.2016-06.io.spdk:cnode1, \
hostnqn=nqn.2014-08.org.nvmexpress:uuid:729783b4ab38485d8d767b7741108a8)`

To annotate a trace, one first needs to enable and record the DTrace
probes:

```
$ scripts/bpf/trace.py --record `pidof spdk_tgt` > bpftraces
^C
```

Of course, the probe events are only recorded when the script is
executing, so in order to generate the annotations properly, it must be
started before the annotated objects are created. For instance, for
NVMeoF, it needs to be running before a connection is made.

After the BPF probes are recored, the traces can be annotated:

```
$ build/bin/spdk_trace -p `pidof spdk_tgt` -s spdk_tgt -j | \
scripts/bpf/trace.py -b bpftraces
```

For now, the script only annotates traces from the rdma module, as it's
the only one with tracpoints recording qpair pointers now, but it could
be extended to support more tracepoints.

Similarly, more objects could be annotated in the future by extending
the `SPDKObject` class and defining additional DTrace probe points.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2225f4b1ab266b41af561b5e7d942411cd0c13c1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8107
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...


# 01ae68f7 27-May-2021 Konrad Sztyber <konrad.sztyber@intel.com>

scripts/trace: parse and generate usdt bpftrace scripts

This patch introduces definitions responsible for generating bpftrace
scripts and parsing its output. That output will be used in subsequent

scripts/trace: parse and generate usdt bpftrace scripts

This patch introduces definitions responsible for generating bpftrace
scripts and parsing its output. That output will be used in subsequent
patches to provide annotations for SPDK traces.

The script has a hardcoded set of probe points that are used to generate
the bpftrace script. They're also checked against the probes present in
code to sanitize them and make sure that they're in sync.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I1b8c95e1a035bd7affed2c44b056828a5da94abd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8106
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...


# b0d3f29a 25-May-2021 Konrad Sztyber <konrad.sztyber@intel.com>

scripts/trace: added script to annotate traces

This script will use the output generated by the spdk_trace app (JSON) and
replace some of the raw pointers with the information gathered through
the D

scripts/trace: added script to annotate traces

This script will use the output generated by the spdk_trace app (JSON) and
replace some of the raw pointers with the information gathered through
the DTrace probes.

For now, this only prints out the traces without doing any annotations,
so the output should be nearly identical to what's printed by
spdk_trace. The annotations will be added in subsequent patches.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I406e8d4fde9cb88b31c38aa96f11b88e607e9436
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8105
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>

show more ...