add new trace BLOB_REQ_SET_START and BLOB_REQ_SET_COMPLETE.

Through the both new trace, the bdev_io trace of lvol and the bdev_io
trace of base bdev of blobstore can be connected in series. Like thi

add new trace BLOB_REQ_SET_START and BLOB_REQ_SET_COMPLETE.

Through the both new trace, the bdev_io trace of lvol and the bdev_io
trace of base bdev of blobstore can be connected in series. Like this:

RDMA_REQ_RDY_TO_EXECUTE id: r31
BDEV_IO_START size: 8 id: i2964 (r31)
BLOB_REQ_SET_START id: a11 (i2964)
BDEV_IO_START size: 8 id: i2965 (a11)
BDEV_NVME_IO_START id: N2957 (i2965)
NVME_PCIE_SUBMIT id: p2959 (N2957)
NVME_PCIE_COMPLETE id: p2959 (N2957)
BDEV_NVME_IO_DONE id: N2957 (i2965)
BDEV_IO_DONE id: i2965 (a11)
BLOB_REQ_SET_COMPLETE id: a11 (i2964)
BDEV_IO_DONE id: i2964 (r31)
RDMA_REQ_EXECUTED id: r31

Change-Id: I607428b094031814627ac4192b9e3defe7ed5337
Signed-off-by: Xupeng Mingtu <jingmamour@gmail.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24130
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

trace: declare g_user_thread_index_start, g_ut_array and g_ut_array_mutex static

Change-Id: Idaaac00b597e349cddb374fd5b2182c13cb0a708
Signed-off-by: Pierre Lestringant <plestringant@kalray.eu>
Revie

trace: declare g_user_thread_index_start, g_ut_array and g_ut_array_mutex static

Change-Id: Idaaac00b597e349cddb374fd5b2182c13cb0a708
Signed-off-by: Pierre Lestringant <plestringant@kalray.eu>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24271
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>

show more ...

trace: fix trace id assertion

Trace id 0 is valid. Likely issue was not hit as for some reason trace
group id 0 is/was not used so far (group id determines base for trace
id, see SPDK_TPOINT_ID).

C

trace: fix trace id assertion

Trace id 0 is valid. Likely issue was not hit as for some reason trace
group id 0 is/was not used so far (group id determines base for trace
id, see SPDK_TPOINT_ID).

Change-Id: Id014288a109206b47a729484bea73e853857c8b2
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23645
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

trace: rename trace_event's poller_id to owner_id

"poller_id" was sort of a very early and crude attempt at associating
trace events with who generated them. The new "owner" concept is replacing
tha

trace: rename trace_event's poller_id to owner_id

"poller_id" was sort of a very early and crude attempt at associating
trace events with who generated them. The new "owner" concept is replacing
that, so just rename the existing "poller_id" to "owner_id".

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: Icd61e39c5225030690ef0238cd0b3a90846fcc6b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22647
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

trace: add concept of "owner" to trace files

An "owner" is now represented by a textual string describing the owner of
a trace event.

A trace file can have up to 16K outstanding owners at any given

trace: add concept of "owner" to trace files

An "owner" is now represented by a textual string describing the owner of
a trace event.

A trace file can have up to 16K outstanding owners at any given time. We need
this many to be able to track all of the various NVMe-oF qpairs, bdevs,
NVMe driver qpairs, accel channels, etc. A library creating trace events
can call spdk_trace_register_owner() with a description string to allocate a
a 16-bit owner_id. This owner_id is then passed as a parameter to
spdk_trace_record().

Libraries can change the description of a registered owner by calling
spdk_trace_owner_set_description(). This can be useful for cases like NVMe-oF,
where where a newly established qpair may only have a source IP/port, but later
can be changed to also add the subsystem nqn once the CONNECT command arrives.

A library is responsible for calling spdk_trace_unregister_owner() once the
associated object is destroyed (i.e. a qpair disconnects or a bdev is
unregistered).

An owner lifetime is tracked by recording the tsc when the owner was
registered. This allows the trace app to handle the case where an owner_id
gets reused during the span of a trace file.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I79c9979a0769c32a85ee646c8237fe8329993c45
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22646
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

trace: add trace_flags_fini()

This will be needed in an upcoming patch.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I8c38d00219e7ba52ddebc2746087b882fd6809bf
Reviewed-on: https://

trace: add trace_flags_fini()

This will be needed in an upcoming patch.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I8c38d00219e7ba52ddebc2746087b882fd6809bf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22876
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

trace: rename owner to owner_type

We will be adding the real concept of an "owner" in upcoming patches. So change
the existing "owner" to "owner_type" which more accurate reflects its meaning.
For e

trace: rename owner to owner_type

We will be adding the real concept of an "owner" in upcoming patches. So change
the existing "owner" to "owner_type" which more accurate reflects its meaning.
For example, an "owner_type" can be OWNER_BDEV, but an actual "owner" would
be some specific bdev identified by its name.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: Ieb25eb5a574a88b3812291f2345796e5bb66a54f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22645
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

trace: change trace_flags_init() to return int

Future patches will add possible error return values to this function, so prep
now by changing this function to return int.

Signed-off-by: Jim Harris

trace: change trace_flags_init() to return int

Future patches will add possible error return values to this function, so prep
now by changing this function to return int.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I139755d780a6ff67a52caf615a3c12ec6c0cf94a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22644
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

trace: make spdk_trace_flags_init() a private function

This does not need to be in public API. So move it internally and rename it
to trace_flags_init().

Signed-off-by: Jim Harris <jim.harris@samsu

trace: make spdk_trace_flags_init() a private function

This does not need to be in public API. So move it internally and rename it
to trace_flags_init().

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I871eace8ba3a028d6d61d9f3b155ae8651a32b66
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22643
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lib/event: shorten opts descriptions' lines

Currently we display some of our app opts descriptions without any
length limit. This means that these lines will wrap only when hitting
the horizontal te

lib/event: shorten opts descriptions' lines

Currently we display some of our app opts descriptions without any
length limit. This means that these lines will wrap only when hitting
the horizontal terminal size. To improve readability and overall
visual experience, make sure the lines do not exceed 100 characters.

Change-Id: If002cba7c4ef4178f11f2bb0e37269c5626852c0
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21999
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot

show more ...

trace: add size field to spdk_trace_file

This is more straightforward than using an extra entry in the
lcore_history_offsets array to store the file size. This will also make
more sense if/when we a

trace: add size field to spdk_trace_file

This is more straightforward than using an extra entry in the
lcore_history_offsets array to store the file size. This will also make
more sense if/when we add other types of variable sized data to the trace
files in the future.

The bulk of the changes here are in the trace_record app. We now need to
write out the file_size explicitly to the out_fd.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I3f22359b23ae5915146142b76e38924568782334
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21791
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

trace: merge spdk_trace_flags into spdk_trace_file

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I8a070f986825de2470d5873eaeed3849d36bf1af
Reviewed-on: https://review.spdk.io/gerrit/

trace: merge spdk_trace_flags into spdk_trace_file

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I8a070f986825de2470d5873eaeed3849d36bf1af
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21790
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

trace: rename spdk_trace_histories to spdk_trace_file.

The new name is obviously a better fit for what this structure has
become.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I91e9

trace: rename spdk_trace_histories to spdk_trace_file.

The new name is obviously a better fit for what this structure has
become.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I91e96bea1d1d507b601e131e222f3181909f35de
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21826
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lib/trace : Display names for user created threads

Names for the user created threads are displayed in the
trace logs.

Change-Id: I0030df05276ab8c3583d10effdfb54bed6d3c28a
Signed-off-by: Krishna Ka

lib/trace : Display names for user created threads

Names for the user created threads are displayed in the
trace logs.

Change-Id: I0030df05276ab8c3583d10effdfb54bed6d3c28a
Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21713
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

so_ver: increase all major versions after SPDK 24.01

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to preve

so_ver: increase all major versions after SPDK 24.01

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.

Increasing major so version after LTS, allows the future releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.

This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components

Due to patch below being introduced after v24.01 code freeze,
bdev lib version should not be increased:
61623c5ca9 bdev: add spdk_bdev_io padding in place of io_submit_ch

Short reference to how the versions were changed:
MAX=$(git grep "SO_VER := " | cut -d" " -f 3 | sort -ubnr | head -1)
for((i=$MAX;i>0;i-=1)); do find . -name "Makefile" -exec \
sed -i -e "s/SO_VER := $i\$/SO_VER := $(($i+1))/g" {} +; done
find . -name "Makefile" -exec \
sed -i -e "s/SO_MINOR := .*/SO_MINOR := 0/g" {} +

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I0166d6be6c17bbbf985d48b5dfcb36f1d4af1b48
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21582
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lib/trace : Unregister tracing support for user created threads

Cleanup the tracing support whenever the user created threads
are exited.

fio_plugin is modified to verify the cleanup of tracing sup

lib/trace : Unregister tracing support for user created threads

Cleanup the tracing support whenever the user created threads
are exited.

fio_plugin is modified to verify the cleanup of tracing support
for the user created threads.

Change-Id: Ifebdf5d76f6cc190bde75d542797ce4455860c1a
Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21304
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lib/trace : Enable tracing support for user created threads

Only the environment created threads have the tracing support.
Added tracing support for the user created threads by reserving
lcore_histo

lib/trace : Enable tracing support for user created threads

Only the environment created threads have the tracing support.
Added tracing support for the user created threads by reserving
lcore_history entries.

fio_plugin is modified to verify the tracing support for the user
created threads.

This allows all SPDK applications to use SPDK tracing,
even if they are not using the SPDK app framework.

Change-Id: Ib7831b30d7de420fcc4a5849f3757733755a5406
Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/20852
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

build: enable Wpointer-arith in selected libraries

There's an organization, called Timerland SIG [1], that uses SPDK's NVMe
driver to implement NVMe Boot Specification under UEFI. The UEFI EDK2
bui

build: enable Wpointer-arith in selected libraries

There's an organization, called Timerland SIG [1], that uses SPDK's NVMe
driver to implement NVMe Boot Specification under UEFI. The UEFI EDK2
build environment uses both gcc and msvc. While SPDK can be built
without any issues with gcc, msvc complains about a few things, one of
which is void pointer arithmetic. So, to make Timberland's effort
easier, Wpointer-arith is enabled in the libraries they're using.

[1] https://github.com/timberland-sig

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ic90330baa4e69ee72f7e040de91893093b96f476
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18529
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

so_ver: increase all major versions

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where

so_ver: increase all major versions

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.

Increasing major so version after LTS, allows the future releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.

Disabled test for increasing SO version without ABI change, as
that is goal of this patch. This check shall be removed with SPDK 23.05
release.
Looks like this was left over from prior LTS, to avoid that
make sure it is only skipped when running against v23.01.x as latest
release.

This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components
- removes suppressions for ABI tests


Short reference to how the versions were changed:
MAX=$(git grep "SO_VER := " | cut -d" " -f 3 | sort -ubnr | head -1)
for((i=$MAX;i>0;i-=1)); do find . -name "Makefile" -exec \
sed -i -e "s/SO_VER := $i\$/SO_VER := $(($i+1))/g" {} +; done
find . -name "Makefile" -exec \
sed -i -e "s/SO_MINOR := .*/SO_MINOR := 0/g" {} +

Change-Id: I3e5681802c0a5ac6d7d652a18896997cd07cc8bf
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16419
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lib/trace: add trace_get_info RPC

Add rpc method trace_get_info to show name of shared memory file,
list of the available trace point groups and mask of the available trace points for each group.

F

lib/trace: add trace_get_info RPC

Add rpc method trace_get_info to show name of shared memory file,
list of the available trace point groups and mask of the available trace points for each group.

Fixes #2747

Signed-off-by: Xinrui Mao <xinrui.mao@intel.com>
Change-Id: I2098283bed454dc46644fd2ca1b9568ab2aea81b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15426
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot

show more ...