trace: do not define fn in SPDK_TRACE_REGISTER_FN

Newer lcov try to detect issues with function definitions, and throw
warnings if function definitions have overlapping lines. Since
SPDK_TRACE_REGIS

trace: do not define fn in SPDK_TRACE_REGISTER_FN

Newer lcov try to detect issues with function definitions, and throw
warnings if function definitions have overlapping lines. Since
SPDK_TRACE_REGISTER_FN defines both the constructor function and
the trace function, it treats this as an overlap.

Note this is the only one of our REGISTER functions in SPDK that
effectively defines two functions in the same macro.

So change the macro so that the user actually defines the
trace function instead.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: Ib9d0d1af7674c5ff9b65c12c1e4a5b05e6e110f5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24836
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot

show more ...

module/accel/dsa: add DIX Verify

DIX Verify is not defined as a standard operation with its own OPCODE
and descriptors inside DSA module. This means we cannot submit it as
is to the hardware.
To int

module/accel/dsa: add DIX Verify

DIX Verify is not defined as a standard operation with its own OPCODE
and descriptors inside DSA module. This means we cannot submit it as
is to the hardware.
To introduce support for DIX Verify use DIX Generate on available
source buffer and then compare the resulting protection information
with the one provided inside original metadata buffer.

Change-Id: Ia427e745cfcc13e9e615bf28cce452be415b2acf
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Signed-off-by: Slawomir Ptak <slawomir.ptak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24342
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lib/idxd: add DIX generate

Add spdk_idxd_submit_dix_generate() function to submit DIX generate
operation to DSA.
Additionally, enable support for this operation in accel_dsa module.

Change-Id: I99e

lib/idxd: add DIX generate

Add spdk_idxd_submit_dix_generate() function to submit DIX generate
operation to DSA.
Additionally, enable support for this operation in accel_dsa module.

Change-Id: I99e7c6d0441599fbac092e059ed6394882c51fb9
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Signed-off-by: Slawomir Ptak <slawomir.ptak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24312
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

env: add spdk_env_get_numa_id()

This will effectively replace spdk_env_get_socket_id(), which is
marked obsolete as part of this patch.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id:

env: add spdk_env_get_numa_id()

This will effectively replace spdk_env_get_socket_id(), which is
marked obsolete as part of this patch.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I5d39e5e1b98e07f709b14c86382e59ea76584def
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24608
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <ben@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

accel/dsa: DIF verify copy DSA implementation

Extend the module API with DIF Verify-copy operation support.

Change-Id: I24ed2e7b048f612560a7c39aebb5bd44b8210e98
Signed-off-by: Krzysztof Sprzaczkows

accel/dsa: DIF verify copy DSA implementation

Extend the module API with DIF Verify-copy operation support.

Change-Id: I24ed2e7b048f612560a7c39aebb5bd44b8210e98
Signed-off-by: Krzysztof Sprzaczkowski <krzysztof.sprzaczkowski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23035
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

trace: rename owner to owner_type

We will be adding the real concept of an "owner" in upcoming patches. So change
the existing "owner" to "owner_type" which more accurate reflects its meaning.
For e

trace: rename owner to owner_type

We will be adding the real concept of an "owner" in upcoming patches. So change
the existing "owner" to "owner_type" which more accurate reflects its meaning.
For example, an "owner_type" can be OWNER_BDEV, but an actual "owner" would
be some specific bdev identified by its name.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: Ieb25eb5a574a88b3812291f2345796e5bb66a54f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22645
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lib/accel: fallback when no devices during module_init()

Related to #3266

During discussion in issue above, it came up that if
dpdk_compressdev and dpdk_cryptodev detects no devices
they are still

lib/accel: fallback when no devices during module_init()

Related to #3266

During discussion in issue above, it came up that if
dpdk_compressdev and dpdk_cryptodev detects no devices
they are still assigned to the relevant accel ops.

Rather than explicitly failing just due to lack of
the devices, each accel module can report -ENODEV.
In such case the module will not be used, but that
error is not propagated to accel framework init.

As a result accel ops assignment will not use such module,
and fallback to available modules.

Reported-by: liuqinfei <lucas.liuqinfei@huawei.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib52dfcb6dc6a1a93b8493b98ffb028f9d9419d6d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22026
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

so_ver: increase all major versions after SPDK 24.01

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to preve

so_ver: increase all major versions after SPDK 24.01

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.

Increasing major so version after LTS, allows the future releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.

This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components

Due to patch below being introduced after v24.01 code freeze,
bdev lib version should not be increased:
61623c5ca9 bdev: add spdk_bdev_io padding in place of io_submit_ch

Short reference to how the versions were changed:
MAX=$(git grep "SO_VER := " | cut -d" " -f 3 | sort -ubnr | head -1)
for((i=$MAX;i>0;i-=1)); do find . -name "Makefile" -exec \
sed -i -e "s/SO_VER := $i\$/SO_VER := $(($i+1))/g" {} +; done
find . -name "Makefile" -exec \
sed -i -e "s/SO_MINOR := .*/SO_MINOR := 0/g" {} +

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I0166d6be6c17bbbf985d48b5dfcb36f1d4af1b48
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21582
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lib/idxd: DIF insert DSA implementation

Extend the IDXD API with DIF Insert operation support.

The DIF Insert operation is used to add Data Integrity Fields (DIF)
when the source data does not cont

lib/idxd: DIF insert DSA implementation

Extend the IDXD API with DIF Insert operation support.

The DIF Insert operation is used to add Data Integrity Fields (DIF)
when the source data does not contain them. When performing
a DIF Insert operation, the device performs the following actions
on each block of source data:
- Calculate the Guard Tag.
- Combine the Guard Tag, Application Tag, and Reference Tag into a
DIF value.
- Write the source data to the destination and appending the DIF value.
- Update the Application Tag and Reference Tag for the next block of
data, based on the DIF Flags.

Change-Id: I945dc3a7d6ba7c99241d0bd2b007c6bd24d92c8d
Signed-off-by: Krzysztof Sprzaczkowski <krzysztof.sprzaczkowski@intel.com>
Signed-off-by: Slawomir Ptak <slawomir.ptak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19854
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lib/idxd: DIF check DSA implementation

Extend the IDXD API with DIF Check operation support.

The DIF Check operation is used to check the correctness of
the Data Integrity Fields (DIF) field in the

lib/idxd: DIF check DSA implementation

Extend the IDXD API with DIF Check operation support.

The DIF Check operation is used to check the correctness of
the Data Integrity Fields (DIF) field in the source data.
When performing a DIF Check operation, the device performs
the following actions on each block of source data and
the associated DIF:
- Calculate the Guard Tag and compare it to the Guard Tag
field in the source DIF value.
- Verify the Application Tag and Reference Tag
in the source DIF value.
- Update the Application Tag and Reference Tag for the next block
of data, based on the DIF Flags.

Change-Id: I858700f8a370fdccd92115ebaa91def85540cb34
Signed-off-by: Krzysztof Sprzaczkowski <krzysztof.sprzaczkowski@intel.com>
Signed-off-by: Slawomir Ptak <slawomir.ptak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19779
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

accel: Use STAILQ to link tasks

That is the first patch in the series to
make accel structures more cache friendly
Size of the task structure before and after
all changes:
before:
/* size: 384, cach

accel: Use STAILQ to link tasks

That is the first patch in the series to
make accel structures more cache friendly
Size of the task structure before and after
all changes:
before:
/* size: 384, cachelines: 6, members: 23 */
after:
/* size: 176, cachelines: 3, members: 20 */

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I85a3a528d427b33cc0e45a44498c0c1930fc7629
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21119
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Ben Walker <ben@nvidia.com>

show more ...

accel/dsa: separate task submission from queued tasks

DSA can queue up submitted accel task when spdk_idxd_submit_*()
fails with EBUSY, for example when there are no more free
batches on the channel

accel/dsa: separate task submission from queued tasks

DSA can queue up submitted accel task when spdk_idxd_submit_*()
fails with EBUSY, for example when there are no more free
batches on the channel.

After idxd_poll() processes currently outstanding commands,
queued tasks are then proceeded using the same function as
any new task (dsa_submit_tasks()).
The issue is with the queue_tasks label in dsa_submit_tasks(),
when not all entries from the queued_tasks list can be submitted.
Loop there basically rewrites whole queued_tasks list one by one.
With large enough queue depth a considerable portion of time
was spent rewriting that list.

This might have been an omission or was done to consolidate
single task and queued task submission in single function.

To address this, both cases are now separate. Simplifying
single task submission. Meanwhile queued tasks are removed
from list as they are submitted. This way avoiding
rewriting the list.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I9162fe01c2d4ee406cf9eb5ec361c6bfd573c831
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21186
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot

show more ...

module/accel: Prevent double RPC call causes app crash (dsa_scan_accel_module)

Prevent double RPC call causes app crash for dsa_scan_accel_module

The RPC dsa_scan_accel_module method may be perform

module/accel: Prevent double RPC call causes app crash (dsa_scan_accel_module)

Prevent double RPC call causes app crash for dsa_scan_accel_module

The RPC dsa_scan_accel_module method may be performed to use
the DSA Module. It should be called before starting the framework,
but only the first call should take effect, next calls should
provide relevant information for a user. In this fix will be an RPC
error code with information that 'the operation is already in progress'.

Fixes issue #3174

Change-Id: I4dcf75afcf0987b68a03722a10fefb3a67ccd02d
Signed-off-by: Krzysztof Sprzaczkowski <krzysztof.sprzaczkowski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/20786
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot

show more ...

accel/idxd: fix off by one device selection

Selecting device iterated over all devices one too many times.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Icc66e9843af6d51946f

accel/idxd: fix off by one device selection

Selecting device iterated over all devices one too many times.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Icc66e9843af6d51946f6af3679d56b88d5b753c9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21012
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>

show more ...

accel: add the spdk_ prefix to accel_opcode

This enum is part of the public interface, so it should have the spdk_
prefix.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie1d4a

accel: add the spdk_ prefix to accel_opcode

This enum is part of the public interface, so it should have the spdk_
prefix.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie1d4aee707ede70c7f181561e1002ce443628ca4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19685
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@gmail.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>

show more ...

accel: add debug log about initialzation on generic layer

With that notice logs are removed from specific modules.

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ibd565f503a6cbb5fd

accel: add debug log about initialzation on generic layer

With that notice logs are removed from specific modules.

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ibd565f503a6cbb5fd99f163b42ce2ef4f68289a9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18256
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

accel/dsa: Do not register module automatically

Register the module when a dedicated RPC to enable dsa
module is called. With this change this module is not
added to a global list and isn't used at

accel/dsa: Do not register module automatically

Register the module when a dedicated RPC to enable dsa
module is called. With this change this module is not
added to a global list and isn't used at all.

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Id7594e8d69134c987050d51f5d951df786702ded
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18177
Community-CI: Mellanox Build Bot
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>

show more ...

accel: move accel_module.h to include/spdk

This file should be external to enable out-of-tree accel modules.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2e973d0e88d7145d0fc

accel: move accel_module.h to include/spdk

This file should be external to enable out-of-tree accel modules.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2e973d0e88d7145d0fc9714f56db48486b00f3b7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17419
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

lib/accel: removed references to ACCEL_FLAG_PERSISTENT

- Removed ACCEL_FLAG_PERSISTENT and related code

Change-Id: Icc8890ad7143fab4bd4b544e82acdf9b84c0a1eb
Signed-off-by: Marcin Spiewak <marcin.sp

lib/accel: removed references to ACCEL_FLAG_PERSISTENT

- Removed ACCEL_FLAG_PERSISTENT and related code

Change-Id: Icc8890ad7143fab4bd4b544e82acdf9b84c0a1eb
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16951
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...