env: add SPDK_ENV_NUMA_ID_ANY and replace socket_id with numa_id

We will try to avoid further proliferation of "SOCKET_ID" to refer
to a NUMA socket ID moving forward, and just use "NUMA_ID" to avoi

env: add SPDK_ENV_NUMA_ID_ANY and replace socket_id with numa_id

We will try to avoid further proliferation of "SOCKET_ID" to refer
to a NUMA socket ID moving forward, and just use "NUMA_ID" to avoid
confusion with TCP sockets.

Change all of the existing in-tree SPDK_ENV_SOCKET_ID_ANY uses to
SPDK_ENV_NUMA_ID_ANY, but keep the old #define around, at least for
now. Also change all 'socket_id' parameters to 'numa_id'.

We still have spdk_env_get_socket_id(), we will need to keep this
but next patch will add spdk_env_get_numa_id().

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: Idc31c29e32b708c24d88f9c6fecaf9a99e34ba1e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24607
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <ben@nvidia.com>
Community-CI: Mellanox Build Bot

show more ...

env: add opts_size to spdk_env_opts

This will help with ABI compatibility as new fields get added
to this structure.

User is responsible for setting opts_size. Major version has been
bumped, so use

env: add opts_size to spdk_env_opts

This will help with ABI compatibility as new fields get added
to this structure.

User is responsible for setting opts_size. Major version has been
bumped, so users will need to rebuild against latest code, if they
do not set opts_size we will just treat that as if the structure
ends right before opts_size. We will also print an ERRLOG to warn
user to set opts_size appropriately.

As part of this patch, update all in-tree callers of
spdk_env_opts_init() to set the opts_size.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: If2611eccc8b3c43e1dc1654028d000a8a12a544e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23940
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz@tzawadzki.com>

show more ...

accel: remove flags

They were unused, deprecated, and scheduled for removal in v24.05.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I48e7e59df56cf2368f23e925360644665948585a

accel: remove flags

They were unused, deprecated, and scheduled for removal in v24.05.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I48e7e59df56cf2368f23e925360644665948585a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22950
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot

show more ...

nvme: add raw field for controller attributes

Add raw field for controller attributes and change it to union from
structure. Update the respective location of code where its being used.

Change-Id:

nvme: add raw field for controller attributes

Add raw field for controller attributes and change it to union from
structure. Update the respective location of code where its being used.

Change-Id: I56278daba9ab947bc5f49271d8107e097abb3576
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22788
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

apps: set NULL as RPC address in some apps

Set opts.rpc_addr to NULL in SPDK applications that usually do not
benefit from starting RPC interface. These include programs in app,
examples and test di

apps: set NULL as RPC address in some apps

Set opts.rpc_addr to NULL in SPDK applications that usually do not
benefit from starting RPC interface. These include programs in app,
examples and test directories.

Some of the tests now require that we explicitly set RPC address,
because some apps no longer use the default SPDK address.

Change-Id: I417a3a756581f44c711257e6d2408d933b40386a
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22634
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot

show more ...

isal: If the user builds with shared libraries, use ISA-L shared
libraries

Change-Id: I6e5882a2d0347971e99bbed26a2291cb3361800b
Signed-off-by: Ben Walker <ben@nvidia.com>
Reviewed-on: https://review

isal: If the user builds with shared libraries, use ISA-L shared
libraries

Change-Id: I6e5882a2d0347971e99bbed26a2291cb3361800b
Signed-off-by: Ben Walker <ben@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21490
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

test/external_code: Use an installed SPDK in this test

External modules will be built against an installed SPDK, so change the
test to do a make install.

Change-Id: I4d9d2da4a4024dca1a96bdeea182ebd

test/external_code: Use an installed SPDK in this test

External modules will be built against an installed SPDK, so change the
test to do a make install.

Change-Id: I4d9d2da4a4024dca1a96bdeea182ebd8ac91e130
Signed-off-by: Ben Walker <ben@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21602
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

accel: Use STAILQ to link tasks

That is the first patch in the series to
make accel structures more cache friendly
Size of the task structure before and after
all changes:
before:
/* size: 384, cach

accel: Use STAILQ to link tasks

That is the first patch in the series to
make accel structures more cache friendly
Size of the task structure before and after
all changes:
before:
/* size: 384, cachelines: 6, members: 23 */
after:
/* size: 176, cachelines: 3, members: 20 */

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I85a3a528d427b33cc0e45a44498c0c1930fc7629
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21119
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Ben Walker <ben@nvidia.com>

show more ...

env_dpdk: initialize OpenSSL library

Call OpenSSL library initialization explicitly.

Change-Id: I9c283bcd59cb0c1ebacaca0ac43db6024276acb5
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>

env_dpdk: initialize OpenSSL library

Call OpenSSL library initialization explicitly.

Change-Id: I9c283bcd59cb0c1ebacaca0ac43db6024276acb5
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/20665
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>

show more ...

external_code/accel: Add external accel driver

Example implementation of accel driver. This change includes the accel driver itself and application to test it.

Change-Id: Ic4615cb45f1dc059ff5b9adf7

external_code/accel: Add external accel driver

Example implementation of accel driver. This change includes the accel driver itself and application to test it.

Change-Id: Ic4615cb45f1dc059ff5b9adf7f7a5cc29ee5f81b
Signed-off-by: Eugene Kobyak <eugene.kobyak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/20458
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>

show more ...

external_code/accel: Add external accel module

Example implementation of accel module. This change includes the module itself and also an application to test all supported operations.

Change-Id: I0

external_code/accel: Add external accel module

Example implementation of accel module. This change includes the module itself and also an application to test all supported operations.

Change-Id: I029f877f9ec9a90ceaf5207e4213bbad9b7d2ccd
Signed-off-by: Eugene Kobyak <eugene.kobyak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/20340
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

test/external_code: set iocqes/iosqes during init

Newer versions of qemu (v8.0.0) have these values cleared, so if we
don't set them, controller initialization will fail.

Fixes #3017

Signed-off-by

test/external_code: set iocqes/iosqes during init

Newer versions of qemu (v8.0.0) have these values cleared, so if we
don't set them, controller initialization will fail.

Fixes #3017

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5c5698b1368e095fe3fc9b39e7f6081f7e013c88
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18087
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>

show more ...

test/external_code: check csts.cfs during initialization

The regular NVMe driver also checks this bit, so we should also do that
in the external_code tests.

Signed-off-by: Konrad Sztyber <konrad.sz

test/external_code: check csts.cfs during initialization

The regular NVMe driver also checks this bit, so we should also do that
in the external_code tests.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I74443d743a9ac0fa5b38cf7e6ee83ecd223584d6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18086
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

test/external_code: set default include/lib dirs

This makes it a lot easier to build these tests.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I969e3dada5d8c48e6d089a1c18c109

test/external_code: set default include/lib dirs

This makes it a lot easier to build these tests.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I969e3dada5d8c48e6d089a1c18c109f0c06e26c1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18085
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>

show more ...

tests: Run make clean only when SPDK was configured

Otherwise the cmd fails due to the lack of config.mk. This forces
autopackage.sh, test_make.sh, etc. to be executed in the environment
where SPDK

tests: Run make clean only when SPDK was configured

Otherwise the cmd fails due to the lack of config.mk. This forces
autopackage.sh, test_make.sh, etc. to be executed in the environment
where SPDK was already built which shouldn't be really a hard
requirement as they always re-build the SPDK for their own purposes.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ie3971acbf354734ed9c5c72d49bb93fc2ccc45f8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15743
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>

show more ...

external_bdev_passthru: Use bdev_unregister_by_name() to delete a passthru bdev

spdk_bdev_unregister_by_name() is safer than spdk_bdev_get_by_name() +
spdk_bdev_unregister(). It is ensured that the

external_bdev_passthru: Use bdev_unregister_by_name() to delete a passthru bdev

spdk_bdev_unregister_by_name() is safer than spdk_bdev_get_by_name() +
spdk_bdev_unregister(). It is ensured that the specified callback is
executed after unwinding stack for spdk_bdev_unregister_by_name().

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I5cc1e88a7e389ab173ef1195bd817d2aac6a70d1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16556
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

module/bdev: Use error_response() rather than bool_response(false) for JSON RPCs

For JSON RPC, boolean response with false value may not be regarded as error.
Previously many cases were replaced to

module/bdev: Use error_response() rather than bool_response(false) for JSON RPCs

For JSON RPC, boolean response with false value may not be regarded as error.
Previously many cases were replaced to use
spdk_jsonrpc_send_error_response() explicitly. However, this pattern did
not become established and new bdev modules used
spdk_jsonrpc_send_bool_response() with false.

Once again, replace remaining cases in this patch.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ie8858bc8ff7c36bc3a829977044a91d459db76f2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16555
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

accel: Add crypto operation support

Add functions to submit encrypt/decrypt operations
Add RPCS to register and dump crypto keys
Software accel module uses isa-l_crypto AEX_XTS
functionality

Signed

accel: Add crypto operation support

Add functions to submit encrypt/decrypt operations
Add RPCS to register and dump crypto keys
Software accel module uses isa-l_crypto AEX_XTS
functionality

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Iecf0e9913edf11ab85171d0fa467a2a62dfff984
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14858
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <qun.wan@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>

show more ...

add (c) and SPDX header to bash files as needed

per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --forma

add (c) and SPDX header to bash files as needed

per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date default <file> | tail -1

and then pull just the year from the result.

Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc)

For intel copyrights added, --follow and -C95% were used.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I2ef86976095b88a9bf5b1003e59f3943cd6bbe4c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15209
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...