bdev/crypto: support resize of base bdev

Add handling of SPDK_BDEV_EVENT_RESIZE event for crypto bdev.
After obtaining it, resize crypto bdev to base bdev size and
pass RESIZE event furter down the

bdev/crypto: support resize of base bdev

Add handling of SPDK_BDEV_EVENT_RESIZE event for crypto bdev.
After obtaining it, resize crypto bdev to base bdev size and
pass RESIZE event furter down the bdev stack.

Fixes #3549

Change-Id: Ie2827622e39f5ac820fa0372a36fd1a0377c9ebf
Signed-off-by: Vasilii Ivanov <iwanovvvasilij@gmail.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25195
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

bdev/crypto: Report encrypt op memory domain

That allows to propagate accel module memory domain
to the application

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Ic90695ad388e85f0

bdev/crypto: Report encrypt op memory domain

That allows to propagate accel module memory domain
to the application

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Ic90695ad388e85f05e7f6cac69e901f5556256e3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23098
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <ben@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

harmonize spdk_*_get/set_opts()

spdk_bdev_get/set_opts() is careful to check its size argument, so that
it can add options in a backwards-compatible manner. However,
spdk_iobuf_get/set_opts() and sp

harmonize spdk_*_get/set_opts()

spdk_bdev_get/set_opts() is careful to check its size argument, so that
it can add options in a backwards-compatible manner. However,
spdk_iobuf_get/set_opts() and spdk_accel_get/set_opts() both have
slightly different interfaces to the bdev variant, and are less careful.

Make all three variants operate in the same manner instead.

For spdk_iobuf_set_opts(), make all validation consistently return an
error instead of trying to adjust automatically.

Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: I4077a5f1df7039992a556544acdcb1ef379887ae
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22093
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

accel: remove flags

They were unused, deprecated, and scheduled for removal in v24.05.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I48e7e59df56cf2368f23e925360644665948585a

accel: remove flags

They were unused, deprecated, and scheduled for removal in v24.05.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I48e7e59df56cf2368f23e925360644665948585a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22950
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot

show more ...

bdev/crypto: fix misleading var name

Block size is used for the encryption and not the length of
transaction. The same var name is used for both encryption and
decryption now.

Change-Id: I09bc4fb2c

bdev/crypto: fix misleading var name

Block size is used for the encryption and not the length of
transaction. The same var name is used for both encryption and
decryption now.

Change-Id: I09bc4fb2cf560e0fb14af4bb59d5b786045f0657
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22596
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot

show more ...

bdev/crypto: use spdk_bdev_io_complete_base_io_status()

This ensures that users will receive more detailed error information
from the base bdev.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.

bdev/crypto: use spdk_bdev_io_complete_base_io_status()

This ensures that users will receive more detailed error information
from the base bdev.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I398a830f69974511b2840780e8a8b5f44bcb8efd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/20732
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>

show more ...

bdev/crypto: use max_rw_size to limit IO size to large iobuf size

This ensures that IOs that do not exceed large iobuf size won't be split
(which could happen with optimal_io_boundary if an IO cross

bdev/crypto: use max_rw_size to limit IO size to large iobuf size

This ensures that IOs that do not exceed large iobuf size won't be split
(which could happen with optimal_io_boundary if an IO crossed that
boundary).

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I8d28484f7f7992b60c9933fd9091feca5bf3bcd4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/20585
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

bdev/crypto: set required_alignment based on accel's buf align

Only some modules require buffers to be aligned to a block size, so we
will now report more accurate values for modules that don't have

bdev/crypto: set required_alignment based on accel's buf align

Only some modules require buffers to be aligned to a block size, so we
will now report more accurate values for modules that don't have such
requirements.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Iac76460435582eb39a6690fe83f08cd38718cfac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19765
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Jim Harris <jim.harris@gmail.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

bdev/crypto: check get_io_channel() result

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I3436b17721aebac9bedb9e980971db32dddf31b6
Reviewed-on: https://review.spdk.io/gerrit/c/

bdev/crypto: check get_io_channel() result

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I3436b17721aebac9bedb9e980971db32dddf31b6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18434
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

bdev/crypto: generate UUID from parent bdev UUID

Use base bdev UUID to generate new UUID for crypto bdev.

Change-Id: I164df2957b18e8141e6ff1cadf5d3a2f68dc3dd8
Signed-off-by: Krzysztof Karas <krzysz

bdev/crypto: generate UUID from parent bdev UUID

Use base bdev UUID to generate new UUID for crypto bdev.

Change-Id: I164df2957b18e8141e6ff1cadf5d3a2f68dc3dd8
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17127
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>

show more ...

bdev/crypto: Put accel buffer when write completes

Accel buffer is released when encrypt operation
completes, however it doesn't mean that base
bdev finishes writing encrypted data. As result,
accel

bdev/crypto: Put accel buffer when write completes

Accel buffer is released when encrypt operation
completes, however it doesn't mean that base
bdev finishes writing encrypted data. As result,
accel buffer might be reused in another IO, that
leads to data corruption.

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I1acf7c30da2f92989ecc44e96b00f7609058ec5a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17655
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot

show more ...

bdev/crypto: complete IOs on ENOMEM from accel

spdk_bdev_queue_io_wait() can only be used when one of bdev submission
functions returns ENOMEM (i.e. there are no more spdk_bdev_ios on that
IO channe

bdev/crypto: complete IOs on ENOMEM from accel

spdk_bdev_queue_io_wait() can only be used when one of bdev submission
functions returns ENOMEM (i.e. there are no more spdk_bdev_ios on that
IO channel). Using it in any other case, e.g. on spdk_accel_append_*()
returning ENOMEM, will most likely result in failure. Therefore, to
avoid that, the IOs are completed with NOMEM status relying on the bdev
layer to retry them.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie0f03496e5d3180c481815b3f1b021e74ae2f46d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17319
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>

show more ...

bdev/crypto: report accel sequence support

The code was already ready for that, so it's only a matter of reporting
that to the bdev layer.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
C

bdev/crypto: report accel sequence support

The code was already ready for that, so it's only a matter of reporting
that to the bdev layer.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I46ea2e6794e00590930651c5ff8c36588de641b4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17042
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>

show more ...

bdev/crypto: memory domain support

Now that all crypto operations are using interfaces that support memory
domains, bdev_crypto can report support for memory domains.

Signed-off-by: Konrad Sztyber

bdev/crypto: memory domain support

Now that all crypto operations are using interfaces that support memory
domains, bdev_crypto can report support for memory domains.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I13a128a599f6560197fed3405599c2a6bb609703
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17041
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>

show more ...

bdev/crypto: add crypto_io_fail()

There are lots of places when we need to abort accel sequence and
complete bdev_io with a failed status, so move that code to a dedicated
function.

Signed-off-by:

bdev/crypto: add crypto_io_fail()

There are lots of places when we need to abort accel sequence and
complete bdev_io with a failed status, so move that code to a dedicated
function.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5f6f4146c736d9c8e04b5667117d6e7ed824a654
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17088
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>

show more ...

bdev/crypto: merge IO-type specific callbacks

All these callbacks look identical now, so there's little point in
having three different functions doing the same thing.

Signed-off-by: Konrad Sztyber

bdev/crypto: merge IO-type specific callbacks

All these callbacks look identical now, so there's little point in
having three different functions doing the same thing.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I60d1426b5d2b20d924776699885e6a9dd176504a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17024
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot

show more ...

bdev/crypto: remove quiescing reset code

After changing to spdk_accel_append_*, this code is no longer necessary,
as accel operations are now executed by the base bdev/bdev layer.

Signed-off-by: Ko

bdev/crypto: remove quiescing reset code

After changing to spdk_accel_append_*, this code is no longer necessary,
as accel operations are now executed by the base bdev/bdev layer.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2e31249dfdf7abeee420744a5cc75b8273ecbad9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17023
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

bdev/crypto: use accel appends for encryption

Similarly to reads, writes path is now also using the spdk_accel_append*
interface for performing encrypt operation.

Additionally, this patch also chan

bdev/crypto: use accel appends for encryption

Similarly to reads, writes path is now also using the spdk_accel_append*
interface for performing encrypt operation.

Additionally, this patch also changes the way aux buffer is allocated -
spdk_bdev_io_get_aux_buf() was replaced with spdk_accel_get_buf(). This
ensures that the actual data buffer will be only allocated if it's
actually needed.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I3cd1d4f5753a95709d7b81de23d9227102a74261
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17022
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot

show more ...

bdev/crypto: use accel appends for decrypting data

The new spdk_accel_apend_decrypt() interface is now used for all read
requests for decrypting data. This makes it possible to chain decrypt
operat

bdev/crypto: use accel appends for decrypting data

The new spdk_accel_apend_decrypt() interface is now used for all read
requests for decrypting data. This makes it possible to chain decrypt
operations with other operations in a sequence. Support for encryption
in the write path will be added in the following patches.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2f65044eeaf92232d839ddd166ae50889b3f386d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17021
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot

show more ...

bdev/crypto: Use iobuf large_bufsize to set opt IO boundary

We used hardcoded value set by bdev module, however
this value might be changed, so get a real value
from iobug config

Signed-off-by: Ale

bdev/crypto: Use iobuf large_bufsize to set opt IO boundary

We used hardcoded value set by bdev module, however
this value might be changed, so get a real value
from iobug config

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Id15b602f699f3dd63aeff11fb063bb62c384663d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16601
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

bdev/crypto: Free aux buf when crypto op fails

Fixes issue #2886

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I19e2ba3763822bc9092fba584d35a6802c520ab7
Reviewed-on: https://revie

bdev/crypto: Free aux buf when crypto op fails

Fixes issue #2886

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I19e2ba3763822bc9092fba584d35a6802c520ab7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16600
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>

show more ...

bdev/crypto: Use accel framework

All DPDK related code is removed, handling of
RESET command was sligthly updated.
Handling of -ENOMEM was updated for cases when
accel API returns -ENOMEM

Crypto te

bdev/crypto: Use accel framework

All DPDK related code is removed, handling of
RESET command was sligthly updated.
Handling of -ENOMEM was updated for cases when
accel API returns -ENOMEM

Crypto tests in blockdev.sh were extended with more
crypto_bdevs to verify NOMEM cases - that failed
with original vbdev_crypto implementation

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: If1feba2449bee852c6c4daca4b3406414db6fded
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14860
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

bdev/crypto: make sure that vbdev_crypto_destruct() returns 1

Make vbdev_crypto_destruct() return 1 to signal that program
execution should wait for spdk_bdev_destruct_done() function,
which is adde

bdev/crypto: make sure that vbdev_crypto_destruct() returns 1

Make vbdev_crypto_destruct() return 1 to signal that program
execution should wait for spdk_bdev_destruct_done() function,
which is added inside _device_unregister_cb().

This change is related to _vdev_dev_get() not being able
to find the devices, when called from _cryptodev_sym_session_free(),
as it uses device driver name, which might already be freed.
This occurs only during bdev module finish, when crypto bdevs
are being unregistered and vbdev_crypto_finish() proceeds to
call bdev name deletion without waiting for the unregister
callbacks to complete, which ultimately results in reading
freed pointers.

This only happens when code execution takes path for DPDK 22.11+.

Change-Id: Id9a43d07c90aef7a82867383fd77354ac521a3e7
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16290
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

bdev/crypto: Add new callback to spdk_bdev_unregister_bdev_by_name()

Currently delete_crypto_disk() calls spdk_bdev_unregister_bdev_by_name()
and immediately after exiting that function, it proceeds

bdev/crypto: Add new callback to spdk_bdev_unregister_bdev_by_name()

Currently delete_crypto_disk() calls spdk_bdev_unregister_bdev_by_name()
and immediately after exiting that function, it proceeds to bdev name
deletion. This causes callbacks triggered by above call to run with
references to bdev names that are partially unavailable due to free()
functions in vbdev_crypto_delete_name().

Add an intermediate callback that will call vbdev_crypto_delete_name()
and then the original callback that was sent as an argument to
delete_crypto_disk(). That way we are sure that bdev name deletion
is performed right after bdev unregister.

Change-Id: I483ce5c3c63596f54d44220ca61c5e97c3807faf
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16253
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

bdev/crypto: do not create mempool for session private data

In DPDK 22.11 rte_cryptodev_sym_session_create() now takes
a single mempool with element size big enough to hold session
data and session

bdev/crypto: do not create mempool for session private data

In DPDK 22.11 rte_cryptodev_sym_session_create() now takes
a single mempool with element size big enough to hold session
data and session private data.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6c9db063825843a903d1ff84dd8d77f198a841a1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15435
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot

show more ...