env: add SPDK_ENV_NUMA_ID_ANY and replace socket_id with numa_id

We will try to avoid further proliferation of "SOCKET_ID" to refer
to a NUMA socket ID moving forward, and just use "NUMA_ID" to avoi

env: add SPDK_ENV_NUMA_ID_ANY and replace socket_id with numa_id

We will try to avoid further proliferation of "SOCKET_ID" to refer
to a NUMA socket ID moving forward, and just use "NUMA_ID" to avoid
confusion with TCP sockets.

Change all of the existing in-tree SPDK_ENV_SOCKET_ID_ANY uses to
SPDK_ENV_NUMA_ID_ANY, but keep the old #define around, at least for
now. Also change all 'socket_id' parameters to 'numa_id'.

We still have spdk_env_get_socket_id(), we will need to keep this
but next patch will add spdk_env_get_numa_id().

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: Idc31c29e32b708c24d88f9c6fecaf9a99e34ba1e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24607
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <ben@nvidia.com>
Community-CI: Mellanox Build Bot

show more ...

lib/accel: fallback when no devices during module_init()

Related to #3266

During discussion in issue above, it came up that if
dpdk_compressdev and dpdk_cryptodev detects no devices
they are still

lib/accel: fallback when no devices during module_init()

Related to #3266

During discussion in issue above, it came up that if
dpdk_compressdev and dpdk_cryptodev detects no devices
they are still assigned to the relevant accel ops.

Rather than explicitly failing just due to lack of
the devices, each accel module can report -ENODEV.
In such case the module will not be used, but that
error is not propagated to accel framework init.

As a result accel ops assignment will not use such module,
and fallback to available modules.

Reported-by: liuqinfei <lucas.liuqinfei@huawei.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib52dfcb6dc6a1a93b8493b98ffb028f9d9419d6d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22026
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

accel: add the spdk_ prefix to accel_opcode

This enum is part of the public interface, so it should have the spdk_
prefix.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie1d4a

accel: add the spdk_ prefix to accel_opcode

This enum is part of the public interface, so it should have the spdk_
prefix.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie1d4aee707ede70c7f181561e1002ce443628ca4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19685
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@gmail.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>

show more ...

test/unit: use spdk_ut_run_tests()

Replaced direct calls to the CUnit's functions to run the tests with
spdk_ut_run_tests(). That way, each test will have the ability to run
a specific test case.

test/unit: use spdk_ut_run_tests()

Replaced direct calls to the CUnit's functions to run the tests with
spdk_ut_run_tests(). That way, each test will have the ability to run
a specific test case.

The blob.c unit test wasn't changed, because it runs all tests multiple
times with different parameter combinations, so it cannot be easily
converted. In the future, each such combination could be split into a
separate test suite, which would make it compatible with
spdk_ut_run_tests().

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I4463f808f89844e9bf32b5b31eda197c5d729d1d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19288
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

test/unit: move spdk_cunit.h to include/spdk_internal

It'll make it easier to include this file outside of unit tests.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I171ddb864

test/unit: move spdk_cunit.h to include/spdk_internal

It'll make it easier to include this file outside of unit tests.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I171ddb8649f67b5786f08647560e2907603d0574
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19284
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>

show more ...

accel: introduce cipher enum

Enum is introduced so accel crypto modules doesn't need to define
such on its own, moreover logic is simplified as module doesn't
need to operate on c-string but simple

accel: introduce cipher enum

Enum is introduced so accel crypto modules doesn't need to define
such on its own, moreover logic is simplified as module doesn't
need to operate on c-string but simple enum type.

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Iaa8481ee4818f999300043f86b6576876f9ed6a6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18195
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

accel/dpdk_cryptodev: Handle OP_STATUS_SUCCESS

SW PMD might process a crypto operation but failed
to submit it to a completions ring.
Such operation can't be retried if crypto operation
is inplace.

accel/dpdk_cryptodev: Handle OP_STATUS_SUCCESS

SW PMD might process a crypto operation but failed
to submit it to a completions ring.
Such operation can't be retried if crypto operation
is inplace.
Handle such crypto op as a completed.
Verified by integrating rte openssl driver and
adding additional logs to check that SUCCESS
status received and completed as expected.

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Ida161cec045167af752ebd5b57f41b2bbfe8b97c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16995
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

accel/dpdk_cryptodev: Remove queued_cry_ops

If we were not able to submit all configured
crypto ops, then we can just release crypto_ops
and mbuf object of these crypto ops and save
the actual numbe

accel/dpdk_cryptodev: Remove queued_cry_ops

If we were not able to submit all configured
crypto ops, then we can just release crypto_ops
and mbuf object of these crypto ops and save
the actual number of submitted operation in
the accel task. Once all submitted operations
complete, poller will call
accel_dpdk_cryptodev_process_task func to submit
cyrpto operations for reamining data blocks.
If no crypto ops were submitted then the task
will be palced in the channel's queued_tasks
and poller will try to resubmit the task.
That in theory should increase performance
since we attempted to resubmit queued ops
with burst size==1 which is not efficient

Fixes issue #2907

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I4d17e8ed1ad5383848e4d09c46009c6cb2834360
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16784
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

accel/dpdk_cryptodev: Fix sgl init with offset

When accel task is processed is processed in
several iterations (submit part of cryops, wait
for completion and submit next part of cryops),
sgl is ini

accel/dpdk_cryptodev: Fix sgl init with offset

When accel task is processed is processed in
several iterations (submit part of cryops, wait
for completion and submit next part of cryops),
sgl is initialized with offset to exclude previously
processed blocks. However there is a bug since
spdk_iov_sgl_init doesn't advance iovs, as result
when we do sgl->iov->iov_len - sgl->iov_offset,
we may get unsigned int underflow.
Fix is init sgl with 0 offset and then advance it
with offset.
Modified unit test and added an assert in code to
verify this fix.

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Ib53ff30f0c90d521f2cf6b3ec847b0d06869c2b5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17456
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

accel/dpdk_cryptodev: Update rte_cryptodev usage for DPDK 22.11

This patch is a combination of commits which update vdev_crypto:

110d8411e bdev/crypto: do not create mempool for session private dat

accel/dpdk_cryptodev: Update rte_cryptodev usage for DPDK 22.11

This patch is a combination of commits which update vdev_crypto:

110d8411e bdev/crypto: do not create mempool for session private data
495055b05 bdev/crypto: update rte_cryptodev usage for DPDK 22.11
02caed6b5 bdev/crypto: remove mempool usage matching < DPDK 19.02
5887eb321 bdev/crypto: do not track type of crypto session

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I30c4f76e4e7b4865a7daa638d357888bb5e02071
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16039
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>

show more ...

dpdk_cryptodev: Check queue capacity before submitting a task

When we submit more tasks than supported by qp,
extra tasks are queued on io_channel. Later completion
poller tries to resubmit these ta

dpdk_cryptodev: Check queue capacity before submitting a task

When we submit more tasks than supported by qp,
extra tasks are queued on io_channel. Later completion
poller tries to resubmit these tasks one by one. That
is not efficient since every enqueu_burst may cause
doorbell updates in HW.

Instead add a check for qpir capacity and submit
appropriate number of requests. If qpair is full,
tasks are queued in dedicated list. This approach
should remove or minimize the need to resubmit
individual crypto operations.

This also handles a case where there are no entries
in global pools (crypto_ops or rte_mbuf)

Fixes issue #2756

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Iab50e623e7a82a4f5bef7a1e4434e593240ab633
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15769
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>

show more ...

dpdk_cryptodev: Remove limit on max IO size

Previously vbdev_crypto used DPDK directly and
the restriction on max IO size was propagated to
generic bdev layer which split big IO requests.

Now, when

dpdk_cryptodev: Remove limit on max IO size

Previously vbdev_crypto used DPDK directly and
the restriction on max IO size was propagated to
generic bdev layer which split big IO requests.

Now, when DPDK code is a standalone accel module,
this restriction on max IO size is not visible to
the user and we should get rid of it.

To remove this limitation, allow to submit crypto
operations for part of logical blocks in big IO,
the rest blocks will be processed when all submitted
crypto ops are completed.

To verify this patch, add a functional test which
submits big IO verify mode

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I0ee89e98195a5c744f3fb2bfc752b578965c3bc5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15768
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

accel: remove nbytes from spdk_accel_task

All operations are using iovecs to describe their buffers and only
encrypt/decrypt additionally used nbytes to store the total size of a
src buffer. We don

accel: remove nbytes from spdk_accel_task

All operations are using iovecs to describe their buffers and only
encrypt/decrypt additionally used nbytes to store the total size of a
src buffer. We don't really need this value in the generic accel code,
so we can let modules calculate it, if necessary. That way, we won't
waste cycles calculating it if a module doesn't use it and it makes the
code a bit easier, as we won't have to deal with the fact that nbytes is
only valid for certain operations.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I29252be34a9af9fd40f4c7fec9d0a0c1139c562d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16306
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

module/accel: Add accel_dpdk_cryptodev

The new module replaces functionality in vbdev_crypto.
This module is bdev agnostic, so some inernal parts
were reworked.

io_channel: contains a qp of every c

module/accel: Add accel_dpdk_cryptodev

The new module replaces functionality in vbdev_crypto.
This module is bdev agnostic, so some inernal parts
were reworked.

io_channel: contains a qp of every configured DPDK PMD
crypto key: for mlx5_pci we register a key on each available
device since keys are bound to Protection Domain.

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: If1845cb87eadacbb921c593ba82207a97f2209a3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14859
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>

show more ...

accel/dpdk_cryptodev: start with copy of vbdev_crypto

This patch is just a copy of vbdev_crypto.c and the
corresponding UT file. It makes it easier to review
the next patch which adds accel operatio

accel/dpdk_cryptodev: start with copy of vbdev_crypto

This patch is just a copy of vbdev_crypto.c and the
corresponding UT file. It makes it easier to review
the next patch which adds accel operations

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Ib88b45d573b011b1acb35da9bf4dab922d8fb183
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16182
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...