cryptodev: move RSA padding information into xform

RSA padding information could be a xform entity rather than part of
crypto op, as it seems associated with hashing algorithm used for
the entire cr

cryptodev: move RSA padding information into xform

RSA padding information could be a xform entity rather than part of
crypto op, as it seems associated with hashing algorithm used for
the entire crypto session, where this algorithm is used in message
digest itself. Even in virtIO standard spec, this info is associated
in the asymmetric session creation. Hence, moving this info from
crypto op into xform structure.

Signed-off-by: Gowrishankar Muthukrishnan <gmuthukrishn@marvell.com>
Acked-by: Arkadiusz Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

show more ...

crypto/openssl: support EdDSA

Support EdDSA crypto algorithm in OpenSSL PMD.

Signed-off-by: Gowrishankar Muthukrishnan <gmuthukrishn@marvell.com>
Acked-by: Kai Ji <kai.ji@intel.com>

drivers: use per line logging in helpers

Use RTE_LOG(_DP)?_LINE(_PREFIX)? in existing macros that append a \n.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Chengwen Feng <fen

drivers: use per line logging in helpers

Use RTE_LOG(_DP)?_LINE(_PREFIX)? in existing macros that append a \n.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Chengwen Feng <fengchengwen@huawei.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Thomas Monjalon <thomas@monjalon.net>

show more ...

crypto/openssl: make per-QP auth context clones

Currently EVP auth ctxs (e.g. EVP_MD_CTX, EVP_MAC_CTX) are allocated,
copied to (from openssl_session), and then freed for every auth
operation (ie. p

crypto/openssl: make per-QP auth context clones

Currently EVP auth ctxs (e.g. EVP_MD_CTX, EVP_MAC_CTX) are allocated,
copied to (from openssl_session), and then freed for every auth
operation (ie. per packet). This is very inefficient, and avoidable.

Make each openssl_session hold an array of structures, containing
pointers to per-queue-pair cipher and auth context copies. These are
populated on first use by allocating a new context and copying from the
main context. These copies can then be used in a thread-safe manner by
different worker lcores simultaneously. Consequently the auth context
allocation and copy only has to happen once - the first time a given qp
uses an openssl_session. This brings about a large performance boost.

Throughput performance uplift measurements for HMAC-SHA1 generate on
Ampere Altra Max platform:
1 worker lcore
| buffer sz (B) | prev (Gbps) | optimised (Gbps) | uplift |
|-----------------+---------------+--------------------+----------|
| 64 | 0.63 | 1.42 | 123.5% |
| 256 | 2.24 | 4.40 | 96.4% |
| 1024 | 6.15 | 9.26 | 50.6% |
| 2048 | 8.68 | 11.38 | 31.1% |
| 4096 | 10.92 | 12.84 | 17.6% |

8 worker lcores
| buffer sz (B) | prev (Gbps) | optimised (Gbps) | uplift |
|-----------------+---------------+--------------------+----------|
| 64 | 0.93 | 11.35 | 1122.5% |
| 256 | 3.70 | 35.30 | 853.7% |
| 1024 | 15.22 | 74.27 | 387.8% |
| 2048 | 30.20 | 91.08 | 201.6% |
| 4096 | 56.92 | 102.76 | 80.5% |

Cc: stable@dpdk.org

Signed-off-by: Jack Bond-Preston <jack.bond-preston@foss.arm.com>
Acked-by: Kai Ji <kai.ji@intel.com>
Reviewed-by: Wathsala Vithanage <wathsala.vithanage@arm.com>

show more ...

crypto/openssl: make per-QP cipher context clones

Currently EVP_CIPHER_CTXs are allocated, copied to (from
openssl_session), and then freed for every cipher operation (ie. per
packet). This is very

crypto/openssl: make per-QP cipher context clones

Currently EVP_CIPHER_CTXs are allocated, copied to (from
openssl_session), and then freed for every cipher operation (ie. per
packet). This is very inefficient, and avoidable.

Make each openssl_session hold an array of pointers to per-queue-pair
cipher context copies. These are populated on first use by allocating a
new context and copying from the main context. These copies can then be
used in a thread-safe manner by different worker lcores simultaneously.
Consequently the cipher context allocation and copy only has to happen
once - the first time a given qp uses an openssl_session. This brings
about a large performance boost.

Throughput performance uplift measurements for AES-CBC-128 encrypt on
Ampere Altra Max platform:
1 worker lcore
| buffer sz (B) | prev (Gbps) | optimised (Gbps) | uplift |
|-----------------+---------------+--------------------+----------|
| 64 | 1.51 | 2.94 | 94.4% |
| 256 | 4.90 | 8.05 | 64.3% |
| 1024 | 11.07 | 14.21 | 28.3% |
| 2048 | 14.03 | 16.28 | 16.0% |
| 4096 | 16.20 | 17.59 | 8.6% |

8 worker lcores
| buffer sz (B) | prev (Gbps) | optimised (Gbps) | uplift |
|-----------------+---------------+--------------------+----------|
| 64 | 3.05 | 23.74 | 678.8% |
| 256 | 10.46 | 64.86 | 520.3% |
| 1024 | 40.97 | 113.80 | 177.7% |
| 2048 | 73.25 | 130.21 | 77.8% |
| 4096 | 103.89 | 140.62 | 35.4% |

Cc: stable@dpdk.org

Signed-off-by: Jack Bond-Preston <jack.bond-preston@foss.arm.com>
Acked-by: Kai Ji <kai.ji@intel.com>
Reviewed-by: Wathsala Vithanage <wathsala.vithanage@arm.com>

show more ...

drivers: move alignment attribute on types for MSVC

Move location of __rte_aligned(a) to new conventional location. The new
placement between {struct,union} and the tag allows the desired
alignment

drivers: move alignment attribute on types for MSVC

Move location of __rte_aligned(a) to new conventional location. The new
placement between {struct,union} and the tag allows the desired
alignment to be imparted on the type regardless of the toolchain being
used for both C and C++. Additionally, it avoids confusion by Doxygen
when generating documentation.

Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>

show more ...

crypto/openssl: fix memory leaks in asym session

Numerous memory leaks were detected by ASAN
in the OpenSSL PMD asymmetric code path.

These are now fixed to free all variables allocated
by OpenSSL

crypto/openssl: fix memory leaks in asym session

Numerous memory leaks were detected by ASAN
in the OpenSSL PMD asymmetric code path.

These are now fixed to free all variables allocated
by OpenSSL functions such as BN_bin2bn and
OSSL_PARAM_BLD_new.

Some need to exist until the op is processed,
for example the BIGNUMs associated with DSA.
The pointers for these are added to the private
asym session so they can be accessed later when calling free.

Some cases need to be treated differently if OpenSSL < 3.0.
It has slightly different handling of memory, as functions such as
RSA_set0_key() take over memory management of values,
so the caller should not free the values.

Fixes: 4c7ae22f1f83 ("crypto/openssl: update DSA routine with 3.0 EVP API")
Fixes: c794b40c9258 ("crypto/openssl: update DH routine with 3.0 EVP API")
Fixes: 3b7d638fb11f ("crypto/openssl: support asymmetric SM2")
Fixes: ac42813a0a7c ("crypto/openssl: add DH and DSA asym operations")
Fixes: d7bd42f6db19 ("crypto/openssl: update RSA routine with 3.0 EVP API")
Fixes: 9d91c3047dfd ("crypto/openssl: fix memory leaks in asym operations")
Cc: stable@dpdk.org

Signed-off-by: Ciara Power <ciara.power@intel.com>
Acked-by: Kai Ji <kai.ji@intel.com>

show more ...

crypto/openssl: support asymmetric SM2

Add SM2 asymmetric algorithm support in openssl PMD.

Signed-off-by: Gowrishankar Muthukrishnan <gmuthukrishn@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvel

crypto/openssl: support asymmetric SM2

Add SM2 asymmetric algorithm support in openssl PMD.

Signed-off-by: Gowrishankar Muthukrishnan <gmuthukrishn@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

show more ...

cryptodev: rework session framework

As per current design, rte_cryptodev_sym_session_create() and
rte_cryptodev_sym_session_init() use separate mempool objects
for a single session.
And structure rt

cryptodev: rework session framework

As per current design, rte_cryptodev_sym_session_create() and
rte_cryptodev_sym_session_init() use separate mempool objects
for a single session.
And structure rte_cryptodev_sym_session is not directly used
by the application, it may cause ABI breakage if the structure
is modified in future.

To address these two issues, the rte_cryptodev_sym_session_create
will take one mempool object that the session and session private
data are virtually/physically contiguous, and initializes both
fields. The API rte_cryptodev_sym_session_init is removed.

rte_cryptodev_sym_session_create will now return an opaque session
pointer which will be used by the app and other APIs.

In data path, opaque session pointer is attached to rte_crypto_op
and the PMD can call an internal library API to get the session
private data pointer based on the driver id.

Note: currently single session may be used by different device
drivers, given it is initialized by them. After the change the
session created by one device driver cannot be used or
reinitialized by another driver.

Signed-off-by: Akhil Goyal <gakhil@marvell.com>
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Kai Ji <kai.ji@intel.com>
Tested-by: Gagandeep Singh <g.singh@nxp.com>
Tested-by: David Coyle <david.coyle@intel.com>
Tested-by: Kevin O'Sullivan <kevin.osullivan@intel.com>

show more ...

crypto/openssl: support AES-CMAC operations

Extend openssl crypto PMD to support AES CMAC operations.

Signed-off-by: Ashwin Sekhar T K <asekhar@marvell.com>
Acked-by: Kai Ji <kai.ji@intel.com>

crypto/openssl: update DSA routine with 3.0 EVP API

This patch updates asymmetric DSA routine in crypto openssl pmd
to adopt openssl 3.0 EVP apis. Divided the single combined DSA sign
test to two in

crypto/openssl: update DSA routine with 3.0 EVP API

This patch updates asymmetric DSA routine in crypto openssl pmd
to adopt openssl 3.0 EVP apis. Divided the single combined DSA sign
test to two individual DSA sign and DSA verfiy tests.

Signed-off-by: Kai Ji <kai.ji@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

show more ...

crypto/openssl: update DH routine with 3.0 EVP API

This patch updates asymmetric DH routine in crypto openssl pmd
to adopt openssl 3.0 EVP apis.

Signed-off-by: Kai Ji <kai.ji@intel.com>
Acked-by: F

crypto/openssl: update DH routine with 3.0 EVP API

This patch updates asymmetric DH routine in crypto openssl pmd
to adopt openssl 3.0 EVP apis.

Signed-off-by: Kai Ji <kai.ji@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

show more ...

crypto/openssl: update RSA routine with 3.0 EVP API

This patch updates asymmetric RSA routine in crypto openssl pmd
to adopt openssl 3.0 EVP apis.

Signed-off-by: Kai Ji <kai.ji@intel.com>
Acked-by:

crypto/openssl: update RSA routine with 3.0 EVP API

This patch updates asymmetric RSA routine in crypto openssl pmd
to adopt openssl 3.0 EVP apis.

Signed-off-by: Kai Ji <kai.ji@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

show more ...

crypto/openssl: update HMAC routine with 3.0 EVP API

This patch update the symmetric HMAC routine in crypto openssl
pmd to adopt openssl 3.0 EVP apis.

Signed-off-by: Kai Ji <kai.ji@intel.com>
Acked

crypto/openssl: update HMAC routine with 3.0 EVP API

This patch update the symmetric HMAC routine in crypto openssl
pmd to adopt openssl 3.0 EVP apis.

Signed-off-by: Kai Ji <kai.ji@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

show more ...

drivers: fix log type variables for -fno-common

The zlib compression driver, as well as the aesni-gcm, aesni-mb and openssl
crypto drivers all defined the logtype variable in the header file
directl

drivers: fix log type variables for -fno-common

The zlib compression driver, as well as the aesni-gcm, aesni-mb and openssl
crypto drivers all defined the logtype variable in the header file
directly. This gives errors with gcc 10, due to -fno-common being the
default, so we need to apply the same fix in all cases:
* move the variable definition to a suitable .c file
* mark the forward declaration of the variable in the header as "extern"

Fixes: 0c4e4c16b004 ("compress/zlib: introduce zlib PMD")
Fixes: 90c8a2d02a31 ("crypto/aesni_gcm: add dynamic logging")
Fixes: 276624ae2ef6 ("crypto/aesni_mb: add dynamic logging")
Fixes: 094b2386f461 ("crypto/openssl: add dynamic logging")
Cc: stable@dpdk.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Lukasz Wojciechowski <l.wojciechow@partner.samsung.com>

show more ...

rename private header files

Some of the internal header files have 'rte_' prefix
and some don't.
Remove 'rte_' prefix from all internal header files.

Suggested-by: Thomas Monjalon <thomas@monjalon.

rename private header files

Some of the internal header files have 'rte_' prefix
and some don't.
Remove 'rte_' prefix from all internal header files.

Suggested-by: Thomas Monjalon <thomas@monjalon.net>
Signed-off-by: Dharmik Thakkar <dharmik.thakkar@arm.com>
Reviewed-by: Phil Yang <phil.yang@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>

show more ...