crypto/openssl: remove unnecessary null free checks

The function EVP_PKEY_CTX_free allows NULL parameter
(similar to free()).

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>

crypto/openssl: fix potential string overflow

The algorithm name is a string and should be copied with strlcpy()
rather than rte_memcpy(). This fixes a warning detected with
clang and ASAN.

Bugzill

crypto/openssl: fix potential string overflow

The algorithm name is a string and should be copied with strlcpy()
rather than rte_memcpy(). This fixes a warning detected with
clang and ASAN.

Bugzilla ID: 1565
Fixes: 2b9c693f6ef5 ("crypto/openssl: support AES-CMAC operations")
Cc: stable@dpdk.org

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>

show more ...

crypto/openssl: fix 3DES-CTR with big endian CPUs

Caught by code review.

Don't byte swap unconditionally (assuming that CPU is little endian is
wrong). Instead, convert from big endian to cpu and v

crypto/openssl: fix 3DES-CTR with big endian CPUs

Caught by code review.

Don't byte swap unconditionally (assuming that CPU is little endian is
wrong). Instead, convert from big endian to cpu and vice versa.

Besides, avoid unaligned accesses and remove the ctr_inc helper that is
not used anywhere else.

Fixes: d61f70b4c918 ("crypto/libcrypto: add driver for OpenSSL library")
Cc: stable@dpdk.org

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>

show more ...

cryptodev: move RSA padding information into xform

RSA padding information could be a xform entity rather than part of
crypto op, as it seems associated with hashing algorithm used for
the entire cr

cryptodev: move RSA padding information into xform

RSA padding information could be a xform entity rather than part of
crypto op, as it seems associated with hashing algorithm used for
the entire crypto session, where this algorithm is used in message
digest itself. Even in virtIO standard spec, this info is associated
in the asymmetric session creation. Hence, moving this info from
crypto op into xform structure.

Signed-off-by: Gowrishankar Muthukrishnan <gmuthukrishn@marvell.com>
Acked-by: Arkadiusz Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

show more ...

crypto/openssl: support EdDSA

Support EdDSA crypto algorithm in OpenSSL PMD.

Signed-off-by: Gowrishankar Muthukrishnan <gmuthukrishn@marvell.com>
Acked-by: Kai Ji <kai.ji@intel.com>

drivers: remove redundant newline from logs

Fix places where two newline characters may be logged.

Cc: stable@dpdk.org

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Chengwen

drivers: remove redundant newline from logs

Fix places where two newline characters may be logged.

Cc: stable@dpdk.org

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Chengwen Feng <fengchengwen@huawei.com>

show more ...

crypto/openssl: set cipher padding once

Setting the cipher padding has a noticeable performance footprint,
and it doesn't need to be done for every call to
process_openssl_cipher_{en,de}crypt(). Set

crypto/openssl: set cipher padding once

Setting the cipher padding has a noticeable performance footprint,
and it doesn't need to be done for every call to
process_openssl_cipher_{en,de}crypt(). Setting it causes OpenSSL to set
it on every future context re-init. Thus, for every buffer after the
first one, the padding is being set twice.

Instead, just set the cipher padding once - when configuring the session
parameters - avoiding the unnecessary double setting behaviour. This is
skipped for AEAD ciphers, where disabling padding is not necessary.

Throughput performance uplift measurements for AES-CBC-128 encrypt on
Ampere Altra Max platform:
1 worker lcore
| buffer sz (B) | prev (Gbps) | optimised (Gbps) | uplift |
|-----------------+---------------+--------------------+----------|
| 64 | 2.97 | 3.72 | 25.2% |
| 256 | 8.10 | 9.42 | 16.3% |
| 1024 | 14.22 | 15.18 | 6.8% |
| 2048 | 16.28 | 16.93 | 4.0% |
| 4096 | 17.58 | 17.97 | 2.2% |

8 worker lcores
| buffer sz (B) | prev (Gbps) | optimised (Gbps) | uplift |
|-----------------+---------------+--------------------+----------|
| 64 | 21.27 | 29.85 | 40.3% |
| 256 | 60.05 | 75.53 | 25.8% |
| 1024 | 110.11 | 121.56 | 10.4% |
| 2048 | 128.05 | 135.40 | 5.7% |
| 4096 | 139.45 | 143.76 | 3.1% |

Cc: stable@dpdk.org

Signed-off-by: Jack Bond-Preston <jack.bond-preston@foss.arm.com>
Acked-by: Kai Ji <kai.ji@intel.com>
Reviewed-by: Wathsala Vithanage <wathsala.vithanage@arm.com>

show more ...

crypto/openssl: make per-QP auth context clones

Currently EVP auth ctxs (e.g. EVP_MD_CTX, EVP_MAC_CTX) are allocated,
copied to (from openssl_session), and then freed for every auth
operation (ie. p

crypto/openssl: make per-QP auth context clones

Currently EVP auth ctxs (e.g. EVP_MD_CTX, EVP_MAC_CTX) are allocated,
copied to (from openssl_session), and then freed for every auth
operation (ie. per packet). This is very inefficient, and avoidable.

Make each openssl_session hold an array of structures, containing
pointers to per-queue-pair cipher and auth context copies. These are
populated on first use by allocating a new context and copying from the
main context. These copies can then be used in a thread-safe manner by
different worker lcores simultaneously. Consequently the auth context
allocation and copy only has to happen once - the first time a given qp
uses an openssl_session. This brings about a large performance boost.

Throughput performance uplift measurements for HMAC-SHA1 generate on
Ampere Altra Max platform:
1 worker lcore
| buffer sz (B) | prev (Gbps) | optimised (Gbps) | uplift |
|-----------------+---------------+--------------------+----------|
| 64 | 0.63 | 1.42 | 123.5% |
| 256 | 2.24 | 4.40 | 96.4% |
| 1024 | 6.15 | 9.26 | 50.6% |
| 2048 | 8.68 | 11.38 | 31.1% |
| 4096 | 10.92 | 12.84 | 17.6% |

8 worker lcores
| buffer sz (B) | prev (Gbps) | optimised (Gbps) | uplift |
|-----------------+---------------+--------------------+----------|
| 64 | 0.93 | 11.35 | 1122.5% |
| 256 | 3.70 | 35.30 | 853.7% |
| 1024 | 15.22 | 74.27 | 387.8% |
| 2048 | 30.20 | 91.08 | 201.6% |
| 4096 | 56.92 | 102.76 | 80.5% |

Cc: stable@dpdk.org

Signed-off-by: Jack Bond-Preston <jack.bond-preston@foss.arm.com>
Acked-by: Kai Ji <kai.ji@intel.com>
Reviewed-by: Wathsala Vithanage <wathsala.vithanage@arm.com>

show more ...

crypto/openssl: make per-QP cipher context clones

Currently EVP_CIPHER_CTXs are allocated, copied to (from
openssl_session), and then freed for every cipher operation (ie. per
packet). This is very

crypto/openssl: make per-QP cipher context clones

Currently EVP_CIPHER_CTXs are allocated, copied to (from
openssl_session), and then freed for every cipher operation (ie. per
packet). This is very inefficient, and avoidable.

Make each openssl_session hold an array of pointers to per-queue-pair
cipher context copies. These are populated on first use by allocating a
new context and copying from the main context. These copies can then be
used in a thread-safe manner by different worker lcores simultaneously.
Consequently the cipher context allocation and copy only has to happen
once - the first time a given qp uses an openssl_session. This brings
about a large performance boost.

Throughput performance uplift measurements for AES-CBC-128 encrypt on
Ampere Altra Max platform:
1 worker lcore
| buffer sz (B) | prev (Gbps) | optimised (Gbps) | uplift |
|-----------------+---------------+--------------------+----------|
| 64 | 1.51 | 2.94 | 94.4% |
| 256 | 4.90 | 8.05 | 64.3% |
| 1024 | 11.07 | 14.21 | 28.3% |
| 2048 | 14.03 | 16.28 | 16.0% |
| 4096 | 16.20 | 17.59 | 8.6% |

8 worker lcores
| buffer sz (B) | prev (Gbps) | optimised (Gbps) | uplift |
|-----------------+---------------+--------------------+----------|
| 64 | 3.05 | 23.74 | 678.8% |
| 256 | 10.46 | 64.86 | 520.3% |
| 1024 | 40.97 | 113.80 | 177.7% |
| 2048 | 73.25 | 130.21 | 77.8% |
| 4096 | 103.89 | 140.62 | 35.4% |

Cc: stable@dpdk.org

Signed-off-by: Jack Bond-Preston <jack.bond-preston@foss.arm.com>
Acked-by: Kai Ji <kai.ji@intel.com>
Reviewed-by: Wathsala Vithanage <wathsala.vithanage@arm.com>

show more ...

crypto/openssl: optimize 3DES-CTR context init

Currently the 3DES-CTR cipher context is initialised for every buffer,
setting the cipher implementation and key - even though for every
buffer in the

crypto/openssl: optimize 3DES-CTR context init

Currently the 3DES-CTR cipher context is initialised for every buffer,
setting the cipher implementation and key - even though for every
buffer in the session these values will be the same.

Change to initialising the cipher context once, before any buffers are
processed, instead.

Throughput performance uplift measurements for 3DES-CTR encrypt on
Ampere Altra Max platform:
1 worker lcore
| buffer sz (B) | prev (Gbps) | optimised (Gbps) | uplift |
|-----------------+---------------+--------------------+----------|
| 64 | 0.16 | 0.21 | 35.3% |
| 256 | 0.20 | 0.22 | 9.4% |
| 1024 | 0.22 | 0.23 | 2.3% |
| 2048 | 0.22 | 0.23 | 0.9% |
| 4096 | 0.22 | 0.23 | 0.9% |

8 worker lcores
| buffer sz (B) | prev (Gbps) | optimised (Gbps) | uplift |
|-----------------+---------------+--------------------+----------|
| 64 | 1.01 | 1.34 | 32.9% |
| 256 | 1.51 | 1.66 | 9.9% |
| 1024 | 1.72 | 1.77 | 2.6% |
| 2048 | 1.76 | 1.78 | 1.1% |
| 4096 | 1.79 | 1.80 | 0.6% |

Cc: stable@dpdk.org

Signed-off-by: Jack Bond-Preston <jack.bond-preston@foss.arm.com>
Acked-by: Kai Ji <kai.ji@intel.com>
Reviewed-by: Wathsala Vithanage <wathsala.vithanage@arm.com>

show more ...

crypto/openssl: fix GCM and CCM thread unsafe contexts

Commit 67ab783b5d70 ("crypto/openssl: use local copy for session
contexts") introduced a fix for concurrency bugs which could occur when
using

crypto/openssl: fix GCM and CCM thread unsafe contexts

Commit 67ab783b5d70 ("crypto/openssl: use local copy for session
contexts") introduced a fix for concurrency bugs which could occur when
using one OpenSSL PMD session across multiple cores simultaneously. The
solution was to clone the EVP contexts per-buffer to avoid them being
used concurrently.

However, part of commit 75adf1eae44f ("crypto/openssl: update HMAC
routine with 3.0 EVP API") reverted this fix, only for combined ops
(AES-GCM and AES-CCM).

Fix the concurrency issue by cloning EVP contexts per-buffer. An extra
workaround is required for OpenSSL versions which are >= 3.0.0, and
<= 3.2.0. This is because, prior to OpenSSL 3.2.0, EVP_CIPHER_CTX_copy()
is not implemented for AES-GCM or AES-CCM. When using these OpenSSL
versions, create and initialise the context from scratch, per-buffer.

Throughput performance uplift measurements for AES-GCM-128 encrypt on
Ampere Altra Max platform:
1 worker lcore
| buffer sz (B) | prev (Gbps) | optimised (Gbps) | uplift |
|-----------------+---------------+--------------------+----------|
| 64 | 2.60 | 1.31 | -49.5% |
| 256 | 7.69 | 4.45 | -42.1% |
| 1024 | 15.33 | 11.30 | -26.3% |
| 2048 | 18.74 | 15.37 | -18.0% |
| 4096 | 21.11 | 18.80 | -10.9% |

8 worker lcores
| buffer sz (B) | prev (Gbps) | optimised (Gbps) | uplift |
|-----------------+---------------+--------------------+----------|
| 64 | 19.94 | 2.83 | -85.8% |
| 256 | 58.84 | 11.00 | -81.3% |
| 1024 | 119.71 | 42.46 | -64.5% |
| 2048 | 147.69 | 80.91 | -45.2% |
| 4096 | 167.39 | 121.25 | -27.6% |

Fixes: 75adf1eae44f ("crypto/openssl: update HMAC routine with 3.0 EVP API")
Cc: stable@dpdk.org

Signed-off-by: Jack Bond-Preston <jack.bond-preston@foss.arm.com>
Acked-by: Kai Ji <kai.ji@intel.com>
Reviewed-by: Wathsala Vithanage <wathsala.vithanage@arm.com>

show more ...

crypto/openssl: fix memory leaks in asym session

Numerous memory leaks were detected by ASAN
in the OpenSSL PMD asymmetric code path.

These are now fixed to free all variables allocated
by OpenSSL

crypto/openssl: fix memory leaks in asym session

Numerous memory leaks were detected by ASAN
in the OpenSSL PMD asymmetric code path.

These are now fixed to free all variables allocated
by OpenSSL functions such as BN_bin2bn and
OSSL_PARAM_BLD_new.

Some need to exist until the op is processed,
for example the BIGNUMs associated with DSA.
The pointers for these are added to the private
asym session so they can be accessed later when calling free.

Some cases need to be treated differently if OpenSSL < 3.0.
It has slightly different handling of memory, as functions such as
RSA_set0_key() take over memory management of values,
so the caller should not free the values.

Fixes: 4c7ae22f1f83 ("crypto/openssl: update DSA routine with 3.0 EVP API")
Fixes: c794b40c9258 ("crypto/openssl: update DH routine with 3.0 EVP API")
Fixes: 3b7d638fb11f ("crypto/openssl: support asymmetric SM2")
Fixes: ac42813a0a7c ("crypto/openssl: add DH and DSA asym operations")
Fixes: d7bd42f6db19 ("crypto/openssl: update RSA routine with 3.0 EVP API")
Fixes: 9d91c3047dfd ("crypto/openssl: fix memory leaks in asym operations")
Cc: stable@dpdk.org

Signed-off-by: Ciara Power <ciara.power@intel.com>
Acked-by: Kai Ji <kai.ji@intel.com>

show more ...

crypto/openssl: fix memory leaks in asym operations

Fix memory leaks in Asymmetric ops, as reported by Valgrind.

Fixes: 4c7ae22f1f83 ("crypto/openssl: update DSA routine with 3.0 EVP API")
Fixes: c

crypto/openssl: fix memory leaks in asym operations

Fix memory leaks in Asymmetric ops, as reported by Valgrind.

Fixes: 4c7ae22f1f83 ("crypto/openssl: update DSA routine with 3.0 EVP API")
Fixes: c794b40c9258 ("crypto/openssl: update DH routine with 3.0 EVP API")
Fixes: 3b7d638fb11f ("crypto/openssl: support asymmetric SM2")
Fixes: ac42813a0a7c ("crypto/openssl: add DH and DSA asym operations")
Fixes: d7bd42f6db19 ("crypto/openssl: update RSA routine with 3.0 EVP API")
Cc: stable@dpdk.org

Signed-off-by: Gowrishankar Muthukrishnan <gmuthukrishn@marvell.com>
Acked-by: Ciara Power <ciara.power@intel.com>

show more ...

crypto/openssl: remove null checks before free

Found with nullfree.cocci

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Kai Ji <kai.ji@intel.com>

cryptodev: set private and public keys in EC session

The EC private and public keys can be maintained per session,
hence, moved these keys from per packet asym op to EC xform.

Signed-off-by: Gowris

cryptodev: set private and public keys in EC session

The EC private and public keys can be maintained per session,
hence, moved these keys from per packet asym op to EC xform.

Signed-off-by: Gowrishankar Muthukrishnan <gmuthukrishn@marvell.com>
Acked-by: Ciara Power <ciara.power@intel.com>

show more ...

crypto/openssl: fix crash with OpenSSL 3

In some OpenSSL 3 libraries, uninitialized output variable
cause segfault. It is always nice to initialize it.

Bugzilla ID: 1250
Fixes: 3b7d638fb11f ("crypt

crypto/openssl: fix crash with OpenSSL 3

In some OpenSSL 3 libraries, uninitialized output variable
cause segfault. It is always nice to initialize it.

Bugzilla ID: 1250
Fixes: 3b7d638fb11f ("crypto/openssl: support asymmetric SM2")

Signed-off-by: Gowrishankar Muthukrishnan <gmuthukrishn@marvell.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>

show more ...

crypto/openssl: skip workaround at compilation time

This workaround was needed before version 1.0.1f.
Do not build it for versions >= 1.1.

Fixes: d61f70b4c918 ("crypto/libcrypto: add driver for Ope

crypto/openssl: skip workaround at compilation time

This workaround was needed before version 1.0.1f.
Do not build it for versions >= 1.1.

Fixes: d61f70b4c918 ("crypto/libcrypto: add driver for OpenSSL library")
Cc: stable@dpdk.org

Signed-off-by: Didier Pallard <didier.pallard@6wind.com>
Acked-by: Kai Ji <kai.ji@intel.com>

show more ...

crypto/openssl: fix memory leak in auth processing

Contexts allocated with EVP_MAC_CTX_new calls are leaking, they are created
then overwritten by the return value of EVP_MAC_CTX_dup call.

Fixes: 7

crypto/openssl: fix memory leak in auth processing

Contexts allocated with EVP_MAC_CTX_new calls are leaking, they are created
then overwritten by the return value of EVP_MAC_CTX_dup call.

Fixes: 75adf1eae44f ("crypto/openssl: update HMAC routine with 3.0 EVP API")
Fixes: 2b9c693f6ef5 ("crypto/openssl: support AES-CMAC operations")
Cc: stable@dpdk.org

Signed-off-by: Didier Pallard <didier.pallard@6wind.com>
Acked-by: Kai Ji <kai.ji@intel.com>

show more ...

crypto/openssl: support asymmetric SM2

Add SM2 asymmetric algorithm support in openssl PMD.

Signed-off-by: Gowrishankar Muthukrishnan <gmuthukrishn@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvel

crypto/openssl: support asymmetric SM2

Add SM2 asymmetric algorithm support in openssl PMD.

Signed-off-by: Gowrishankar Muthukrishnan <gmuthukrishn@marvell.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>

show more ...

crypto/openssl: fix memory free

Allocated memory was being freed using 'free' when it should have been
freed using 'OPENSSL_free'. This has now been modified so that the
correct method is used to fr

crypto/openssl: fix memory free

Allocated memory was being freed using 'free' when it should have been
freed using 'OPENSSL_free'. This has now been modified so that the
correct method is used to free allocated memory.

Coverity issue: 384415
Fixes: 4c7ae22f1f83 ("crypto/openssl: update DSA routine with 3.0 EVP API")
Cc: stable@dpdk.org

Signed-off-by: Saoirse O'Donovan <saoirse.odonovan@intel.com>
Signed-off-by: Ciara Power <ciara.power@intel.com>
Acked-by: Kai Ji <kai.ji@intel.com>

show more ...

crypto/openssl: fix freeing in RSA EVP

Fixed ASAN heap-use-after-free error on tmp buffer.

Fixes: d7bd42f6db19 ("crypto/openssl: update RSA routine with 3.0 EVP API")
Cc: stable@dpdk.org

Signed-of

crypto/openssl: fix freeing in RSA EVP

Fixed ASAN heap-use-after-free error on tmp buffer.

Fixes: d7bd42f6db19 ("crypto/openssl: update RSA routine with 3.0 EVP API")
Cc: stable@dpdk.org

Signed-off-by: Kai Ji <kai.ji@intel.com>
Acked-by: Ciara Power <ciara.power@intel.com>

show more ...

crypto/openssl: fix warning on copy length

When building with gcc 11.2.0, the compiler warns as follows:
In function 'memcpy',
inlined from 'openssl_set_session_auth_parameters' at
../driv

crypto/openssl: fix warning on copy length

When building with gcc 11.2.0, the compiler warns as follows:
In function 'memcpy',
inlined from 'openssl_set_session_auth_parameters' at
../drivers/crypto/openssl/rte_openssl_pmd.c:699:3,
inlined from 'openssl_set_session_parameters' at
../drivers/crypto/openssl/rte_openssl_pmd.c:826:9:
/usr/include/aarch64-linux-gnu/bits/string_fortified.h:29:10: warning:
'__builtin_memcpy' forming offset [4, 8] is out of the bounds [0, 4]

Fixed the warning by copying up to string / buffer size.

Fixes: 75adf1eae44f ("crypto/openssl: update HMAC routine with 3.0 EVP API")
Cc: stable@dpdk.org

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Feifei Wang <feifei.wang2@arm.com>

show more ...

cryptodev: hide symmetric session structure

Structure rte_cryptodev_sym_session is moved to internal
headers which are not visible to applications.
The only field which should be used by app is opaq

cryptodev: hide symmetric session structure

Structure rte_cryptodev_sym_session is moved to internal
headers which are not visible to applications.
The only field which should be used by app is opaque_data.
This field can now be accessed via set/get APIs added in this
patch.
Subsequent changes in app and lib are made to compile the code.

Signed-off-by: Akhil Goyal <gakhil@marvell.com>
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Kai Ji <kai.ji@intel.com>
Tested-by: Gagandeep Singh <g.singh@nxp.com>
Tested-by: David Coyle <david.coyle@intel.com>
Tested-by: Kevin O'Sullivan <kevin.osullivan@intel.com>

show more ...

cryptodev: rework session framework

As per current design, rte_cryptodev_sym_session_create() and
rte_cryptodev_sym_session_init() use separate mempool objects
for a single session.
And structure rt

cryptodev: rework session framework

As per current design, rte_cryptodev_sym_session_create() and
rte_cryptodev_sym_session_init() use separate mempool objects
for a single session.
And structure rte_cryptodev_sym_session is not directly used
by the application, it may cause ABI breakage if the structure
is modified in future.

To address these two issues, the rte_cryptodev_sym_session_create
will take one mempool object that the session and session private
data are virtually/physically contiguous, and initializes both
fields. The API rte_cryptodev_sym_session_init is removed.

rte_cryptodev_sym_session_create will now return an opaque session
pointer which will be used by the app and other APIs.

In data path, opaque session pointer is attached to rte_crypto_op
and the PMD can call an internal library API to get the session
private data pointer based on the driver id.

Note: currently single session may be used by different device
drivers, given it is initialized by them. After the change the
session created by one device driver cannot be used or
reinitialized by another driver.

Signed-off-by: Akhil Goyal <gakhil@marvell.com>
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Kai Ji <kai.ji@intel.com>
Tested-by: Gagandeep Singh <g.singh@nxp.com>
Tested-by: David Coyle <david.coyle@intel.com>
Tested-by: Kevin O'Sullivan <kevin.osullivan@intel.com>

show more ...

crypto/openssl: fix HMAC output length

This patch sets the max length of dst in OpenSSL 3.0 lib EVP MAC final
routine to ensure enough space for the result.

Fixes: 75adf1eae44f ("crypto/openssl: up

crypto/openssl: fix HMAC output length

This patch sets the max length of dst in OpenSSL 3.0 lib EVP MAC final
routine to ensure enough space for the result.

Fixes: 75adf1eae44f ("crypto/openssl: update HMAC routine with 3.0 EVP API")
Cc: stable@dpdk.org

Signed-off-by: Kai Ji <kai.ji@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>

show more ...