opencrypto(9): Fix missing initialization in error branch.

Reported-by: syzbot+8c519140cac567be1ee1@syzkaller.appspotmail.com

opencrypto: Assert session id is valid in crypto_freesession.

This gives us the opportunity to detect usage mistakes like
use-after-free.

Exception: Continue to silently ignore sid=0.

opencrypto: crypto_dispatch never fails now. Make it return void.

Same with crypto_kdispatch.

opencrypto: Assert driver process routine returns 0 or ERESTART.

No other errors are allowed -- other errors must be transmitted by
crypto_done. All drivers in tree (sun8i_crypto, glxsb, via_padloc

opencrypto: Assert driver process routine returns 0 or ERESTART.

No other errors are allowed -- other errors must be transmitted by
crypto_done. All drivers in tree (sun8i_crypto, glxsb, via_padlock,
mvcesa, mvxpsec, hifn, qat, ubsec, cryptosoft) have been audited for
this.

show more ...

opencrypto: Rip out EAGAIN logic when unregistering crypto drivers.

I'm pretty sure this never worked reliably based on code inspection,
and it's unlikely to have ever been tested because it only ap

opencrypto: Rip out EAGAIN logic when unregistering crypto drivers.

I'm pretty sure this never worked reliably based on code inspection,
and it's unlikely to have ever been tested because it only applies
when unregistering a driver -- but we have no crypto drivers for
removable devices, so it would only apply if we went out of our way
to trigger detach with drvctl.

Instead, just make the operation fail with ENODEV, and remove all the
callback logic to resubmit the request on EAGAIN. (Maybe this should
be ENXIO, but crypto_kdispatch already does ENODEV.)

show more ...

opencrypto: Assert nonnull callback up front in crypto_dispatch.

Same with crypto_kdispatch.

Convert some dead branches downstream to assertions too.

opencrypto: Make crypto_freesession return void.

No callers use the return value. It is not sensible to allow this to
fail.

opencrypto: Make freesession callback return void.

No functional change intended: all drivers already return zero
unconditionally.

opencrypto: Assert crp_desc and crp_buf are nonnull.

- crypto_getreq ensures crp_desc is nonnull.
- Caller is responsible for setting crp_buf.

opencrypto: Assert num>0 in crypto_getreq, num=1 in crypto_kgetreq.

- For crypto_getreq this makes downstream reasoning easier: on
success, crp_desc is guaranteed to be nonnull.

- For crypto_kget

opencrypto: Assert num>0 in crypto_getreq, num=1 in crypto_kgetreq.

- For crypto_getreq this makes downstream reasoning easier: on
success, crp_desc is guaranteed to be nonnull.

- For crypto_kgetreq, this was already assumed, just silently
ignored and not checked by anything.

show more ...

opencrypto: Nix CRYPTO_F_DONE.

Nothing uses it any more.

opencrypto: Make sid=0 always invalid, but OK to free.

Previously, crypto_newsession could sometimes return 0 as the
driver-specific part of the session id, and 0 as the hid, for sid=0.
But netipsec

opencrypto: Make sid=0 always invalid, but OK to free.

Previously, crypto_newsession could sometimes return 0 as the
driver-specific part of the session id, and 0 as the hid, for sid=0.
But netipsec assumes that it is always safe to free sid=0 from
zero-initialized memory even if crypto_newsession has never
succeeded. So it was up to every driver in tree to gracefully handle
sid=0, if it happened to get assigned hid=0. And, as long as the
freesession callback was expected to just return an error code when
given a bogus session id, that worked out fine...because nothing ever
used the error code.

That was a terrible fragile system that should never have been
invented. Instead, let's just ensure that valid session ids are
nonzero, and make crypto_freesession with sid=0 be a no-op.

show more ...

opencrypto: Assert !cpu_intr_p() on dispatch and invoke.

These should only ever have been potentially called from hard
interrupt context by CRYPTO_F_CBIMM callbacks (CBIMM = call back
immediately).

opencrypto: Assert !cpu_intr_p() on dispatch and invoke.

These should only ever have been potentially called from hard
interrupt context by CRYPTO_F_CBIMM callbacks (CBIMM = call back
immediately). CRYPTO_F_CBIMM is no more, so there is no more need to
allow this case of call from hard interrupt context.

show more ...

opencrypto: Nix CRYPTO_F_USER, CRYPTO_F_CBIMM, CRYPTO_F_CBIFSYNC.

CRYPTO_F_USER is no longer needed. It was introduced in 2008 by
darran@ in crypto.c 1.30, cryptodev.c 1.45 in an attempt to avoid
d

opencrypto: Nix CRYPTO_F_USER, CRYPTO_F_CBIMM, CRYPTO_F_CBIFSYNC.

CRYPTO_F_USER is no longer needed. It was introduced in 2008 by
darran@ in crypto.c 1.30, cryptodev.c 1.45 in an attempt to avoid
double-free between the issuing thread and asynchronous callback.
But the `fix' didn't work. In 2017, knakahara@ fixed it properly in
cryptodev.c 1.87 by distinguishing `the crypto operation has
completed' (CRYPTO_F_DONE) from `the callback is done touching the
crp object' (CRYPTO_F_DQRETQ, now renamed to CRYPTODEV_F_RET).

CRYPTO_F_CBIMM formerly served to invoke the callback synchronously
from the driver's interrupt completion routine, to reduce contention
on what was once a single cryptoret thread. Now, there is a per-CPU
queue and softint for much cheaper processing, so there is less
motivation for this in the first place. So let's remove the
complicated logic. This means the callbacks never run in hard
interrupt context, which means we don't need to worry about recursion
into crypto_dispatch in hard interrupt context.

show more ...

opencrypto: Factor setting CRYPTO_F_DONE out of branches.

This had been done in 1.30 when the locking was different. No need
any more. No functional change intended.

fix typo in CRK_ALGORITHM_MIN definition to match CRK_ALGORITHM_MAX one.
while here fix few typos in comments.

fix typos in asymmetry, asymmetric(al), symmetrical.

Revert previous change to use SYSCTL_SETUP since it breaks on macppc.

For some reason, the crypto module fails to link, and this results in
opencrypto sysctl failures.

Should resolve PR kern/55154

Use the module subsystem's ability to process SYSCTL_SETUP() entries to
automate installation of sysctl nodes.

Note that there are still a number of device and pseudo-device modules
that create entr

Use the module subsystem's ability to process SYSCTL_SETUP() entries to
automate installation of sysctl nodes.

Note that there are still a number of device and pseudo-device modules
that create entries tied to individual device units, rather than to the
module itself. These are not changed.

show more ...

softint_disestablish does xc_barrier(0) for us already.

Switch opencrypto to percpu_create.

Can't sleep for allocation in percpu_foreach.

xc_barrier - convenience function to xc_broadcast() a nop.

Make the intent more clear and also avoid a bunch of (xcfunc_t)nullop
casts that gcc 8 -Wcast-function-type is not happy about.

in many device attach paths, allocate memory with KM_SLEEP instead of KM_NOSLEEP
and remove code to handle failures that can no longer happen.

relinguish our lock while we are autoloading.

Try to load swcrypto if we we did not find any software drivers.