Remove non-MULTIPROCESSOR chunk defining copyin32 as copyin now that all
platforms provide it.

remove unneeded includes in sys/kern
ok mpi@ miod@

Let futex_wait() run without kernel lock

The KERNEL_LOCK() is no longer necessary with rwsleep() and PCATCH
because the sleep machinery now does the locking internally.

OK mpi@

Make futexes work in shared anonymous memory.

ok mpi@

Fix the return value for the FUTEX_WAIT/FUTEX_WAIT_PRIVATE futex(2)
operation. System calls should return -1 and set errno when they fail.
They should not return an errno value directly. This match

Fix the return value for the FUTEX_WAIT/FUTEX_WAIT_PRIVATE futex(2)
operation. System calls should return -1 and set errno when they fail.
They should not return an errno value directly. This matches how
the Linux version of futex(2) behaves and what Mesa expects. This fixes
a bug in Mesa where a timeout wouldn't be reported properly.

Technically this is an ABI break. But libc and libpthread were changed
to be compatible with both the old and new ABI, and code outside of base
almost certainly expects Linux compatible behaviour. If you have not
rebuilt libc and the last few days, upgrade using a snap.

Mesa issue discovered by jsg@
ok mpi@, deraadt@

show more ...

spelling

ok gnezdo@ semarie@ mpi@

futex(2): FUTEX_WAIT: rwsleep_nsec(9) at least one nanosecond

mpi@ and I added a warning log to *sleep_nsec(9) last year to smoke
out division-to-zero bugs when converting kernel code from *sleep(9)

futex(2): FUTEX_WAIT: rwsleep_nsec(9) at least one nanosecond

mpi@ and I added a warning log to *sleep_nsec(9) last year to smoke
out division-to-zero bugs when converting kernel code from *sleep(9)
to the new interfaces. It whines if you tell it to sleep for zero
nanoseconds.

Now that rwsleep_nsec(9) is exposed to userspace via futex(2), though,
it is possible to get a legitimate zero-nanosecond timeout from the
user. This can cause a lot of logging, which apparently can cause
hiccups and hangs in Mesa.

As a quick fix we can round the timeout up to one nanosecond and
silence the warning. No logs, no delays, no hiccups or hangs.

--

Aside: it is unclear what we are supposed to do in the FUTEX_WAIT
zero-nanosecond timeout case: block for a tick or return ETIMEDOUT
immediately. The Linux futex(2) manpage does not mention the case.
It'd be nice to knew what the proper behavior is.

--

Prompted by matthieu@. Input from kettenis@ and deraadt@.

Tested by matthieu@, ajacoutot@.

In snaps since Mar 27 2020.

ok ajacoutot@, deraadt@, kettenis@.

show more ...

futex(2): futex_wait(): ensure timeout is set when calling rwsleep_nsec(9)

rwsleep_nsec(9) will not set a timeout if the nsecs parameter is
equal to INFSLP (UINT64_MAX). We need to limit the durati

futex(2): futex_wait(): ensure timeout is set when calling rwsleep_nsec(9)

rwsleep_nsec(9) will not set a timeout if the nsecs parameter is
equal to INFSLP (UINT64_MAX). We need to limit the duration to
MAXTSLP (UINT64_MAX - 1) to ensure a timeout is set.

show more ...

Introduce TIMESPEC_TO_NSEC() and use it to convert userland facing
tsleep(9) to tsleep_nsec(9).

ok bluhm@

Stop sleeping at PUSER.

This allows to enforce that sleeping priorities will now always be <
PUSER.

ok visa@, ratchov@

Use ktrreltimespec() as the timeout is relative, pointed by matthew@.

ok cheloha@

Sprinkle a pinch of timerisvalid/timespecisvalid over the rest of sys/kern

futex(2): validate relative timeout before sleeping.

Linux does validation.

Document this new failure case as an EINVAL, like Linux.

"stop waiting" deraadt

Split the system-wide list of all futexes into process-specific lists
of private futexes and a shared list of shared futexes. This speeds up
futex lookups.

Tested by and OK krw@
OK mpi@

Implement process-shared futexes. These are now the default implementation.
Introduce process-private futexs. These will bypass the uvm mapping lookup
and should be slightly more efficient.

ok vis

Implement process-shared futexes. These are now the default implementation.
Introduce process-private futexs. These will bypass the uvm mapping lookup
and should be slightly more efficient.

ok visa@, mpi@

show more ...

Make futex(2) also return ECANCELED in case SA_RESTART was set.

Prodded by and ok mpi@

Do not pass a userland pointer to ktrabstimespec().

Prevents an infinite pagefault/pmap_enter() loop when ktracing apps
doing a lot of futex(2) calls like firefox & chrome.

curproc access isn't necessarily as cheap as a local variable access,
so only get it once
ok guenther

New flag PR_RWLOCK for pool_init(9) makes the pool use rwlocks instead
of mutexes. Use this immediately for the pool_cache futex pools.

Mostly worked out with dlg@ during e2k17
ok mpi@ tedu@

Use copyin32(9) to atomically copy the futex from user space.

On !MULTIPROCESSOR kernels we still fall back on copyin(9), but that is
fine. This will break m88k MULTIPROCESSOR kernels.

ok deraadt@

Use copyin32(9) to atomically copy the futex from user space.

On !MULTIPROCESSOR kernels we still fall back on copyin(9), but that is
fine. This will break m88k MULTIPROCESSOR kernels.

ok deraadt@, mpi@, visa@

show more ...

Return ENOSYS for unsupported operation.

Add futex(2) syscall based on a sane subset of its Linux equivalent.

The syscall is marked NOLOCK and only FUTEX_WAIT grabs the KERNEL_LOCK()
because of PCATCH and the signal nightmare.

Serializati

Add futex(2) syscall based on a sane subset of its Linux equivalent.

The syscall is marked NOLOCK and only FUTEX_WAIT grabs the KERNEL_LOCK()
because of PCATCH and the signal nightmare.

Serialization of threads is currently done with a global & exclusive
rwlock.

Note that the current implementation still use copyin(9) which is not
guaranteed to be atomic. Committing now such that remaining issues can
be addressed in-tree.

With inputs from guenther@, kettenis@ and visa@.

ok deraadt@, visa@

show more ...