remove __mp_release_all_but_one(), unused since sched_bsd.c rev 1.92
ok claudio@

Convert SCHED_LOCK from a recursive kernel lock to a mutex.

Over the last weeks the last SCHED_LOCK recursion was removed so this
is now possible and will allow to split up the SCHED_LOCK in a upcom

Convert SCHED_LOCK from a recursive kernel lock to a mutex.

Over the last weeks the last SCHED_LOCK recursion was removed so this
is now possible and will allow to split up the SCHED_LOCK in a upcoming
step.

Instead of implementing an MP and SP version of SCHED_LOCK this just
always uses the mutex implementation.
While this makes the local s argument unused (the spl is now tracked by
the mutex itself) it is still there to keep this diff minimal.

Tested by many.
OK jca@ mpi@

show more ...

Improve spinning in mtx_enter().

Instead of calling mtx_enter_try() in each spinning loop, do it
only if the result of a lockless read indicates that the mutex has
been released. This avoids some e

Improve spinning in mtx_enter().

Instead of calling mtx_enter_try() in each spinning loop, do it
only if the result of a lockless read indicates that the mutex has
been released. This avoids some expensive atomic compare-and-swap
operations. Up to 5% reduction of spinning time during kernel build
can been seen on a 8 core amd64 machine. On other machines there
was no visible effect.

Test on powerpc64 has revealed a bug in mtx_owner declaration. Not
the variable was volatile, but the object it points to. Move the
volatile declaration in struct mutex to avoid a hang when going to
multiuser.

from Mateusz Guzik; input kettenis@ jca@; OK mpi@

show more ...

Bump __mp_lock_spinout to INT_MAX.

The previous value set years ago was causing amd64 kernels to spin
out when run with MP_LOCKDEBUG during boot.

ok kettenis@

The 'lock spun out' db_printf needs a newline. All other MP_LOCKDEBUG
messages do have the newline already.
OK anton@ kettenis@

Let SP kernel work with WITNESS. The necessary instrumentation was
missing from the SP variant of mtx_enter() and mtx_enter_try().
mtx_leave() was correct already.

Prompted by and OK patrick@

Remove file name and line number output from witness(4)

Reduce code clutter by removing the file name and line number output
from witness(4). Typically it is easy enough to locate offending locks
us

Remove file name and line number output from witness(4)

Reduce code clutter by removing the file name and line number output
from witness(4). Typically it is easy enough to locate offending locks
using the stack traces that are shown in lock order conflict reports.
Tricky cases can be tracked using sysctl kern.witness.locktrace=1 .

This patch additionally removes the witness(4) wrapper for mutexes.
Now each mutex implementation has to invoke the WITNESS_*() macros
in order to utilize the checker.

Discussed with and OK dlg@, OK mpi@

show more ...

Add a simple spinning mutex for ddb. Unlike mutex(9), this lock keeps
on spinning even if `db_active' or `panicstr' has been set. The new
mutex also disables IPIs in the critical section.

OK mpi@ pa

Add a simple spinning mutex for ddb. Unlike mutex(9), this lock keeps
on spinning even if `db_active' or `panicstr' has been set. The new
mutex also disables IPIs in the critical section.

OK mpi@ patrick@

show more ...

Fix memory barrier in __mtx_leave(). membar_exit_before_atomic() cannot
be used in the routine because there is no subsequent atomic operation.
membar_exit() has to be used instead.

The mistake has

Fix memory barrier in __mtx_leave(). membar_exit_before_atomic() cannot
be used in the routine because there is no subsequent atomic operation.
membar_exit() has to be used instead.

The mistake has not caused problems because on most platforms
membar_exit_before_atomic() is membar_exit(). Only amd64 and i386 have
a dedicated membar_exit_before_atomic(), and their exit barriers are
no-ops.

OK dlg@

show more ...

Simplify #ifdefs. The kernel_lock symbol is no longer needed when
building a uniprocessor kernel with WITNESS.

OK mpi@

Constipate all the struct lock_type's so they go into .rodata

ok visa@

Stopping counting and reporting CPU time spent spinning on a lock as
system time.

Introduce a new CP_SPIN "scheduler state" and modify userland tools
to display the % of timer a CPU spents spinning.

Stopping counting and reporting CPU time spent spinning on a lock as
system time.

Introduce a new CP_SPIN "scheduler state" and modify userland tools
to display the % of timer a CPU spents spinning.

Based on a diff from jmatthew@, ok pirofti@, bluhm@, visa@, deraadt@

show more ...

Drop into ddb(4) if pmap_tlb_shoot*() take too much time in MP_LOCKDEBUG
kernels.

While here sync all MP_LOCKDEBUG/while loops.

ok mlarkin@, visa@

Teach mtx_enter_try(9) to avoid deadlocks after a panic.

ok deraadt@

Try harder to execute code protected by mutexes after entering ddb(4).

Should prevent a panic after panic reported by mlarkin@.

ok mlarkin@, visa@

Do not panic from ddb(4) when a lock requirement isn't fulfilled.

Extend the logic already present for panic() to any DDB-related
operation such that if ddb(4) is entered because of a fault or
other

Do not panic from ddb(4) when a lock requirement isn't fulfilled.

Extend the logic already present for panic() to any DDB-related
operation such that if ddb(4) is entered because of a fault or
other trap it is still possible to call 'boot reboot'.

While here stop printing splassert() messages as well, to not fill
the buffer.

ok visa@, deraadt@

show more ...

Include <sys/mutex.h> directly instead of relying on other headers to
include it.

Directly include sys/mplock.h when needed instead of depending on
indirect inclusion. Fixes non-MULTIPROCESSOR WITNESS build.

ok visa@ mpi@

Put WITNESS only functions with the rest of the locking primitives.

Merge license blocks now that they are identical.

Artur Grabowski agreed to relicense his C mutex implementation under ISC.

This will prevent a copyright-o-rama in kern_lock.c

Remove CSRG copyright, there isn't any code left from Berkeley here.

In 2016 natano@ removed the last two functions remaining from the CSRG
time: lockinit() and lockstatus(). At that time they were

Remove CSRG copyright, there isn't any code left from Berkeley here.

In 2016 natano@ removed the last two functions remaining from the CSRG
time: lockinit() and lockstatus(). At that time they were already wrappers
around recursive rwlocks functions from thib@ that tedu@ committed in 2013.

ok deraadt@

show more ...

Move common mutex implementations to a MI place.

Archs not yet converted can to the jump by defining __USE_MI_MUTEX.

ok visa@

Change __mp_lock_held() to work with an arbitrary CPU info structure and
extend ddb(4) "ps /o" output to print which CPU is currently holding the
KERNEL_LOCK().

Tested by dhill@, ok visa@

Add a machine-independent implementation for the mplock.
This reduces code duplication and makes it easier to instrument
lock primitives.

The MI mplock uses the ticket lock code that has been in use

Add a machine-independent implementation for the mplock.
This reduces code duplication and makes it easier to instrument
lock primitives.

The MI mplock uses the ticket lock code that has been in use
on amd64, i386 and sparc64. These are the architectures that now
switch to the MI code.

The lock_machdep.c files are unhooked from the build but not
removed yet, in case something goes wrong.

OK mpi@, kettenis@

show more ...