Rename nostop to after_sleep in sleep_signal_check() and cleanup some
comments to better match reality.
OK mpi@

Do not call proc_stop in the single_thread_check branch of sleep_signal_check

proc_stop still does a lot more than just stopping a thread. In the
single_thread_check branch the only thing the code n

Do not call proc_stop in the single_thread_check branch of sleep_signal_check

proc_stop still does a lot more than just stopping a thread. In the
single_thread_check branch the only thing the code needs to do is to
set p_stat to SSTOP.

show more ...

Make single_thread_check() always return when deep is true and not suspend
the curproc.

There are only two cases where deep != 0. One is the sleep API the other
is single_thread_set depending on fla

Make single_thread_check() always return when deep is true and not suspend
the curproc.

There are only two cases where deep != 0. One is the sleep API the other
is single_thread_set depending on flag. For single_thread_set() the error
never matters if SINGLE_DEEP is set. The call in exec will always return
ERESTART and the ones for pledge et al. do no error checking at all.

For the sleep API sleep_signal_check() now checks the returned errno and
for the new EWOULDBLOCK case it will either stop or ignore the error
depending on the nostop flag.

Also just return ERESTART for the unwind and exit cases since there is
no need to differentiate between the two.

On top of this check the proc p_stat before returning EWOULDBLOCK and
return 0 in case the thread is already stopped. This happens when a
thread calls sleep_setup() and then right after that single_thread_set()
is executed by another thread. The first thread is then immediatly put
to SSTOP and so sleep_signal_check() no longer needs to do something.
sleep_finish() will take care. This should fix debugging of multithreaded
processes.

OK mpi@

show more ...

cursig() can return a normally ignored signal if the process is ptraced.
So make sure that sleep_signal_check() returns ERESTART in that case so
that the syscall is retried once ptrace intercepted th

cursig() can return a normally ignored signal if the process is ptraced.
So make sure that sleep_signal_check() returns ERESTART in that case so
that the syscall is retried once ptrace intercepted the signal.

This should fix unexpected EINTR returns of waitpid for precesses that
left SIGCHLD ignored (default). Not the perfect fix but a good enough
bandaid to allow people to debug processes doing forks and waitpid calls.

Problem reported and fix tested by stsp@
OK kettenis@ stsp@

show more ...

avoid lock contention in __thrsleep and __thrwakeup syscalls

turns out the __thrsleep and __thrwakeup syscalls largely coordinate
using a single lock per process. if you have heavily threaded code
b

avoid lock contention in __thrsleep and __thrwakeup syscalls

turns out the __thrsleep and __thrwakeup syscalls largely coordinate
using a single lock per process. if you have heavily threaded code
building locks in userland out of thrsleep, this kernel lock gets
hammered. this is true even if userland thinks it's operating on
separate locks, it all ends up serialised in the kernel. this reduces
the throughput of these heavily threaded programs.

the big change is hashing thrsleep waiters into an different
locks/lists based on their "id" to try and avoid all locks in a
process contending on a single lock. the hash is shared by all
processes though.

the change also avoids having a waiter re-take the lock to avoid
contention on the thrwakeup code which is currently holding the lock.

__thrsleep and __thrwakeup seem to be largely unused these days,
except by go. go still uses it as a backend to it's locks, and also
creates a lot of threads which end up contending on the lock. these
changes provide an improvement for go programs.

the contention was pointed out by nick owens
jsing@ and nick owens did a bit of testing

show more ...

If nostop is set properly ignore stop signals with default handlers.

The check right now is in the wrong spot so fix this.
OK mpi@

Before stopping a thread because of stop signal set ps_xsig to the signum
so that wait(2) returns the right signal.
Found and tested by sthen@

sleep_signal_check() is called twice in sleep_finish().

The first time a pending stop should result in a stopped thread but
the second call should not call proc_stop since the sleep is done and
the

sleep_signal_check() is called twice in sleep_finish().

The first time a pending stop should result in a stopped thread but
the second call should not call proc_stop since the sleep is done and
the proc should continue as SONPROC until it hits the next cursig call.

Fix for a panic seen by bluhm@
OK mpi@

show more ...

Properly handle stop signals in cursig if deep.

In setsigctx() set sig_stop to 1 if the process should be stopped.
In cursig() still return early if deep but then in sleep_signal_check()
use this in

Properly handle stop signals in cursig if deep.

In setsigctx() set sig_stop to 1 if the process should be stopped.
In cursig() still return early if deep but then in sleep_signal_check()
use this information to call proc_stop and stop the proc.
This should fix the problem in the waitid regress test.
OK mpi@

show more ...

Need to call unsleep before doing the SSTOP check. We need to ensure that
if a sleep is interrupted but the thread is also stopped that on a wakeup
the thread runs again.
OK mpi@

In sleep_finish() is the process state is SSTOP force a mi_switch().

A SIGSTOP delivered between sleep_setup() and sleep_finish() will
just change the proc state to SSTOP and sleep_finis() needs to

In sleep_finish() is the process state is SSTOP force a mi_switch().

A SIGSTOP delivered between sleep_setup() and sleep_finish() will
just change the proc state to SSTOP and sleep_finis() needs to respect
that and make sure the thread stays stopped until a SIGCONT is issued.
OK mpi@

show more ...

Shortcut cursig when called during sleep setup.

Add deep flag as function argument which is used by the sleep API but
nowhere else. Both calls to sleep_signal_check() should skip the ugly
bits of cu

Shortcut cursig when called during sleep setup.

Add deep flag as function argument which is used by the sleep API but
nowhere else. Both calls to sleep_signal_check() should skip the ugly
bits of cursig().

In cursig() if deep once it is clear a signal will be taken keep the
signal on the thread siglist and return. sleep_signal_check() will then
return EINTR or ERESTART based on the signal context. There is no reason
to do more in this special case. Especially stop/cont and the ptrace trap
must be skipped here. Once the call makes it to userret the signal will be
picked up again and handled in a safe location.

Stopping singals need some additional logic since we don't want to abort
the sleep just to stop a process. Since our SIGSTOP handling requires
a major rewrite this will be posponed until then.

OK mpi@

show more ...

Pass curproc pointer down from sleep_finish() instead of pulling it in
again in sleep_signal_check().
OK dlg@

Remove the now unsued s argument to SCHED_LOCK and SCHED_UNLOCK.

The SPL level is not tacked by the mutex and we no longer need to track
this in the callers.
OK miod@ mlarkin@ tb@ jca@

When clearing the wait channel also clear the wait message.

There is no reason to keep the wait message in place since it will
never show up even in ddb show proc output.
OK jca@

Rework interaction between sleep API and exit1() and start unlocking ps_threads

This diff adjusts how single_thread_set() accounts the threads by using
ps_threadcnt as initial value and counting all

Rework interaction between sleep API and exit1() and start unlocking ps_threads

This diff adjusts how single_thread_set() accounts the threads by using
ps_threadcnt as initial value and counting all threads out that are already
parked. In single_thread_check call exit1() before decreasing ps_singlecount
this is now done in exit1().

exit1() and thread_fork() ensure that ps_threadcnt is updated with the
pr->ps_mtx held and in exit1() also account for exiting threads since
exit1() can sleep.

OK mpi@

show more ...

Clear PCATCH for procs that have P_WEXIT set.

Exiting procs will not return to userland and can not deliver signals so
it is better to not even try.
OK mpi@

Prevent a recursion inside wakeup(9) when scheduler tracepoints are enabled.

Tracepoints like "sched:enqueue" and "sched:unsleep" were called from inside
the loop iterating over sleeping threads as

Prevent a recursion inside wakeup(9) when scheduler tracepoints are enabled.

Tracepoints like "sched:enqueue" and "sched:unsleep" were called from inside
the loop iterating over sleeping threads as part of wakeup_proc(). When such
tracepoints were enabled they could result in another wakeup(9) possibly
corrupting the sleepqueue.

Rewrite wakeup(9) in two stages, first dequeue threads from the sleepqueue then
call setrunnable() and possible tracepoints for each of them.

This requires moving unsleep() outside of setrunnable() because it messes with
the sleepqueue.

ok claudio@

show more ...

Revert commitid: yfAefyNWibUyjkU2, ESyyH5EKxtrXGkS6 and itscfpFvJLOj8mHB;

The change to the single thread API results in crashes inside exit1()
as found by Syzkaller. There seems to be a race in the

Revert commitid: yfAefyNWibUyjkU2, ESyyH5EKxtrXGkS6 and itscfpFvJLOj8mHB;

The change to the single thread API results in crashes inside exit1()
as found by Syzkaller. There seems to be a race in the exit codepath.
What exactly fails is not really clear therefor revert for now.

This should fix the following Syzkaller reports:
Reported-by: syzbot+38efb425eada701ca8bb@syzkaller.appspotmail.com
Reported-by: syzbot+ecc0e8628b3db39b5b17@syzkaller.appspotmail.com
and maybe more.

Reverted commits:
----------------------------
Protect ps_single, ps_singlecnt and ps_threadcnt by the process mutex.

The single thread API needs to lock the process to enter single thread
mode and does not need to stop the scheduler.

This code changes ps_singlecount from a count down to zero to ps_singlecnt
which counts up until equal to ps_threadcnt (in which case all threads
are properly asleep).

Tested by phessler@, OK mpi@ cheloha@
----------------------------
Change how ps_threads and p_thr_link are locked away from using SCHED_LOCK.

The per process thread list can be traversed (read) by holding either
the KERNEL_LOCK or the per process ps_mtx (instead of SCHED_LOCK).
Abusing the SCHED_LOCK for this makes it impossible to split up the
scheduler lock into something more fine grained.

Tested by phessler@, ok mpi@
----------------------------
Fix SCHED_LOCK() leak in single_thread_set()

In the (q->p_flag & P_WEXIT) branch is a continue that did not release
the SCHED_LOCK. Refactor the code a bit to simplify the places SCHED_LOCK
is grabbed and released.

Reported-by: syzbot+ea26d351acfad3bb3f15@syzkaller.appspotmail.com
OK kettenis@

show more ...

Change how ps_threads and p_thr_link are locked away from using SCHED_LOCK.

The per process thread list can be traversed (read) by holding either
the KERNEL_LOCK or the per process ps_mtx (instead o

Change how ps_threads and p_thr_link are locked away from using SCHED_LOCK.

The per process thread list can be traversed (read) by holding either
the KERNEL_LOCK or the per process ps_mtx (instead of SCHED_LOCK).
Abusing the SCHED_LOCK for this makes it impossible to split up the
scheduler lock into something more fine grained.

Tested by phessler@, ok mpi@

show more ...

Move SCHED_LOCK after sleep_signal_check.

sleep_signal_check() is there to look for pending signals / single thread
requests which were posted before sleep_setup() finished. Once p_stat
is set to SS

Move SCHED_LOCK after sleep_signal_check.

sleep_signal_check() is there to look for pending signals / single thread
requests which were posted before sleep_setup() finished. Once p_stat
is set to SSLEEP the wakeup and delivery of signals is taken care of
by ptsignal and single_thread_set().

Moving the SCHED_LOCK further down allows to cleanup cursig() and to
remove a SCHED_LOCK recursion in single_thread_check().

OK mpi@

show more ...

Extend scheduler tracepoints to follow CPU jumping.

- Add two new tracpoints sched:fork & sched:steal
- Include selected CPU number in sched:wakeup
- Add sched:unsleep corresponding to sched:sleep w

Extend scheduler tracepoints to follow CPU jumping.

- Add two new tracpoints sched:fork & sched:steal
- Include selected CPU number in sched:wakeup
- Add sched:unsleep corresponding to sched:sleep which matches add/removal
of threads on the sleep queue

ok claudio@

show more ...

Add some KASSERT on the proc p_stat in sleep_finish()
OK mpi@

struct sleep_state is no longer used, remove it.
Also remove the priority argument to sleep_finish() the code can use
the p_flag P_SINTR flag to know if the signal check is needed or not.
OK cheloha@

struct sleep_state is no longer used, remove it.
Also remove the priority argument to sleep_finish() the code can use
the p_flag P_SINTR flag to know if the signal check is needed or not.
OK cheloha@ kettenis@ mpi@

show more ...

Rework sleep_setup()/sleep_finish() to no longer hold the scheduler lock
between calls.

Instead of forcing an atomic operation across multiple calls use a three
step transaction.
1. setup sleep stat

Rework sleep_setup()/sleep_finish() to no longer hold the scheduler lock
between calls.

Instead of forcing an atomic operation across multiple calls use a three
step transaction.
1. setup sleep state by calling sleep_setup()
2. recheck sleep condition to ensure that the event did not fire before
sleep_setup() registered the proc onto the sleep queue
3. call sleep_finish() to either sleep or keep on running based on the
step 2 outcome and any possible signal delivery

To make this work wakeup from signals, single thread api and wakeup(9) need
to be aware if a process is between step 1 and step 3 so that the process
is not enqueued back onto the runqueue while going to sleep. Introduce
the p_flag P_WSLEEP to detect this situation.

On top of this remove the spl dance in msleep() which is no longer required.
It is ok to process interrupts between step 1 and 3.

OK mpi@ cheloha@

show more ...