replace uvm.h includes with uvm_extern.h where possible

Replace ps_oppid (old parent pid) with ps_opptr (a pointer to the old parent process struct).

This simplifies some code and removes the no need to call prfind in
process_untrace().
OK mpi@

Adjust lock requirements for ps_pptr, ps_ppid and ps_oppid.

ps_pptr, ps_ppid and ps_oppid require the KERNEL_LOCK and the process mutex
to be modified. At the same time either KERNEL_LOCK or process

Adjust lock requirements for ps_pptr, ps_ppid and ps_oppid.

ps_pptr, ps_ppid and ps_oppid require the KERNEL_LOCK and the process mutex
to be modified. At the same time either KERNEL_LOCK or process mutex needs
to be taken to read the values.

This is needed to further unlock ptsignal().
OK kettenis@, mvs@

show more ...

We do not need the PS_LIBCPIN and PS_PIN flag fields anymore, which were
used during devlopment (for visibility). There is speculation claudio will
immediately use these bits for something else.

Unlock KERN_MAXPROC and KERN_MAXTHREAD from `kern_vars'. Both
`maxprocess' and `maxthread' are atomically accessed integers.

ok mpi

Atomically modify `nthreads' and move proc_free() out of KERNEL_LOCK().

ok jca@

Make exit1() wait sysctl(2) `allprocess' loops.

Regardless on wired userland memory, KERN_FILE_BYPID and KERN_FILE_BYUID
`allprocess' loops have netlock provided sleep points, so concurrent
process

Make exit1() wait sysctl(2) `allprocess' loops.

Regardless on wired userland memory, KERN_FILE_BYPID and KERN_FILE_BYUID
`allprocess' loops have netlock provided sleep points, so concurrent
process exit(1) could crash kernel.

The main exit1() problem is that process teardown begins while process
is still linked to `allprocess' list, and current code doesn't allow to
unlink it first. Wait for concurrent sysctl(2) `allprocess' loops
between PS_EXITING bit setting and list unlinking. Both KERN_FILE_BYPID
and KERN_FILE_BYUID loops do PS_EXITING check and won't deal with dying
process. Concurrent exit1() thread will wait loops keeping process
linked to `allprocess' list.

Tested with i386 dpb(1) run.
Stress tests and ok bluhm.

show more ...

Stop using KERNEL_LOCK to protect the per process kqueue list

Instead of the KERNEL_LOCK use the ps_mtx for most operations.
If the ps_klist is modified an additional global rwlock (kqueue_ps_list_l

Stop using KERNEL_LOCK to protect the per process kqueue list

Instead of the KERNEL_LOCK use the ps_mtx for most operations.
If the ps_klist is modified an additional global rwlock (kqueue_ps_list_lock)
is required. This includes the knotes with NOTE_FORK and NOTE_EXIT since
in either cases a ps_klist is changed. In the NOTE_FORK | NOTE_TRACK case
the call to kqueue_register() can sleep this is why a global rwlock is used.

Adjust the reaper() to call knote_processexit() without KERNEL_LOCK.
Double lock idea from visa@
OK mvs@

show more ...

Remove the now unsued s argument to SCHED_LOCK and SCHED_UNLOCK.

The SPL level is not tacked by the mutex and we no longer need to track
this in the callers.
OK miod@ mlarkin@ tb@ jca@

Convert SCHED_LOCK from a recursive kernel lock to a mutex.

Over the last weeks the last SCHED_LOCK recursion was removed so this
is now possible and will allow to split up the SCHED_LOCK in a upcom

Convert SCHED_LOCK from a recursive kernel lock to a mutex.

Over the last weeks the last SCHED_LOCK recursion was removed so this
is now possible and will allow to split up the SCHED_LOCK in a upcoming
step.

Instead of implementing an MP and SP version of SCHED_LOCK this just
always uses the mutex implementation.
While this makes the local s argument unused (the spl is now tracked by
the mutex itself) it is still there to keep this diff minimal.

Tested by many.
OK jca@ mpi@

show more ...

Rework interaction between sleep API and exit1() and start unlocking ps_threads

This diff adjusts how single_thread_set() accounts the threads by using
ps_threadcnt as initial value and counting all

Rework interaction between sleep API and exit1() and start unlocking ps_threads

This diff adjusts how single_thread_set() accounts the threads by using
ps_threadcnt as initial value and counting all threads out that are already
parked. In single_thread_check call exit1() before decreasing ps_singlecount
this is now done in exit1().

exit1() and thread_fork() ensure that ps_threadcnt is updated with the
pr->ps_mtx held and in exit1() also account for exiting threads since
exit1() can sleep.

OK mpi@

show more ...

clockintr: switch from callee- to caller-allocated clockintr structs

Currently, clockintr_establish() calls malloc(9) to allocate a
clockintr struct on behalf of the caller. mpi@ says this behavior

clockintr: switch from callee- to caller-allocated clockintr structs

Currently, clockintr_establish() calls malloc(9) to allocate a
clockintr struct on behalf of the caller. mpi@ says this behavior is
incompatible with dt(4). In particular, calling malloc(9) during the
initialization of a PCB outside of dt_pcb_alloc() is (a) awkward and
(b) may conflict with future changes/optimizations to PCB allocation.

To side-step the problem, this patch changes the clockintr subsystem
to use caller-allocated clockintr structs instead of callee-allocated
structs.

clockintr_establish() is named after softintr_establish(), which uses
malloc(9) internally to create softintr objects. The clockintr subsystem
is no longer using malloc(9), so the "establish" naming is no longer apt.
To avoid confusion, this patch also renames "clockintr_establish" to
"clockintr_bind".

Requested by mpi@. Tweaked by mpi@.

Thread: https://marc.info/?l=openbsd-tech&m=170597126103504&w=2

ok claudio@ mlarkin@ mpi@

show more ...

Backout priterator() for walking allprocess list.

This approach does not work as LIST_NEXT() of a removed element
does not return NULL. I causes a crash in syzcaller and triggers
kernel diagnostic

Backout priterator() for walking allprocess list.

This approach does not work as LIST_NEXT() of a removed element
does not return NULL. I causes a crash in syzcaller and triggers
kernel diagnostic assertion "vp->v_uvcount == 0" in sys/kern/kern_unveil.c
line 845 during reboot. Unfortunately the backout brings back the
race in fill_file() and fstat(1) may crash the kernel.

Reported-by: syzbot+54fba1c004d7383d5e85@syzkaller.appspotmail.com

show more ...

The kernel will now read pinsyscall tables out of PT_OPENBSD_SYSCALLS in
the main program or ld.so, and accept a submission of that information
for libc.so from ld.so via pinsyscalls(2). At system c

The kernel will now read pinsyscall tables out of PT_OPENBSD_SYSCALLS in
the main program or ld.so, and accept a submission of that information
for libc.so from ld.so via pinsyscalls(2). At system call invocation,
the syscall number is matched to the specific address it must come from.
ok kettenis, gnezdo, testing of variations by many people

show more ...

Introduce priterator(), the `ps_list' iterator. Some of `allprocess'
list walkthroughs have context switch within, so make exit1() wait
until the last reference released.

Reported-by: syzbot+0e9dda7

Introduce priterator(), the `ps_list' iterator. Some of `allprocess'
list walkthroughs have context switch within, so make exit1() wait
until the last reference released.

Reported-by: syzbot+0e9dda76c42c82c626d7@syzkaller.appspotmail.com

ok bluhm claudio

show more ...

Normally context switches happen in mi_switch() but there are 3 cases
where a switch happens outside. Cleanup these code paths and make the
machine independent.

- when a process forks (fork, tfork,

Normally context switches happen in mi_switch() but there are 3 cases
where a switch happens outside. Cleanup these code paths and make the
machine independent.

- when a process forks (fork, tfork, kthread), the new proc needs to
somehow be scheduled for the first time. This is done by proc_trampoline.
Since proc_trampoline is machine dependent assembler code change
the MP specific proc_trampoline_mp() to proc_trampoline_mi() and make
sure it is now always called.
- cpu_hatch: when booting APs the code needs to jump to the first proc
running on that CPU. This should be the idle thread for that CPU.
- sched_exit: when a proc exits it needs to switch away from itself and
then instruct the reaper to clean up the rest. This is done by switching
to the idle loop.

Since the last two cases require a context switch to the idle proc factor
out the common code to sched_toidle() and use it in those places.

Tested by many on all archs.
OK miod@ mpi@ cheloha@

show more ...

Revert commitid: yfAefyNWibUyjkU2, ESyyH5EKxtrXGkS6 and itscfpFvJLOj8mHB;

The change to the single thread API results in crashes inside exit1()
as found by Syzkaller. There seems to be a race in the

Revert commitid: yfAefyNWibUyjkU2, ESyyH5EKxtrXGkS6 and itscfpFvJLOj8mHB;

The change to the single thread API results in crashes inside exit1()
as found by Syzkaller. There seems to be a race in the exit codepath.
What exactly fails is not really clear therefor revert for now.

This should fix the following Syzkaller reports:
Reported-by: syzbot+38efb425eada701ca8bb@syzkaller.appspotmail.com
Reported-by: syzbot+ecc0e8628b3db39b5b17@syzkaller.appspotmail.com
and maybe more.

Reverted commits:
----------------------------
Protect ps_single, ps_singlecnt and ps_threadcnt by the process mutex.

The single thread API needs to lock the process to enter single thread
mode and does not need to stop the scheduler.

This code changes ps_singlecount from a count down to zero to ps_singlecnt
which counts up until equal to ps_threadcnt (in which case all threads
are properly asleep).

Tested by phessler@, OK mpi@ cheloha@
----------------------------
Change how ps_threads and p_thr_link are locked away from using SCHED_LOCK.

The per process thread list can be traversed (read) by holding either
the KERNEL_LOCK or the per process ps_mtx (instead of SCHED_LOCK).
Abusing the SCHED_LOCK for this makes it impossible to split up the
scheduler lock into something more fine grained.

Tested by phessler@, ok mpi@
----------------------------
Fix SCHED_LOCK() leak in single_thread_set()

In the (q->p_flag & P_WEXIT) branch is a continue that did not release
the SCHED_LOCK. Refactor the code a bit to simplify the places SCHED_LOCK
is grabbed and released.

Reported-by: syzbot+ea26d351acfad3bb3f15@syzkaller.appspotmail.com
OK kettenis@

show more ...

Change how ps_threads and p_thr_link are locked away from using SCHED_LOCK.

The per process thread list can be traversed (read) by holding either
the KERNEL_LOCK or the per process ps_mtx (instead o

Change how ps_threads and p_thr_link are locked away from using SCHED_LOCK.

The per process thread list can be traversed (read) by holding either
the KERNEL_LOCK or the per process ps_mtx (instead of SCHED_LOCK).
Abusing the SCHED_LOCK for this makes it impossible to split up the
scheduler lock into something more fine grained.

Tested by phessler@, ok mpi@

show more ...

Protect ps_single, ps_singlecnt and ps_threadcnt by the process mutex.

The single thread API needs to lock the process to enter single thread
mode and does not need to stop the scheduler.

This code

Protect ps_single, ps_singlecnt and ps_threadcnt by the process mutex.

The single thread API needs to lock the process to enter single thread
mode and does not need to stop the scheduler.

This code changes ps_singlecount from a count down to zero to ps_singlecnt
which counts up until equal to ps_threadcnt (in which case all threads
are properly asleep).

Tested by phessler@, OK mpi@ cheloha@

show more ...

Extend scheduler tracepoints to follow CPU jumping.

- Add two new tracpoints sched:fork & sched:steal
- Include selected CPU number in sched:wakeup
- Add sched:unsleep corresponding to sched:sleep w

Extend scheduler tracepoints to follow CPU jumping.

- Add two new tracpoints sched:fork & sched:steal
- Include selected CPU number in sched:wakeup
- Add sched:unsleep corresponding to sched:sleep which matches add/removal
of threads on the sleep queue

ok claudio@

show more ...

PS_CHROOT flag must also be copied when forking. This will block
a ypconnect(2) from reading inside an incorrectly configured space.
ok kettenis

Rename ps_refcnt to ps_threadcnt in struct process and implement
P_HASSIBLING() using this count.
OK mvs@ mpi@

Adjust knote(9) API

Make knote(9) lock the knote list internally, and add knote_locked(9)
for the typical situation where the list is already locked.

Remove the KNOTE(9) macro to simplify the API.

Adjust knote(9) API

Make knote(9) lock the knote list internally, and add knote_locked(9)
for the typical situation where the list is already locked.

Remove the KNOTE(9) macro to simplify the API.

Manual page OK jmc@
OK mpi@ mvs@

show more ...

Add {get,set}thrname(2) for putting thread names in the kernel and
exposed in a new field returned by sysctl(KERN_PROC). Update
pthread_{get,set}_name_np(3) to use the syscalls. Show them, when
set

Add {get,set}thrname(2) for putting thread names in the kernel and
exposed in a new field returned by sysctl(KERN_PROC). Update
pthread_{get,set}_name_np(3) to use the syscalls. Show them, when
set, in ps -H and top -H output.

libc and libpthread minor bumps

ok mpi@, mvs@, deraadt@

show more ...

timeout(9): remove timeout_set_kclock(), TIMEOUT_INITIALIZER_KCLOCK()

We have too many timeout(9) initialization functions and macros.
Let's slim it down and combine some interfaces.

- Remove timeo

timeout(9): remove timeout_set_kclock(), TIMEOUT_INITIALIZER_KCLOCK()

We have too many timeout(9) initialization functions and macros.
Let's slim it down and combine some interfaces.

- Remove timeout_set_kclock(), TIMEOUT_INITIALIZER_KCLOCK().
- Expand timeout_set_flags(), TIMEOUT_INITIALIZER_FLAGS() to accept
an additional "kclock" parameter.
- Reimplement timeout_set(), timeout_set_proc() with timeout_set_flags().
- Reimplement TIMEOUT_INITIALIZER() with TIMEOUT_INITIALIZER_FLAGS().
- Update the sole timeout_set_flags() user to pass a kclock parameter.
- Update the sole timeout_set_kclock() user to call timeout_set_flags().
- Update the sole TIMEOUT_INITIALIZER_FLAGS() user to provide a kclock
parameter.

The timeout(9) code is now a bit out of sync with the manpage. This
will be corrected in a subsequent commit.

ok kn@

show more ...