Create a new header lwp_private.h to contain _lwp_getprivate_fast,
_lwp_gettcb_fast, _lwp_settcb and remove them from mcontext.h, so that:
1. we don't need special hacks to hide them
2. we can includ

Create a new header lwp_private.h to contain _lwp_getprivate_fast,
_lwp_gettcb_fast, _lwp_settcb and remove them from mcontext.h, so that:
1. we don't need special hacks to hide them
2. we can include <lwp.h> where needed to get the necessary prototypes
without redefining them locally.

show more ...

libpthread: New pthread__smt_wait to put CPU in low power for spin.

This is now distinct from pthread__smt_pause, which is for spin lock
backoff with no paired wakeup.

On Arm, there is a single-bit

libpthread: New pthread__smt_wait to put CPU in low power for spin.

This is now distinct from pthread__smt_pause, which is for spin lock
backoff with no paired wakeup.

On Arm, there is a single-bit event register per CPU, and there are two
instructions to manage it:

- wfe, wait for event -- if event register is clear, enter low power
mode and wait until event register is set; then exit low power mode
and clear event register

- sev, signal event -- sets event register on all CPUs (other
circumstances like interrupts also set the event register and cause
wfe to wake)

These can be used to reduce the power consumption of spinning for a
lock, but only if they are actually paired -- if there's no sev, wfe
might hang indefinitely. Currently only pthread_spin(3) actually
pairs them; the other lock primitives (internal lock, mutex, rwlock)
do not -- they have spin lock backoff loops, but no corresponding
wakeup to cancel a wfe.

It may be worthwhile to teach the other lock primitives to pair
wfe/sev, but that requires some performance measurement to verify
it's actually worthwhile. So for now, we just make sure not to use
wfe when there's no sev, and keep everything else the same -- this
should fix severe performance degredation in libpthread on Arm
without hurting anything else.

No change in the generated code on amd64 and i386. No change in the
generated code for pthread_spin.c on arm and aarch64 -- changes only
the generated code for pthread_lock.c, pthread_mutex.c, and
pthread_rwlock.c, as intended.

PR port-arm/57437

XXX pullup-10

show more ...

libpthread: Use __nothing, not /* nothing */, for empty macros.

No functional change intended -- just safer to do it this way in case
the macros are used in if branches or comma expressions.

PR por

libpthread: Use __nothing, not /* nothing */, for empty macros.

No functional change intended -- just safer to do it this way in case
the macros are used in if branches or comma expressions.

PR port-arm/57437 (pthread__smt_pause/wake issue)

XXX pullup-10

show more ...

lib: remove CONSTCOND comment

Since 2021-01-31, lint doesn't need it anymore for the common pattern of
'do ... while (0)'.

libpthread: Move namespacing include to top of .c files.

Stuff like libc's namespace.h, or atomic_op_namespace.h, which does
namespacing tricks like `#define atomic_cas_uint _atomic_cas_uint',
has t

libpthread: Move namespacing include to top of .c files.

Stuff like libc's namespace.h, or atomic_op_namespace.h, which does
namespacing tricks like `#define atomic_cas_uint _atomic_cas_uint',
has to go at the top of each .c file. If it goes in the middle, it
might be too late to affect the declarations, and result in compile
errors.

I tripped over this by including <sys/atomic.h> in mips
<machine/lock.h>.

(Maybe we should create a new pthread_namespace.h file for the
purpose, but this'll do for now.)

show more ...

- Make pthread_condvar and pthread_mutex work on the stack rather than in
pthread_t, so there's less chance of bad things happening if someone calls
(for example) pthread_cond_broadcast() from a

- Make pthread_condvar and pthread_mutex work on the stack rather than in
pthread_t, so there's less chance of bad things happening if someone calls
(for example) pthread_cond_broadcast() from a signal handler.

- Remove all the deferred waiter handling except for the one case that really
matters which is transferring waiters from condvar -> mutex on wakeup, and
do that by splicing the condvar's waiters onto the mutex.

- Remove the mutex waiters bit as it's another complication that's not
strictly needed.

show more ...

Pass down errno when calling pthread__errorfunc after a system call.
Allow format arguments for that reason and use (v)snprintf_ss in
pthread_errorfunc to avoid race conditions and the like.

In the interests of reliability simplify waiter handling more and redo
condvars to manage the list of waiters with atomic ops.

- Try to eliminate a hang in "parked" I've been seeing while stress testing.
Centralise wakeup of deferred waiters in pthread__clear_waiters() and use
throughout libpthread. Make fewer assumptio

- Try to eliminate a hang in "parked" I've been seeing while stress testing.
Centralise wakeup of deferred waiters in pthread__clear_waiters() and use
throughout libpthread. Make fewer assumptions. Be more conservative in
pthread_mutex when dealing with pending waiters.

- Remove the "hint" argument everywhere since the kernel doesn't use it any
more.

show more ...

Revert "Enhance the pthread(3) + malloc(3) init model"

It is reported to hand on aarch64 with gzip.

Enhance the pthread(3) + malloc(3) init model

Separate the pthread_atfork(3) call from pthread_tsd_init()
and move it into a distinct function.

Call inside pthread__init() late TSD initialization r

Enhance the pthread(3) + malloc(3) init model

Separate the pthread_atfork(3) call from pthread_tsd_init()
and move it into a distinct function.

Call inside pthread__init() late TSD initialization route, just after
"pthread_atfork(NULL, NULL, pthread__fork_callback);".

Document that malloc(3) initialization is now controlled again and called
during the first pthread_atfork(3) call.

Remove #if 0 code from pthread_mutex.c as we no longer initialize malloc
prematurely.

show more ...

Retire ifdef ERRORCHECK in pthread(3)

It is enabled unconditionally since 2003 and used only for rwlocks and
spinlocks.

LLVM sanitizers make assumptions that these checks are enabled always.

- A bit more alignment in __pthread_st especially for the rbtree node.
- Use COHERENCY_UNIT from sys/param.h.

pthread_detach(), pthread_join(): go back to using _lwp_detach() and
_lwp_wait(), rather than doing it all in userspace. There's less to go
wrong. Doesn't seem to be a performance penalty.

Rip out some very ambitious optimisations around pthread_mutex that are
don't buy much. This stuff is hard enough to get right in the kernel let
alone userspace, and I don't trust that it's right.

Bump PTHREAD__UNPARK_MAX to 128 as bandaid for locking related hangs.

G/c unused rwlock owner macros copy-pasted from the kernel.

They were brought along with the rwlock flags but never used and never
even adapted to the new home (the struct member name is different
h

G/c unused rwlock owner macros copy-pasted from the kernel.

They were brought along with the rwlock flags but never used and never
even adapted to the new home (the struct member name is different
here). I looked at adapting and using them, but they don't really
help readability that much and there are cases where we need to deal
with "fused" owner values anyway and so can't use them.

show more ...

Transfer all the keys that were created in the libc stub implementation
to the pthread tsd implementation when the main thread is created.
This corrects a problem where a process created keys before

Transfer all the keys that were created in the libc stub implementation
to the pthread tsd implementation when the main thread is created.
This corrects a problem where a process created keys before libpthread
was loaded (either from the libc constructor or because libpthread
was dlopened later). This fixes a problem with jemalloc which creates
keys in the constructor.

show more ...

Export the guard size of the main thread via vm.guard_size. Add a
complementary writable sysctl for the initial guard size of threads
created via pthread_create. Let the existing attribut accessors d

Export the guard size of the main thread via vm.guard_size. Add a
complementary writable sysctl for the initial guard size of threads
created via pthread_create. Let the existing attribut accessors do the
right thing. Raise the default guard size for threads to 64KB.

show more ...

libpthread_dbg(3) deletion from the base distribution

libpthread_dbg(3) is a remnant library from the M:N thread model
(pre-NetBSD-5.0) API to introspect threads within a process and for use
of debu

libpthread_dbg(3) deletion from the base distribution

libpthread_dbg(3) is a remnant library from the M:N thread model
(pre-NetBSD-5.0) API to introspect threads within a process and for use
of debuggers.

Currently in the 1:1 model it's not used in GDB neither in LLDB and it's
not either planned to be used. It's current function to read pthread_t
structures is realizable within a regular debugger capable to
instrospect objects within a tracee (GDB, LLDB...).

Remaining users of this API can still use this library from
pkgsrc/devel/libpthread_dbg.

Sponsored by <The NetBSD Foundation>

show more ...

Fix previous: Can't use calloc/malloc before we complete initialization
of the thread library, because malloc uses pthread_foo_specific, and it will
end up initializing itself incorrectly.

Make PTHREAD_KEYS_MAX dynamically adjustable

NetBSD's PTHREAD_KEYS_MAX is set to 256, which is low compared to
other systems like Linux (1024) or MacOS X (512). As a result some
setups tested on Lin

Make PTHREAD_KEYS_MAX dynamically adjustable

NetBSD's PTHREAD_KEYS_MAX is set to 256, which is low compared to
other systems like Linux (1024) or MacOS X (512). As a result some
setups tested on Linux will exhibit problems on NetBSD because of
pthread_keys usage beyond the limit. This happens for instance on
Apache with various module loaded, and in this case no particular
developper can be blamed for going beyond the limit, since several
modules from different sources contribute to the problem.

This patch makes the limit conigurable through the PTHREAD_KEYS_MAX
environement variable. If undefined, the default remains unchanged
(256). In any case, the value cannot be lowered below POSIX-mandated
_POSIX_THREAD_KEYS_MAX (128).

While there:
- use EXIT_FAILURE instead of 1 when calling err(3) in libpthread.
- Reset _POSIX_THREAD_KEYS_MAX to POSIX mandated 128, instead of 256.

show more ...

Allow for arbitrary MI scheduler implementations.

A concrete result is enabling unpatched libpthread to run on the
rumprun stacks (e.g. Xen and bare metal) with a non-NetBSD scheduler.
Those schedul

Allow for arbitrary MI scheduler implementations.

A concrete result is enabling unpatched libpthread to run on the
rumprun stacks (e.g. Xen and bare metal) with a non-NetBSD scheduler.
Those schedulers hook into the existing _lwp_frobnitz() NetBSD syscall
interfaces (well, "syscall" interfaces in that scenario ;)

More specifically about the change itself:

1) instead of calling _lwp_makecontext() followed by _lwp_create()
and passing the entry point in ucontext_t (MD) through the calls, roll
the calls into pthread__makelwp() and allow alternate implementations
for that MI interface.

2) allow compile-time overriding of __lwp_gettcb_fast() or
__lwp_getprivate_fast, which are inline and leak MD scheduler/thread
details into libpthread


Additionally, two small nits:

I) define LIB=pthread before including mk.conf so that it's possible
to test for LIB==pthread in mk.conf

II) make it possible to leave out pthread_cancelstub.c. This is required
by the current implementation of rumprun-posix (i.e. rumprun on
POSIX hosts) due to symbol collisions. It needs to be fixed properly
some day, but for now allows an almost-correct libpthread to run.
I am sure @justin will be happy to explain the details ;)


no change to NetBSD
tested: anita+atf

show more ...

- Allow libpthread to be dlopened again, by providing libc stubs to libpthread.
- Fail if the dlopened libpthread does pthread_create(). From manu@
- Discussed at length in the mailing lists; approve

- Allow libpthread to be dlopened again, by providing libc stubs to libpthread.
- Fail if the dlopened libpthread does pthread_create(). From manu@
- Discussed at length in the mailing lists; approved by core@
- This was chosen as the least intrusive patch that will provide
the necessary functionality.
XXX: pullup to 6

show more ...

Replace the simple implementation of pthread_key_{create,destroy}
and pthread_{g,s}etspecific functions, to one that invalidates
values of keys in other threads when pthread_key_delete() is called.
T

Replace the simple implementation of pthread_key_{create,destroy}
and pthread_{g,s}etspecific functions, to one that invalidates
values of keys in other threads when pthread_key_delete() is called.
This fixes chromium, which expects pthread_key_delete() to do
cleanup in all threads.

show more ...