man/man9/xcall.9

.\"     $NetBSD: xcall.9,v 1.17 2020/02/01 13:35:11 riastradh Exp $
.\"
.\" Copyright (c) 2010 The NetBSD Foundation, Inc.
.\" All rights reserved.
.\"
.\" This code is derived from software contributed to The NetBSD Foundation
.\" by Andrew Doran.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\" 1. Redistributions of source code must retain the above copyright
.\"    notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\"    notice, this list of conditions and the following disclaimer in the
.\"    documentation and/or other materials provided with the distribution.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
.\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
.\" TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
.\" PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
.\" BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
.\" POSSIBILITY OF SUCH DAMAGE.
.\"
.Dd February 1, 2020
.Dt XCALL 9
.Os
.Sh NAME
.Nm xcall ,
.Nm xc_broadcast ,
.Nm xc_unicast ,
.Nm xc_wait ,
.Nm xc_barrier
.Nd cross-call interface
.Sh SYNOPSIS
.In sys/xcall.h
.Vt typedef void (*xcfunc_t)(void *, void *);
.Ft uint64_t
.Fn xc_broadcast "u_int flags" "xcfunc_t func" "void *arg1" "void *arg2"
.Ft uint64_t
.Fn xc_unicast "u_int flags" "xcfunc_t func" "void *arg1" "void *arg2" "struct cpu_info *ci"
.Ft void
.Fn xc_wait "uint64_t where"
.Ft void
.Fn xc_barrier "u_int flags"
.Sh DESCRIPTION
The machine-independent
.Nm
interface allows any CPU in the system to request that an arbitrary
function be executed on any other CPU.
.Pp
Sometimes it is necessary to modify hardware state that is tied
directly to individual CPUs
.Po
such as a CPU's local timer
.Pc ,
and these updates can not be done remotely by another CPU.
The LWP requesting the update may be unable to guarantee that it
will be running on the CPU where the update must occur, when the
update occurs.
.Pp
Additionally, it is sometimes necessary to modify per-CPU software
state from a remote CPU.
Where these update operations are so rare or the access to the
per-CPU data so frequent that the cost of using locking or atomic
operations to provide coherency is prohibitive, another way must
be found.
.Pp
Cross calls help to solve these types of problem.
However, since this facility is heavyweight, it is expected that
it will not be used often.
.Pp
.Nm
provides a mechanism for making
.Dq "low priority"
cross calls.
The function to be executed runs on the remote CPU within a thread
context, and not from a software interrupt, so it can ensure that it is
not interrupting other code running on the CPU, and so has exclusive
access to the CPU.
Keep in mind that unless disabled, it may cause a kernel preemption.
.Pp
.Nm
also provides a mechanism for making
.Dq "high priority"
cross calls.
The function to be executed runs on the remote CPU within a
software interrupt context, possibly interrupting other lower-priority
code running on the CPU.
.Sh NOTES
Functions being called should be relatively lightweight.
They may block on locks, but carefully and minimally, to not interfere
with other cross calls in the system.
.Sh FUNCTIONS
.Bl -tag -width abcd
.It Fn xc_broadcast "flags" "func" "arg1" "arg2"
Call
.Pf (* Fa func\| ) Ns Fo ""
.Fa "arg1"
.Fa "arg2"
.Fc
on all CPUs in the system.
Return a
.Vt uint64_t
.Dq ticket
to
.Fn xc_wait
on for the cross-call to complete.
.Fa flags
should be
.Dv XC_HIGHPRI
or
.Dv XC_HIGHPRI_IPL\| Ns Fn "" ipl
for a "high priority" call, and 0 for a "low priority" call.
.Dv XC_HIGHPRI
uses an
.Dv IPL_SOFTSERIAL
software interrupt while
.Dv XC_HIGHPRI_IPL
uses a software interrupt with an IPL specified by
.Fa ipl .
.Fn xc_broadcast
should not be called from interrupt context.
.It Fn xc_unicast "flags" "func" "arg1" "arg2" "ci"
Like
.Fn xc_broadcast ,
but call
.Fa func
on only the CPU indicated by
.Fa ci .
.Fn xc_unicast
also returns a
.Dq ticket .
.It Fn xc_wait "where"
Wait on the
.Dq ticket
returned by a prior
.Fn xc_broadcast
or
.Fn xc_unicast
for the corresponding cross-call to complete.
.Fn xc_wait
should be called from a thread context.
.It Fn xc_barrier "flags"
Issue a broadcast cross-call that does nothing,
and wait for it to complete.
.Pp
This functions like a memory barrier that forces all prior operations
in program order to globally happen before all subsequent operations in
program order, as witnessed by every CPU.
.Pp
This additionally waits for all higher-priority activity on the CPU to
complete, according to
.Fa flags :
.Bl -dash -compact
.It
.Fo xc_barrier
.Li 0
.Fc
waits for any pending
.Xr kpreempt_disable 9
sections or activity at interrupt priority level above
.Dv IPL_NONE
to finish on all CPUs.
.It
.Fo xc_barrier
.Dv XC_HIGHPRI_IPL\| Ns Fn "" ipl
.Fc
waits for any pending activity at the software interrupt priority level
.Fa ipl
or higher to finish on all CPUs.
.El
.Pp
.Fn xc_barrier
is much more expensive than
.Xr membar_ops 3 ,
so it should be used sparingly, only to publish information
infrequently \(em for example, during module load and unload \(em when
the cost of a memory barrier on the consumer side would be
prohibitive.
.El
.Sh CODE REFERENCES
The
.Nm
interface is implemented within the file
.Pa sys/kern/subr_xcall.c .
.\" .Sh EXAMPLES
.Sh SEE ALSO
.Xr membar_ops 3 ,
.Xr kpreempt 9 ,
.Xr percpu 9 ,
.Xr softint 9
.Sh HISTORY
The
.Nm
interface first appeared in
.Nx 5.0 .
.Sh AUTHORS
.An Andrew Doran Aq Mt ad@NetBSD.org