1.\" $NetBSD: ipi.9,v 1.6 2022/02/12 01:21:11 riastradh Exp $ 2.\" 3.\" Copyright (c) 2014, 2019 The NetBSD Foundation, Inc. 4.\" All rights reserved. 5.\" 6.\" This code is derived from software contributed to The NetBSD Foundation 7.\" by Mindaugas Rasiukevicius. 8.\" 9.\" Redistribution and use in source and binary forms, with or without 10.\" modification, are permitted provided that the following conditions 11.\" are met: 12.\" 1. Redistributions of source code must retain the above copyright 13.\" notice, this list of conditions and the following disclaimer. 14.\" 2. Redistributions in binary form must reproduce the above copyright 15.\" notice, this list of conditions and the following disclaimer in the 16.\" documentation and/or other materials provided with the distribution. 17.\" 18.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS 19.\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED 20.\" TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 21.\" PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS 22.\" BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 23.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF 24.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 25.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 26.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 27.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 28.\" POSSIBILITY OF SUCH DAMAGE. 29.\" 30.Dd March 31, 2019 31.Dt IPI 9 32.Os 33.Sh NAME 34.Nm ipi 35.Nd machine-independent interprocessor interrupts 36.Sh SYNOPSIS 37.In sys/ipi.h 38.Vt typedef void (*ipi_func_t)(void *); 39.\" ----- 40.Ft u_int 41.Fn ipi_register "ipi_func_t func" "void *arg" 42.Ft void 43.Fn ipi_unregister "u_int ipi_id" 44.Ft void 45.Fn ipi_trigger "u_int ipi_id" "struct cpu_info *ci" 46.Ft void 47.Fn ipi_trigger_multi "u_int ipi_id" "const kcpuset_t *target" 48.Ft void 49.Fn ipi_trigger_broadcast "u_int ipi_id" "bool skip_self" 50.\" ----- 51.Ft void 52.Fn ipi_unicast "ipi_msg_t *msg" "struct cpu_info *ci" 53.Ft void 54.Fn ipi_multicast "ipi_msg_t *msg" "const kcpuset_t *target" 55.Ft void 56.Fn ipi_broadcast "ipi_msg_t *msg" "bool skip_self" 57.Ft void 58.Fn ipi_wait "ipi_msg_t *msg" 59.\" ----- 60.Sh DESCRIPTION 61The machine-independent 62.Nm 63interface provides capability to send inter-processor interrupts (IPIs) 64amongst CPUs. 65The interface has two mechanisms: asynchronous IPI to invoke functions 66with a constant argument and synchronous IPIs with the cross-call support. 67.Pp 68Other synchronization interfaces are built using the MI IPI interface. 69For a general purpose inter-processor cross-calls or remote 70interrupts, use the 71.Xr xcall 9 72or 73.Xr softint 9 74interfaces. 75.Pp 76The primary use cases of the MI IPIs include the following: 77.Bl -hyphen -compact 78.It 79provide a facility for the 80.Xr softint 9 81subsystem to schedule software interrupts on remote CPUs 82.It 83provide a facility for the 84.Xr xcall 9 85subsystem 86.It 87abstract IPI handling and facilitate machine-dependent code 88.El 89.\" ----- 90.Ss Asynchronous IPI interface 91This interface allows dynamic registration of IPI handlers with a constant 92argument and asynchronous triggering of interrupts. 93.Bl -tag -width compact 94.It Fn ipi_register "func" "arg" 95Register an IPI handler 96.Fa func 97with an arbitrary argument 98.Fa arg . 99Returns a non-zero IPI identifier on success and zero on failure. 100.It Fn ipi_unregister "ipi_id" 101Unregister the IPI handler identified by the 102.Fa ipi_id . 103.It Fn ipi_trigger "ipi_id" "ci" 104Trigger an IPI identified by 105.Fa ipi_id 106on a remote CPU specified by 107.Fa ci . 108This function must be called with kernel preemption disabled and 109the target CPU must be remote. 110.It Fn ipi_trigger_multi "ipi_id" "target" 111Trigger an IPI identified by 112.Fa ipi_id 113on all of the CPUs in the set specified by 114.Fa target . 115This function must be called with kernel preemption disabled. 116The sending CPU may be included in the CPU set; when this is the case, 117the IPI on the sending CPU is processes synchronously. 118.It Fn ipi_trigger_broadcast "ipi_id" "skip_self" 119Trigger an IPI identified by 120.Fa ipi_id 121on all of the attached CPUs. 122This function must be called with kernel preemption disabled. 123Optionally, the sending CPU may be skipped by passing 124.Dv true 125for 126.Fa skip_self . 127.El 128.\" ----- 129.Ss Synchronous IPI interface 130This interface provides capability to perform cross-calls, i.e. invoke 131an arbitrary function on a remote CPU. 132The invocations are performed synchronously and the caller must wait 133for completion. 134The cross-call is described by an IPI "message". 135The caller has to fill in an 136.Vt ipi_msg_t 137structure which has the following public members: 138.Bd -literal 139 ipi_func_t func; 140 void arg; 141.Ed 142.Pp 143The 144.Ar func 145member specifies a function to invoke and 146.Ar arg 147is the argument to be passed to the function. 148.Bl -tag -width compact 149.It Fn ipi_unicast "msg" "ci" 150Send an IPI to a remote CPU specified by 151.Fa ci . 152.It Fn ipi_multicast "msg" "target" 153Send IPIs to a CPU set specified by 154.Fa target . 155.It Fn ipi_broadcast "msg" "skip_self" 156Send IPIs to all CPUs. 157Optionally, the sending CPU may be skipped by passing 158.Dv true 159for 160.Fa skip_self . 161.It Fn ipi_wait "msg" 162Wait until all IPIs complete. 163.El 164.Pp 165All described functions, except 166.Fn ipi_wait , 167must be called with the kernel preemption disabled. 168All synchronous IPI invocations must be completed (wait for them with the 169.Fn ipi_wait 170function) before the IPI message structure can be destroyed or new 171cross-call requests can be performed. 172.\" ----- 173.Sh MEMORY ORDER 174All memory operations that happen before triggering an IPI, via 175.Fn ipi_trigger , 176.Fn ipi_trigger_multi , 177.Fn ipi_trigger_broadcast , 178.Fn ipi_unicast , 179.Fn ipi_multicast , 180or 181.Fn ipi_broadcast , 182also happen before any memory operations in the IPI handler function on 183the remote CPU. 184.Pp 185For synchronous IPIs, all memory operations that happen before the IPI 186handler function has returned on the remote CPU also happen before 187.Fn ipi_wait 188returns on the waiting CPU. 189.\" ----- 190.Sh NOTES 191Functions being called must be lightweight. 192They run at 193.Dv IPL_HIGH 194and should generally not use any other synchronization interfaces 195such as 196.Xr mutex 9 . 197If spin-locks are used, they must be used carefully and have no contention. 198.\" ----- 199.Sh CODE REFERENCES 200The 201.Nm 202interface is implemented within the file 203.Pa sys/kern/subr_ipi.c . 204.\" ----- 205.Sh SEE ALSO 206.Xr kcpuset 9 , 207.Xr kpreempt 9 , 208.Xr softint 9 , 209.Xr spl 9 , 210.Xr xcall 9 211.Sh HISTORY 212The 213.Nm 214interface first appeared in 215.Nx 7.0 . 216.Sh AUTHORS 217.An Mindaugas Rasiukevicius Aq Mt rmind@NetBSD.org 218