19ccc7061SDag-Erling Smørgrav.\"- 2*e738085bSDag-Erling Smørgrav.\" Copyright (c) 2002 Dag-Erling Smørgrav 39ccc7061SDag-Erling Smørgrav.\" All rights reserved. 49ccc7061SDag-Erling Smørgrav.\" 59ccc7061SDag-Erling Smørgrav.\" Redistribution and use in source and binary forms, with or without 69ccc7061SDag-Erling Smørgrav.\" modification, are permitted provided that the following conditions 79ccc7061SDag-Erling Smørgrav.\" are met: 89ccc7061SDag-Erling Smørgrav.\" 1. Redistributions of source code must retain the above copyright 99ccc7061SDag-Erling Smørgrav.\" notice, this list of conditions and the following disclaimer. 109ccc7061SDag-Erling Smørgrav.\" 2. Redistributions in binary form must reproduce the above copyright 119ccc7061SDag-Erling Smørgrav.\" notice, this list of conditions and the following disclaimer in the 129ccc7061SDag-Erling Smørgrav.\" documentation and/or other materials provided with the distribution. 139ccc7061SDag-Erling Smørgrav.\" 3. The name of the author may not be used to endorse or promote products 149ccc7061SDag-Erling Smørgrav.\" derived from this software without specific prior written permission. 159ccc7061SDag-Erling Smørgrav.\" 169ccc7061SDag-Erling Smørgrav.\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 179ccc7061SDag-Erling Smørgrav.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 189ccc7061SDag-Erling Smørgrav.\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 199ccc7061SDag-Erling Smørgrav.\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 209ccc7061SDag-Erling Smørgrav.\" FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 219ccc7061SDag-Erling Smørgrav.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 229ccc7061SDag-Erling Smørgrav.\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 239ccc7061SDag-Erling Smørgrav.\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 249ccc7061SDag-Erling Smørgrav.\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 259ccc7061SDag-Erling Smørgrav.\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 269ccc7061SDag-Erling Smørgrav.\" SUCH DAMAGE. 279ccc7061SDag-Erling Smørgrav.\" 28022ca2fcSAlan Somers.Dd January 2, 2021 299ccc7061SDag-Erling Smørgrav.Dt AIO 4 309ccc7061SDag-Erling Smørgrav.Os 319ccc7061SDag-Erling Smørgrav.Sh NAME 329ccc7061SDag-Erling Smørgrav.Nm aio 339ccc7061SDag-Erling Smørgrav.Nd asynchronous I/O 349ccc7061SDag-Erling Smørgrav.Sh DESCRIPTION 359ccc7061SDag-Erling SmørgravThe 369ccc7061SDag-Erling Smørgrav.Nm 379ccc7061SDag-Erling Smørgravfacility provides system calls for asynchronous I/O. 389c20dc99SJohn BaldwinAsynchronous I/O operations are not completed synchronously by the 399c20dc99SJohn Baldwincalling thread. 409c20dc99SJohn BaldwinInstead, the calling thread invokes one system call to request an 419c20dc99SJohn Baldwinasynchronous I/O operation. 429c20dc99SJohn BaldwinThe status of a completed request is retrieved later via a separate 439c20dc99SJohn Baldwinsystem call. 449c20dc99SJohn Baldwin.Pp 459c20dc99SJohn BaldwinAsynchronous I/O operations on some file descriptor types may block an 469c20dc99SJohn BaldwinAIO daemon indefinitely resulting in process and/or system hangs. 479c20dc99SJohn BaldwinOperations on these file descriptor types are considered 489c20dc99SJohn Baldwin.Dq unsafe 499c20dc99SJohn Baldwinand disabled by default. 509c20dc99SJohn BaldwinThey can be enabled by setting 51f3215338SJohn Baldwinthe 52f3215338SJohn Baldwin.Va vfs.aio.enable_unsafe 53f3215338SJohn Baldwinsysctl node to a non-zero value. 54f3215338SJohn Baldwin.Pp 550d378d6bSJohn BaldwinAsynchronous I/O operations on sockets, 560d378d6bSJohn Baldwinraw disk devices, 570d378d6bSJohn Baldwinand regular files on local filesystems do not block 589c20dc99SJohn Baldwinindefinitely and are always enabled. 59f3215338SJohn Baldwin.Pp 60f3215338SJohn BaldwinThe 61f3215338SJohn Baldwin.Nm 62f3215338SJohn Baldwinfacility uses kernel processes 63f3215338SJohn Baldwin(also known as AIO daemons) 64f3215338SJohn Baldwinto service most asynchronous I/O requests. 65f3215338SJohn BaldwinThese processes are grouped into pools containing a variable number of 66f3215338SJohn Baldwinprocesses. 67f3215338SJohn BaldwinEach pool will add or remove processes to the pool based on load. 68f3215338SJohn BaldwinPools can be configured by sysctl nodes that define the minimum 69f3215338SJohn Baldwinand maximum number of processes as well as the amount of time an idle 70f3215338SJohn Baldwinprocess will wait before exiting. 71f3215338SJohn Baldwin.Pp 72f3215338SJohn BaldwinOne pool of AIO daemons is used to service asynchronous I/O requests for 73f3215338SJohn Baldwinsockets. 74f3215338SJohn BaldwinThese processes are named 75f3215338SJohn Baldwin.Dq soaiod<N> . 76f3215338SJohn BaldwinThe following sysctl nodes are used with this pool: 77f3215338SJohn Baldwin.Bl -tag -width indent 78f3215338SJohn Baldwin.It Va kern.ipc.aio.num_procs 79f3215338SJohn BaldwinThe current number of processes in the pool. 80f3215338SJohn Baldwin.It Va kern.ipc.aio.target_procs 81f3215338SJohn BaldwinThe minimum number of processes that should be present in the pool. 82f3215338SJohn Baldwin.It Va kern.ipc.aio.max_procs 83f3215338SJohn BaldwinThe maximum number of processes permitted in the pool. 84f3215338SJohn Baldwin.It Va kern.ipc.aio.lifetime 85f3215338SJohn BaldwinThe amount of time a process is permitted to idle in clock ticks. 86f3215338SJohn BaldwinIf a process is idle for this amount of time and there are more processes 87f3215338SJohn Baldwinin the pool than the target minimum, 88f3215338SJohn Baldwinthe process will exit. 89f3215338SJohn Baldwin.El 90f3215338SJohn Baldwin.Pp 91f3215338SJohn BaldwinA second pool of AIO daemons is used to service all other asynchronous I/O 92f3215338SJohn Baldwinrequests except for I/O requests to raw disks. 93f3215338SJohn BaldwinThese processes are named 94f3215338SJohn Baldwin.Dq aiod<N> . 95f3215338SJohn BaldwinThe following sysctl nodes are used with this pool: 96f3215338SJohn Baldwin.Bl -tag -width indent 97f3215338SJohn Baldwin.It Va vfs.aio.num_aio_procs 98f3215338SJohn BaldwinThe current number of processes in the pool. 99f3215338SJohn Baldwin.It Va vfs.aio.target_aio_procs 100f3215338SJohn BaldwinThe minimum number of processes that should be present in the pool. 101f3215338SJohn Baldwin.It Va vfs.aio.max_aio_procs 102f3215338SJohn BaldwinThe maximum number of processes permitted in the pool. 103f3215338SJohn Baldwin.It Va vfs.aio.aiod_lifetime 104f3215338SJohn BaldwinThe amount of time a process is permitted to idle in clock ticks. 105f3215338SJohn BaldwinIf a process is idle for this amount of time and there are more processes 106f3215338SJohn Baldwinin the pool than the target minimum, 107f3215338SJohn Baldwinthe process will exit. 108f3215338SJohn Baldwin.El 109f3215338SJohn Baldwin.Pp 110f3215338SJohn BaldwinAsynchronous I/O requests for raw disks are queued directly to the disk 111f3215338SJohn Baldwindevice layer after temporarily wiring the user pages associated with the 112f3215338SJohn Baldwinrequest. 113f3215338SJohn BaldwinThese requests are not serviced by any of the AIO daemon pools. 114f3215338SJohn Baldwin.Pp 115f3215338SJohn BaldwinSeveral limits on the number of asynchronous I/O requests are imposed both 116f3215338SJohn Baldwinsystem-wide and per-process. 117f3215338SJohn BaldwinThese limits are configured via the following sysctls: 118f3215338SJohn Baldwin.Bl -tag -width indent 119f3215338SJohn Baldwin.It Va vfs.aio.max_buf_aio 120f3215338SJohn BaldwinThe maximum number of queued asynchronous I/O requests for raw disks permitted 121f3215338SJohn Baldwinfor a single process. 122f3215338SJohn BaldwinAsynchronous I/O requests that have completed but whose status has not been 123f3215338SJohn Baldwinretrieved via 124f3215338SJohn Baldwin.Xr aio_return 2 125f3215338SJohn Baldwinor 126f3215338SJohn Baldwin.Xr aio_waitcomplete 2 127f3215338SJohn Baldwinare not counted against this limit. 128f3215338SJohn Baldwin.It Va vfs.aio.num_buf_aio 129f3215338SJohn BaldwinThe number of queued asynchronous I/O requests for raw disks system-wide. 130f3215338SJohn Baldwin.It Va vfs.aio.max_aio_queue_per_proc 131f3215338SJohn BaldwinThe maximum number of asynchronous I/O requests for a single process 132f3215338SJohn Baldwinserviced concurrently by the default AIO daemon pool. 133f3215338SJohn Baldwin.It Va vfs.aio.max_aio_per_proc 134f3215338SJohn BaldwinThe maximum number of outstanding asynchronous I/O requests permitted for a 135f3215338SJohn Baldwinsingle process. 136f3215338SJohn BaldwinThis includes requests that have not been serviced, 137f3215338SJohn Baldwinrequests currently being serviced, 138f3215338SJohn Baldwinand requests that have completed but whose status has not been retrieved via 139f3215338SJohn Baldwin.Xr aio_return 2 140f3215338SJohn Baldwinor 141f3215338SJohn Baldwin.Xr aio_waitcomplete 2 . 142f3215338SJohn Baldwin.It Va vfs.aio.num_queue_count 143f3215338SJohn BaldwinThe number of outstanding asynchronous I/O requests system-wide. 144f3215338SJohn Baldwin.It Va vfs.aio.max_aio_queue 145f3215338SJohn BaldwinThe maximum number of outstanding asynchronous I/O requests permitted 146f3215338SJohn Baldwinsystem-wide. 147f3215338SJohn Baldwin.El 148463970daSJohn Baldwin.Pp 149463970daSJohn BaldwinAsynchronous I/O control buffers should be zeroed before initializing 150463970daSJohn Baldwinindividual fields. 151463970daSJohn BaldwinThis ensures all fields are initialized. 152463970daSJohn Baldwin.Pp 153463970daSJohn BaldwinAll asynchronous I/O control buffers contain a 154463970daSJohn Baldwin.Vt sigevent 155463970daSJohn Baldwinstructure in the 156463970daSJohn Baldwin.Va aio_sigevent 157463970daSJohn Baldwinfield which can be used to request notification when an operation completes. 158463970daSJohn Baldwin.Pp 159463970daSJohn BaldwinFor 160463970daSJohn Baldwin.Dv SIGEV_KEVENT 161463970daSJohn Baldwinnotifications, 16209986d3bSAlan Somersthe 16309986d3bSAlan Somers.Va sigevent 16409986d3bSAlan Somers.Ap 16509986d3bSAlan Somerss 16609986d3bSAlan Somers.Va sigev_notify_kqueue 16709986d3bSAlan Somersfield should contain the descriptor of the kqueue that the event should be attached 16809986d3bSAlan Somersto, its 16909986d3bSAlan Somers.Va sigev_notify_kevent_flags 17009986d3bSAlan Somersfield may contain 17109986d3bSAlan Somers.Dv EV_ONESHOT , 17209986d3bSAlan Somers.Dv EV_CLEAR , and/or 17309986d3bSAlan Somers.Dv EV_DISPATCH , and its 17409986d3bSAlan Somers.Va sigev_notify 17509986d3bSAlan Somersfield should be set to 17609986d3bSAlan Somers.Dv SIGEV_KEVENT . 17709986d3bSAlan SomersThe posted kevent will contain: 178463970daSJohn Baldwin.Bl -column ".Va filter" 179463970daSJohn Baldwin.It Sy Member Ta Sy Value 180463970daSJohn Baldwin.It Va ident Ta asynchronous I/O control buffer pointer 181463970daSJohn Baldwin.It Va filter Ta Dv EVFILT_AIO 18209986d3bSAlan Somers.It Va flags Ta Dv EV_EOF 183463970daSJohn Baldwin.It Va udata Ta 184463970daSJohn Baldwinvalue stored in 185463970daSJohn Baldwin.Va aio_sigevent.sigev_value 186463970daSJohn Baldwin.El 187463970daSJohn Baldwin.Pp 188463970daSJohn BaldwinFor 189463970daSJohn Baldwin.Dv SIGEV_SIGNO 190463970daSJohn Baldwinand 191463970daSJohn Baldwin.Dv SIGEV_THREAD_ID 192463970daSJohn Baldwinnotifications, 193463970daSJohn Baldwinthe information for the queued signal will include 194463970daSJohn Baldwin.Dv SI_ASYNCIO 195463970daSJohn Baldwinin the 196463970daSJohn Baldwin.Va si_code 197463970daSJohn Baldwinfield and the value stored in 198463970daSJohn Baldwin.Va sigevent.sigev_value 199463970daSJohn Baldwinin the 200463970daSJohn Baldwin.Va si_value 201463970daSJohn Baldwinfield. 202463970daSJohn Baldwin.Pp 203463970daSJohn BaldwinFor 204463970daSJohn Baldwin.Dv SIGEV_THREAD 205463970daSJohn Baldwinnotifications, 206463970daSJohn Baldwinthe value stored in 207463970daSJohn Baldwin.Va aio_sigevent.sigev_value 208463970daSJohn Baldwinis passed to the 209463970daSJohn Baldwin.Va aio_sigevent.sigev_notify_function 210463970daSJohn Baldwinas described in 211463970daSJohn Baldwin.Xr sigevent 3 . 2129ccc7061SDag-Erling Smørgrav.Sh SEE ALSO 2139ccc7061SDag-Erling Smørgrav.Xr aio_cancel 2 , 2149ccc7061SDag-Erling Smørgrav.Xr aio_error 2 , 2159ccc7061SDag-Erling Smørgrav.Xr aio_read 2 , 216022ca2fcSAlan Somers.Xr aio_readv 2 , 2179ccc7061SDag-Erling Smørgrav.Xr aio_return 2 , 2189ccc7061SDag-Erling Smørgrav.Xr aio_suspend 2 , 2199ccc7061SDag-Erling Smørgrav.Xr aio_waitcomplete 2 , 220a7af55afSSheldon Hearn.Xr aio_write 2 , 221022ca2fcSAlan Somers.Xr aio_writev 2 , 2221493ebe7STim J. Robbins.Xr lio_listio 2 , 223463970daSJohn Baldwin.Xr sigevent 3 , 224f3215338SJohn Baldwin.Xr sysctl 8 2259ccc7061SDag-Erling Smørgrav.Sh HISTORY 2269ccc7061SDag-Erling SmørgravThe 2279ccc7061SDag-Erling Smørgrav.Nm 2289ccc7061SDag-Erling Smørgravfacility appeared as a kernel option in 2299ccc7061SDag-Erling Smørgrav.Fx 3.0 . 2309ccc7061SDag-Erling SmørgravThe 2319ccc7061SDag-Erling Smørgrav.Nm 2329ccc7061SDag-Erling Smørgravkernel module appeared in 2339ccc7061SDag-Erling Smørgrav.Fx 5.0 . 234f3215338SJohn BaldwinThe 235f3215338SJohn Baldwin.Nm 236f3215338SJohn Baldwinfacility was integrated into all kernels in 237f3215338SJohn Baldwin.Fx 11.0 . 238