1.\" $NetBSD: dmover.9,v 1.15 2017/07/03 21:28:48 wiz Exp $ 2.\" 3.\" Copyright (c) 2002 Wasabi Systems, Inc. 4.\" All rights reserved. 5.\" 6.\" Written by Jason R. Thorpe for Wasabi Systems, Inc. 7.\" 8.\" Redistribution and use in source and binary forms, with or without 9.\" modification, are permitted provided that the following conditions 10.\" are met: 11.\" 1. Redistributions of source code must retain the above copyright 12.\" notice, this list of conditions and the following disclaimer. 13.\" 2. Redistributions in binary form must reproduce the above copyright 14.\" notice, this list of conditions and the following disclaimer in the 15.\" documentation and/or other materials provided with the distribution. 16.\" 3. All advertising materials mentioning features or use of this software 17.\" must display the following acknowledgement: 18.\" This product includes software developed for the NetBSD Project by 19.\" Wasabi Systems, Inc. 20.\" 4. The name of Wasabi Systems, Inc. may not be used to endorse 21.\" or promote products derived from this software without specific prior 22.\" written permission. 23.\" 24.\" THIS SOFTWARE IS PROVIDED BY WASABI SYSTEMS, INC. ``AS IS'' AND 25.\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED 26.\" TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 27.\" PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WASABI SYSTEMS, INC 28.\" BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 29.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF 30.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 31.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 32.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 33.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 34.\" POSSIBILITY OF SUCH DAMAGE. 35.\" 36.Dd December 4, 2007 37.Dt DMOVER 9 38.Os 39.Sh NAME 40.Nm dmover_backend_register , 41.Nm dmover_backend_unregister , 42.Nm dmover_session_create , 43.Nm dmover_session_destroy , 44.Nm dmover_request_alloc , 45.Nm dmover_request_free , 46.Nm dmover_process , 47.Nm dmover_done 48.Nd hardware-assisted data mover interface 49.Sh SYNOPSIS 50.In dev/dmover/dmovervar.h 51.Pp 52Client interface routines: 53.Pp 54.Ft int 55.Fn "dmover_session_create" "const char *" "struct dmover_session **" 56.Ft void 57.Fn "dmover_session_destroy" "struct dmover_session *" 58.Ft "struct dmover_request *" 59.Fn "dmover_request_alloc" "struct dmover_session *" "dmover_buffer *" 60.Ft void 61.Fn "dmover_request_free" "struct dmover_request *" 62.Ft void 63.Fn "dmover_process" "struct dmover_request *" 64.Pp 65Back-end interface routines: 66.Pp 67.Ft void 68.Fn "dmover_backend_register" "struct dmover_backend *" 69.Ft void 70.Fn "dmover_backend_unregister" "struct dmover_backend *" 71.Ft void 72.Fn "dmover_done" "struct dmover_request *" 73.Sh DESCRIPTION 74The 75.Nm dmover 76facility provides an interface to hardware-assisted data movers. 77This can be used to copy data from one location in memory to another, clear 78a region of memory, fill a region of memory with a pattern, and perform 79simple operations on multiple regions of memory, such as an XOR, without 80intervention by the CPU. 81.Pp 82The drivers for hardware-assisted data movers present themselves to 83.Nm dmover 84by registering their capabilities. 85When a client wishes to use a 86.Nm dmover 87function, it creates a session for that function, which identifies back-ends 88capable of performing that function. 89The client then enqueues requests on that session, which the back-ends 90process asynchronously. 91The client may choose to block until the request is completed, or may 92have a call-back invoked once the request has been completed. 93.Pp 94When a client creates a session, the 95.Nm dmover 96facility identifies back-ends which are capable of handling the requested 97function. 98When a request is scheduled for processing, the 99.Nm dmover 100scheduler will identify the best back-end to process the request from 101the list of candidate back-ends, in an effort to provide load balancing, 102while considering the relative performance of each back-end. 103.Pp 104A 105.Nm dmover 106function always has one output region. 107A function may have zero or more input regions, or may use an immediate 108value as an input. 109For functions which use input regions, the lengths of each input region 110and the output region must be the same. 111All 112.Nm dmover 113functions with the same name will have the same number of and type inputs. 114If a back-end attempts to register a function which violates this invariant, 115behavior is undefined. 116.Pp 117The 118.Nm dmover 119facility supports several types of buffer descriptors. 120For functions which use input regions, each input buffer descriptor and 121the output buffer descriptor must be of the same type. 122This restriction may be removed in a future revision of the interface. 123.Pp 124The 125.Nm dmover 126facility may need to interrupt request processing and restart it. 127Clients of the 128.Nm dmover 129facility should take care to avoid unwanted side-effects should this occur. 130In particular, for functions which use input regions, no input region may 131overlap with the output region. 132.Ss DATA STRUCTURES 133The 134.Nm dmover 135facility shares several data structures between the client and 136back-end in order to describe sessions and requests. 137.Bd -literal -offset indent 138typedef enum { 139 DMOVER_BUF_LINEAR, 140 DMOVER_BUF_UIO 141} dmover_buffer_type; 142 143typedef struct { 144 void *l_addr; 145 size_t l_len; 146} dmover_buf_linear; 147 148typedef union { 149 dmover_buf_linear dmbuf_linear; 150 struct uio *dmbuf_uio; 151} dmover_buffer; 152.Ed 153.Pp 154Together, these data types are used to describe buffer data structures 155which the 156.Nm dmover 157facility understands. 158Additional buffer types may be added in future revisions of the 159.Nm dmover 160interface. 161.Pp 162The 163.Fa dmover_assignment 164structure contains the information about the back-end to which a 165request is currently assigned. 166It contains the following public members: 167.Bl -tag -width "XXXX" 168.It struct dmover_backend *das_backend 169This is a pointer to the back-end. 170.It const struct dmover_algdesc *das_algdesc 171This is a pointer to the algorithm description provided by 172the back-end for the request's function. 173.El 174.Pp 175The 176.Fa dmover_session 177structure contains the following public members: 178.Bl -tag -width "XXXX" 179.It void *dses_cookie 180This is a pointer to client private data. 181.It int dses_ninputs 182This is the number of inputs used by the selected function. 183.El 184.Pp 185The 186.Fa dmover_request 187structure contains the following public members: 188.Bl -tag -width "XXXX" 189.It TAILQ_ENTRY(dmover_request) dreq_dmbq 190Linkage on the back-end's queue of pending requests. 191.It struct dmover_session *dreq_session 192Pointer to the session with which this request is associated. 193This is intended for use by the back-end. 194.It struct dmover_assignment *dreq_assignment 195Pointer to the 196.Fa dmover_assignment 197structure which describes the back-end to which the request is 198currently assigned. 199The back-end is assigned when the request is scheduled with 200.Fn dmover_process . 201.It void (*dreq_callback)(struct dmover_request *) 202This is a pointer to an optional call-back function provided by the 203client. 204If provided, the call-back is invoked when the request is complete. 205This field must be 206.Dv NULL 207if 208.Em DMOVER_REQ_WAIT 209is set in 210.Em dreq_flags . 211.It void *dreq_cookie 212This is a pointer to client private data specific to the request. 213.It void *dreq_dmbcookie 214This is a pointer to back-end private data, for use while the back-end 215is actively processing a request. 216.It volatile int dreq_flags 217The following flags are defined: 218.Bl -tag -width "DMOVER_REQ_RUNNINGXX" 219.It DMOVER_REQ_DONE 220The request has been completed. 221If not using a call-back, the client may poll this bit to determine 222if a request has been processed. 223.It DMOVER_REQ_ERROR 224An error has occurred while processing the request. 225.It DMOVER_REQ_RUNNING 226The request is currently being executed by the back-end. 227Once a command is running, it cannot be cancelled, and must run to completion. 228.It DMOVER_REQ_WAIT 229If set by the client, 230.Fn dmover_process 231will wait for the request to complete using 232.Xr cv_wait 9 . 233This flag may only be used if the caller has a valid thread context. 234If this flag is set, a callback may not be used. 235.El 236.It int dreq_error 237If the 238.Em DMOVER_REQ_ERROR 239bit is set, this contains the 240.Xr errno 2 241value indicating the error that occurred during processing. 242.It dmover_buffer_type dreq_outbuf_type 243The type of the output buffer. 244.It dmover_buffer dreq_outbuf 245The output buffer. 246.It uint8_t dreq_immediate[8] 247This is the input for algorithms which use an immediate value. 248Values smaller than 8 bytes should use the least-significant bytes first. 249For example, a 32-bit integer would occupy bytes 0, 1, 2, and 3. 250.It dmover_buffer_type dreq_inbuf_type 251The type of the input buffer. 252This is only used if the 253.Nm dmover 254function has one or more inputs. 255.It dmover_buffer *dreq_inbuf 256A pointer to an array of input buffers. 257This is only used if the 258.Nm dmover 259function has one or more inputs. 260The number of inputs, and thus the number of valid elements in the 261array, is specified by the algorithm description for the session. 262.El 263.Ss CLIENT INTERFACE 264The following functions are provided to the client: 265.Bl -tag -width "XXXX" 266.It Fn dmover_session_create "function" "sessionp" 267.Pp 268The 269.Fn dmover_session_create 270function creates a data mover session for the specified data movement 271function 272.Fa function . 273A handle to the new session is returned in 274.Fa sessionp . 275.Pp 276The following are valid data movement function names: 277.Bl -tag -width "fill8xx" 278.It Dq zero 279Fill a memory region with zeros. 280This algorithm has an input count of 0. 281.It Dq fill8 282Fill a memory region with an 8-bit pattern. 283This algorithm has an input count of 0. 284The pattern is provided in the 285.Em dreq_imm8 286member of the 287.Fa dmover_request 288structure. 289.It Dq copy 290Copy a memory region from one location to another. 291This algorithm has an input count of 1. 292.It Dq xor2 293Perform an XOR operation on 2 inputs. 294This algorithm has an input count of 2. 295.It Dq xor3 296Perform an XOR operation on 3 inputs. 297This algorithm has an input count of 3. 298.It Dq xor4 299Perform an XOR operation on 4 inputs. 300This algorithm has an input count of 4. 301.It Dq xor5 302Perform an XOR operation on 5 inputs. 303This algorithm has an input count of 5. 304.It Dq xor6 305Perform an XOR operation on 6 inputs. 306This algorithm has an input count of 6. 307.It Dq xor7 308Perform an XOR operation on 7 inputs. 309This algorithm has an input count of 7. 310.It Dq xor8 311Perform an XOR operation on 8 inputs. 312This algorithm has an input count of 8. 313.El 314.Pp 315Users of the 316.Nm dmover 317facility are encouraged to use the following aliases for the well-known 318function names, as doing so saves space and reduces the chance of programming 319errors: 320.Bl -tag -width "DMOVER_FUNC_FILL32xx" 321.It DMOVER_FUNC_ZERO 322.Dq zero 323.Pq Va dmover_funcname_zero 324.It DMOVER_FUNC_FILL8 325.Dq fill8 326.Pq Va dmover_funcname_fill8 327.It DMOVER_FUNC_COPY 328.Dq copy 329.Pq Va dmover_funcname_copy 330.It DMOVER_FUNC_XOR2 331.Dq xor2 332.Pq Va dmover_funcname_xor2 333.It DMOVER_FUNC_XOR3 334.Dq xor3 335.Pq Va dmover_funcname_xor3 336.It DMOVER_FUNC_XOR4 337.Dq xor4 338.Pq Va dmover_funcname_xor4 339.It DMOVER_FUNC_XOR5 340.Dq xor5 341.Pq Va dmover_funcname_xor5 342.It DMOVER_FUNC_XOR6 343.Dq xor6 344.Pq Va dmover_funcname_xor6 345.It DMOVER_FUNC_XOR7 346.Dq xor7 347.Pq Va dmover_funcname_xor7 348.It DMOVER_FUNC_XOR8 349.Dq xor8 350.Pq Va dmover_funcname_xor8 351.El 352.It Fn dmover_session_destroy "session" 353.Pp 354The 355.Fn dmover_session_destroy 356function tears down a data mover session and releases all resources 357associated with it. 358.It Fn dmover_request_alloc "session" "inbuf" 359.Pp 360The 361.Fn dmover_request_alloc 362function allocates a 363.Nm dmover 364request structure and associates it with the specified session. 365If the 366.Fa inbuf 367argument is not 368.Dv NULL , 369.Fa inbuf 370is used as the array of input buffer descriptors in the request. 371Otherwise, if 372.Fa inbuf 373is 374.Dv NULL 375and the 376.Nm dmover 377function requires input buffers, the input buffer descriptors will be 378allocated automatically using 379.Xr malloc 9 . 380.Pp 381If the request structure or input buffer descriptors cannot be allocated, 382.Fn dmover_request_alloc 383return 384.Dv NULL 385to indicate failure. 386.It Fn dmover_request_free "req" 387.Pp 388The 389.Fn dmover_request_free 390function frees a 391.Nm dmover 392request structure. 393If the 394.Nm dmover 395function requires input buffers, and the input buffer descriptors 396associated with 397.Fa req 398were allocated by 399.Fn dmover_request_alloc , 400then the input buffer descriptors will also be freed. 401.It Fn dmover_process "req" 402.Pp 403The 404.Fn dmover_process 405function submits the 406.Nm dmover 407request 408.Fa req 409for processing. 410The call-back specified by the request is invoked when processing is 411complete. 412.El 413.Pp 414The 415.Fn dmover_session_create 416and 417.Fn dmover_session_destroy 418functions must not be called from interrupt context. 419.Pp 420The 421.Fn dmover_request_alloc , 422.Fn dmover_request_free , 423and 424.Fn dmover_process 425functions may be called from interrupt handlers at levels 426.Em IPL_VM , 427.Em IPL_SOFTCLOCK , 428and 429.Em IPL_SOFTNET , 430or in non-interrupt context. 431.Pp 432The request completion call-back is called from a software interrupt 433handler at 434.Em IPL_SOFTCLOCK . 435.Ss BACK-END INTERFACE 436A back-end describes the 437.Nm dmover 438functions it can perform using an array of 439.Fa dmover_algdesc 440structures: 441.Bd -literal -offset indent 442struct dmover_algdesc { 443 const char *dad_name; /* algorithm name */ 444 void *dad_data; /* opaque algorithm description */ 445 int dad_ninputs; /* number of inputs */ 446}; 447.Ed 448.Pp 449The 450.Em dad_name 451member points to a valid 452.Nm dmover 453function name which the client may specify. 454The 455.Em dad_data 456member points to a back-end-specific description of the algorithm. 457.Pp 458A back-end presents itself to the 459.Nm dmover 460facility using the 461.Fa dmover_backend 462structure. 463The back-end must initialize the following members of the structure: 464.Bl -tag -width "XXXX" 465.It const char *dmb_name 466This is the name of the back-end. 467.It u_int dmb_speed 468This is an estimate of the number of kilobytes/second that the 469back-end can process. 470.It void *dmb_cookie 471This is a pointer to back-end private data. 472.It const struct dmover_algdesc *dmb_algdescs 473This points to an array of 474.Fa dmover_algdesc 475structures which describe the functions the data mover can perform. 476.It int dmb_nalgdescs 477This is the number of elements in the 478.Em dmb_algdescs 479array. 480.It void (*dmb_process)(struct dmover_backend *) 481This is the entry point to the back-end used to process requests. 482.El 483.Pp 484When invoked by the 485.Nm dmover 486facility, the back-end's 487.Fn (*dmb_process) 488function should examine the pending request queue in its 489.Fa dmover_backend 490structure: 491.Bl -tag -width "XXXX" 492.It TAILQ_HEAD(, dmover_request) dmb_pendreqs 493This is the queue of pending requests. 494.It int dmb_npendreqs 495This is the number of requests in the 496.Em dmb_pendreqs 497queue. 498.El 499.Pp 500If an error occurs when processing the request, the 501.Em DMOVER_REQ_ERROR 502bit must be set in the 503.Em dreq_flags 504member of the request, and the 505.Em dreq_error 506member set to an 507.Xr errno 2 508value to indicate the error. 509.Pp 510When the back-end has finished processing the request, it must call 511the 512.Fn dmover_done 513function. 514This function eventually invokes the client's call-back routine. 515.Pp 516If a hardware-assisted data mover uses interrupts, the interrupt handlers 517should be registered at IPL_VM. 518.Pp 519The following functions are provided to the back-ends: 520.Bl -tag -width "XXXX" 521.It Fn dmover_backend_register "backend" 522.Pp 523The 524.Fn dmover_backend_register 525function registers the back-end 526.Fa backend 527with the 528.Nm dmover 529facility. 530.It Fn dmover_backend_unregister "backend" 531.Pp 532The 533.Fn dmover_backend_unregister 534function removes the back-end 535.Fa backend 536from the 537.Nm dmover 538facility. 539The back-end must already be registered. 540.It Fn dmover_done "req" 541.Pp 542The 543.Fn dmover_done 544function is called by the back-end when it has finished processing 545a request, whether the request completed successfully or not. 546.El 547.Pp 548The 549.Fn dmover_backend_register 550and 551.Fn dmover_backend_unregister 552functions must not be called from interrupt context. 553.Pp 554The 555.Fn dmover_done 556function may be called at 557.Em IPL_VM , 558.Em IPL_SOFTCLOCK , 559.Em IPL_SOFTNET , 560or in non-interrupt context. 561.Sh EXAMPLES 562The following is an example of a client using 563.Nm dmover 564to zero-fill a region of memory. 565In this example, the CPU will be able to context switch to another 566thread and perform work while the hardware-assisted data mover clears 567the specified block of memory. 568.Bd -literal 569int 570hw_bzero(void *buf, size_t len) 571{ 572 struct dmover_session *dses; 573 struct dmover_request *dreq; 574 int error; 575 576 error = dmover_session_create(DMOVER_FUNC_ZERO, &dses); 577 if (error) 578 return (error); 579 580 dreq = dmover_request_alloc(dses, NULL); 581 if (dreq == NULL) { 582 dmover_session_destroy(dses); 583 return (ENOMEM); 584 } 585 586 dreq->dreq_flags = DMOVER_REQ_WAIT; 587 dreq->dreq_callback = NULL; 588 dreq->dreq_outbuf.dreq_outbuf_type = DMOVER_BUF_LINEAR; 589 dreq->dreq_outbuf.dmbuf_linear.l_addr = buf; 590 dreq->dreq_outbuf.dmbuf_linear.l_len = len; 591 592 dmover_process(dreq); 593 594 error = (dreq->dreq_flags & DMOVER_REQ_ERROR) ? 595 dreq->dreq_error : 0; 596 597 dmover_request_free(dreq); 598 dmover_session_destroy(dses); 599 600 return (error); 601} 602.Ed 603.Sh SEE ALSO 604.Xr queue 3 , 605.Xr dmoverio 4 606.Sh HISTORY 607The 608.Nm dmover 609facility first appeared in 610.Nx 2.0 . 611.Sh AUTHORS 612The 613.Nm dmover 614facility was designed and implemented by 615.An Jason R. Thorpe 616.Aq thorpej@wasabisystems.com 617and contributed by Wasabi Systems, Inc. 618.Sh BUGS 619The mechanism by which a back-end should advertise its performance to 620the request scheduler is not well-defined. 621Therefore, the load-balancing mechanism within the request scheduler is 622also not well-defined. 623