1394324d3SSascha Wildner.\" Copyright (c) 2015-2017 Amazon.com, Inc. or its affiliates. 2394324d3SSascha Wildner.\" All rights reserved. 3394324d3SSascha Wildner.\" 4394324d3SSascha Wildner.\" Redistribution and use in source and binary forms, with or without 5394324d3SSascha Wildner.\" modification, are permitted provided that the following conditions 6394324d3SSascha Wildner.\" are met: 7394324d3SSascha Wildner.\" 8394324d3SSascha Wildner.\" 1. Redistributions of source code must retain the above copyright 9394324d3SSascha Wildner.\" notice, this list of conditions and the following disclaimer. 10394324d3SSascha Wildner.\" 11394324d3SSascha Wildner.\" 2. Redistributions in binary form must reproduce the above copyright 12394324d3SSascha Wildner.\" notice, this list of conditions and the following disclaimer in 13394324d3SSascha Wildner.\" the documentation and/or other materials provided with the 14394324d3SSascha Wildner.\" distribution. 15394324d3SSascha Wildner.\" 16394324d3SSascha Wildner.\" THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 17394324d3SSascha Wildner.\" "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 18394324d3SSascha Wildner.\" LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 19394324d3SSascha Wildner.\" A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 20394324d3SSascha Wildner.\" OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 21394324d3SSascha Wildner.\" SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT 22394324d3SSascha Wildner.\" LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 23394324d3SSascha Wildner.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 24394324d3SSascha Wildner.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 25394324d3SSascha Wildner.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 26394324d3SSascha Wildner.\" OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 27394324d3SSascha Wildner.\" 28394324d3SSascha Wildner.\" $FreeBSD: head/share/man/man4/ena.4 322683 2017-08-19 00:51:45Z emaste $ 29394324d3SSascha Wildner.\" 3082a3fa28SBrad Hoffman.Dd July 22, 2018 31394324d3SSascha Wildner.Dt ENA 4 32394324d3SSascha Wildner.Os 33394324d3SSascha Wildner.Sh NAME 34394324d3SSascha Wildner.Nm ena 3582a3fa28SBrad Hoffman.Nd "DragonFly kernel driver for the Elastic Network Adapter (ENA) family" 36394324d3SSascha Wildner.Sh SYNOPSIS 37394324d3SSascha WildnerTo compile this driver into the kernel, 38394324d3SSascha Wildnerplace the following line in your 39394324d3SSascha Wildnerkernel configuration file: 40394324d3SSascha Wildner.Bd -ragged -offset indent 41394324d3SSascha Wildner.Cd "device ena" 42394324d3SSascha Wildner.Ed 43394324d3SSascha Wildner.Pp 44394324d3SSascha WildnerAlternatively, to load the driver as a 45394324d3SSascha Wildnermodule at boot time, place the following line in 46394324d3SSascha Wildner.Xr loader.conf 5 : 47394324d3SSascha Wildner.Bd -literal -offset indent 48394324d3SSascha Wildnerif_ena_load="YES" 49394324d3SSascha Wildner.Ed 50394324d3SSascha Wildner.Sh DESCRIPTION 51394324d3SSascha WildnerThe ENA is a networking interface designed to make good use of modern CPU 52394324d3SSascha Wildnerfeatures and system architectures. 53394324d3SSascha Wildner.Pp 54394324d3SSascha WildnerThe ENA device exposes a lightweight management interface with a 55394324d3SSascha Wildnerminimal set of memory mapped registers and extendable command set 56394324d3SSascha Wildnerthrough an Admin Queue. 57394324d3SSascha Wildner.Pp 58394324d3SSascha WildnerThe driver supports a range of ENA devices, is link-speed independent 59394324d3SSascha Wildner(i.e., the same driver is used for 10GbE, 25GbE, 40GbE, etc.), and has 60394324d3SSascha Wildnera negotiated and extendable feature set. 61394324d3SSascha Wildner.Pp 62394324d3SSascha WildnerSome ENA devices support SR-IOV. This driver is used for both the 63394324d3SSascha WildnerSR-IOV Physical Function (PF) and Virtual Function (VF) devices. 64394324d3SSascha Wildner.Pp 65394324d3SSascha WildnerThe ENA devices enable high speed and low overhead network traffic 66394324d3SSascha Wildnerprocessing by providing multiple Tx/Rx queue pairs (the maximum number 67394324d3SSascha Wildneris advertised by the device via the Admin Queue), a dedicated MSI-X 68394324d3SSascha Wildnerinterrupt vector per Tx/Rx queue pair, and CPU cacheline optimized 69394324d3SSascha Wildnerdata placement. 70394324d3SSascha Wildner.Pp 71394324d3SSascha WildnerThe 72394324d3SSascha Wildner.Nm 73394324d3SSascha Wildnerdriver supports industry standard TCP/IP offload features such 74394324d3SSascha Wildneras checksum offload and TCP transmit segmentation offload (TSO). 75394324d3SSascha WildnerReceive-side scaling (RSS) is supported for multi-core scaling. 76394324d3SSascha Wildner.Pp 77394324d3SSascha WildnerThe 78394324d3SSascha Wildner.Nm 79394324d3SSascha Wildnerdriver and its corresponding devices implement health 80394324d3SSascha Wildnermonitoring mechanisms such as watchdog, enabling the device and driver 81394324d3SSascha Wildnerto recover in a manner transparent to the application, as well as 82394324d3SSascha Wildnerdebug logs. 83394324d3SSascha Wildner.Pp 84394324d3SSascha WildnerSome of the ENA devices support a working mode called Low-latency 85394324d3SSascha WildnerQueue (LLQ), which saves several more microseconds. This feature will 86394324d3SSascha Wildnerbe implemented for driver in future releases. 87394324d3SSascha Wildner.Sh HARDWARE 88394324d3SSascha WildnerSupported PCI vendor ID/device IDs: 89394324d3SSascha Wildner.Pp 90394324d3SSascha Wildner.Bl -bullet -compact 91394324d3SSascha Wildner.It 92394324d3SSascha Wildner1d0f:0ec2 - ENA PF 93394324d3SSascha Wildner.It 94394324d3SSascha Wildner1d0f:1ec2 - ENA PF with LLQ support 95394324d3SSascha Wildner.It 96394324d3SSascha Wildner1d0f:ec20 - ENA VF 97394324d3SSascha Wildner.It 98394324d3SSascha Wildner1d0f:ec21 - ENA VF with LLQ support 99394324d3SSascha Wildner.El 100394324d3SSascha Wildner.Sh DIAGNOSTICS 101394324d3SSascha Wildner.Ss Device initialization phase: 102394324d3SSascha Wildner.Bl -diag 103394324d3SSascha Wildner.It ena%d: failed to init mmio read less 104394324d3SSascha Wildner.Pp 105394324d3SSascha WildnerError occurred during initialization of the mmio register read request. 106394324d3SSascha Wildner.It ena%d: Can not reset device 107394324d3SSascha Wildner.Pp 108394324d3SSascha WildnerDevice could not be reset; device may not be responding or is already 109394324d3SSascha Wildnerduring reset. 110394324d3SSascha Wildner.It ena%d: device version is too low 111394324d3SSascha Wildner.Pp 112394324d3SSascha WildnerVersion of the controller is too low and it is not supported by the driver. 113394324d3SSascha Wildner.It ena%d: Invalid dma width value %d 114394324d3SSascha Wildner.Pp 115*bbb35c81SSascha WildnerThe controller is able to request dma transaction width. Device stopped 116394324d3SSascha Wildnerresponding or it demanded invalid value. 117394324d3SSascha Wildner.It ena%d: Can not initialize ena admin queue with device 118394324d3SSascha Wildner.Pp 119394324d3SSascha WildnerInitialization of the Admin Queue failed; device may not be responding or there 120394324d3SSascha Wildnerwas a problem with initialization of the resources. 121394324d3SSascha Wildner.It ena%d: Cannot get attribute for ena device rc: %d 122394324d3SSascha Wildner.Pp 123394324d3SSascha WildnerFailed to get attributes of the device from the controller. 124394324d3SSascha Wildner.It ena%d: Cannot configure aenq groups rc: %d 125394324d3SSascha Wildner.Pp 126394324d3SSascha WildnerErrors occurred when trying to configure AENQ groups. 127394324d3SSascha Wildner.El 128394324d3SSascha Wildner.Ss Driver initialisation/shutdown phase: 129394324d3SSascha Wildner.Bl -diag 130394324d3SSascha Wildner.It ena%d: PCI resource allocation failed! 131394324d3SSascha Wildner.It ena%d: allocating ena_dev failed 132394324d3SSascha Wildner.It ena%d: failed to pmap registers bar 133394324d3SSascha Wildner.It ena%d: Error while setting up bufring 134394324d3SSascha Wildner.It ena%d: Error with initialization of IO rings 135394324d3SSascha Wildner.It ena%d: can not allocate ifnet structure 136394324d3SSascha Wildner.It ena%d: Error with network interface setup 137394324d3SSascha Wildner.It ena%d: Failed to enable and set the admin interrupts 138394324d3SSascha Wildner.It ena%d: Failed to allocate %d, vectors %d 139394324d3SSascha Wildner.It ena%d: Failed to enable MSIX, vectors %d rc %d 140394324d3SSascha Wildner.It ena%d: Error with MSI-X enablement 141394324d3SSascha Wildner.It ena%d: could not allocate irq vector: %d 142394324d3SSascha Wildner.It ena%d: Unable to allocate bus resource: registers 143394324d3SSascha Wildner.Pp 144394324d3SSascha WildnerResource allocation failed when initializing the device; driver will not 145394324d3SSascha Wildnerbe attached. 146394324d3SSascha Wildner.It ena%d: ENA device init failed (err: %d) 147394324d3SSascha Wildner.Pp 148394324d3SSascha WildnerDevice initialization failed; driver will not be attached. 149394324d3SSascha Wildner.It ena%d: could not activate irq vector: %d 150394324d3SSascha Wildner.Pp 151394324d3SSascha WildnerError occurred when trying to activate interrupt vectors for Admin Queue. 152394324d3SSascha Wildner.It ena%d: failed to register interrupt handler for irq %ju: %d 153394324d3SSascha Wildner.Pp 154394324d3SSascha WildnerError occurred when trying to register Admin Queue interrupt handler. 155394324d3SSascha Wildner.It ena%d: Cannot setup mgmnt queue intr 156394324d3SSascha Wildner.Pp 157394324d3SSascha WildnerError occurred during configuration of the Admin Queue interrupts. 158394324d3SSascha Wildner.It ena%d: Enable MSI-X failed 159394324d3SSascha Wildner.Pp 160394324d3SSascha WildnerConfiguration of the MSI-X for Admin Queue failed; there could be lack 161394324d3SSascha Wildnerof resources or interrupts could not have been configured; driver will 162394324d3SSascha Wildnernot be attached. 163394324d3SSascha Wildner.It ena%d: VLAN is in use, detach first 164394324d3SSascha Wildner.Pp 165394324d3SSascha WildnerVLANs are being used when trying to detach the driver; VLANs should be detached 166394324d3SSascha Wildnerfirst and then detach routine should be called again. 167394324d3SSascha Wildner.It ena%d: Unmapped RX DMA tag associations 168394324d3SSascha Wildner.It ena%d: Unmapped TX DMA tag associations 169394324d3SSascha Wildner.Pp 170394324d3SSascha WildnerError occurred when trying to destroy RX/TX DMA tag. 171394324d3SSascha Wildner.It ena%d: Cannot init RSS 172394324d3SSascha Wildner.It ena%d: Cannot fill indirect table 173394324d3SSascha Wildner.It ena%d: Cannot fill indirect table 174394324d3SSascha Wildner.It ena%d: Cannot fill hash function 175394324d3SSascha Wildner.It ena%d: Cannot fill hash control 176394324d3SSascha Wildner.It ena%d: WARNING: RSS was not properly initialized, it will affect bandwidth 177394324d3SSascha Wildner.Pp 178394324d3SSascha WildnerError occurred during initialization of one of RSS resources; device is still 179394324d3SSascha Wildnergoing to work but it will affect performance because all RX packets will be 180394324d3SSascha Wildnerpassed to queue 0 and there will be no hash information. 181394324d3SSascha Wildner.It ena%d: failed to tear down irq: %d 182394324d3SSascha Wildner.It ena%d: dev has no parent while releasing res for irq: %d 183394324d3SSascha WildnerRelease of the interrupts failed. 184394324d3SSascha Wildner.El 185394324d3SSascha Wildner.Ss Additional diagnostic: 186394324d3SSascha Wildner.Bl -diag 187394324d3SSascha Wildner.It ena%d: Cannot get attribute for ena device 188394324d3SSascha Wildner.Pp 189394324d3SSascha WildnerThis message appears when trying to change MTU and driver is unable to get 190394324d3SSascha Wildnerattributes from the device. 191394324d3SSascha Wildner.It ena%d: Invalid MTU setting. new_mtu: %d 192394324d3SSascha Wildner.Pp 193394324d3SSascha WildnerRequested MTU value is not supported and will not be set. 194394324d3SSascha Wildner.It ena%d: keep alive watchdog timeout 195394324d3SSascha Wildner.Pp 196394324d3SSascha WildnerDevice stopped responding and will be reset. 197394324d3SSascha Wildner.It ena%d: Found a Tx that wasn't completed on time, qid %d, index %d. 198394324d3SSascha Wildner.Pp 199394324d3SSascha WildnerPacket was pushed to the NIC but not sent within given time limit; it may 200394324d3SSascha Wildnerbe caused by hang of the IO queue. 201394324d3SSascha Wildner.It ena%d: The number of lost tx completion is aboce the threshold (%d > %d). Reset the device 202394324d3SSascha Wildner.Pp 203394324d3SSascha WildnerIf too many Tx wasn't completed on time the device is going to be reset; it may 204394324d3SSascha Wildnerbe caused by hanged queue or device. 205394324d3SSascha Wildner.It ena%d: trigger reset is on 206394324d3SSascha Wildner.Pp 207394324d3SSascha WildnerDevice will be reset; reset is triggered either by watchdog or if too many TX 208394324d3SSascha Wildnerpackets were not completed on time. 209394324d3SSascha Wildner.It ena%d: invalid value recvd 210394324d3SSascha Wildner.Pp 211394324d3SSascha WildnerLink status received from the device in the AENQ handler is invalid. 212394324d3SSascha Wildner.It ena%d: Allocation for Tx Queue %u failed 213394324d3SSascha Wildner.It ena%d: Allocation for Rx Queue %u failed 214394324d3SSascha Wildner.It ena%d: Unable to create Rx DMA map for buffer %d 215394324d3SSascha Wildner.It ena%d: Failed to create io TX queue #%d rc: %d 216394324d3SSascha Wildner.It ena%d: Failed to get TX queue handlers. TX queue num %d rc: %d 217394324d3SSascha Wildner.It ena%d: Failed to create io RX queue[%d] rc: %d 218394324d3SSascha Wildner.It ena%d: Failed to get RX queue handlers. RX queue num %d rc: %d 219394324d3SSascha Wildner.It ena%d: failed to request irq 220394324d3SSascha Wildner.It ena%d: could not allocate irq vector: %d 221394324d3SSascha Wildner.It ena%d: failed to register interrupt handler for irq %ju: %d 222394324d3SSascha Wildner.Pp 223394324d3SSascha WildnerIO resources initialization failed. Interface will not be brought up. 224394324d3SSascha Wildner.It ena%d: LRO[%d] Initialization failed! 225394324d3SSascha Wildner.Pp 226394324d3SSascha WildnerInitialization of the LRO for the RX ring failed. 227394324d3SSascha Wildner.It ena%d: failed to alloc buffer for rx queue 228394324d3SSascha Wildner.It ena%d: failed to add buffer for rx queue %d 229394324d3SSascha Wildner.It ena%d: refilled rx queue %d with %d pages only 230394324d3SSascha Wildner.Pp 231394324d3SSascha WildnerAllocation of resources used on RX path failed; if happened during 232394324d3SSascha Wildnerinitialization of the IO queue, the interface will not be brought up. 233394324d3SSascha Wildner.It ena%d: ioctl promisc/allmulti 234394324d3SSascha Wildner.Pp 235394324d3SSascha WildnerIOCTL request for the device to work in promiscuous/allmulti mode; see 236394324d3SSascha Wildner.Xr ifconfig 8 237394324d3SSascha Wildnerfor more details. 238394324d3SSascha Wildner.It ena%d: too many fragments. Last fragment: %d! 239394324d3SSascha Wildner.Pp 240394324d3SSascha WildnerPacket with unsupported number of segments was queued for sending to the 241394324d3SSascha Wildnerdevice; packet will be dropped. 24282a3fa28SBrad Hoffman.El 243394324d3SSascha Wildner.Sh SUPPORT 244394324d3SSascha WildnerIf an issue is identified with the released source code with a supported adapter 245394324d3SSascha Wildneremail the specific information related to the issue to 246394324d3SSascha Wildner.Aq Mt mk@semihalf.com 247394324d3SSascha Wildnerand 248394324d3SSascha Wildner.Aq Mt mw@semihalf.com . 249394324d3SSascha Wildner.Sh SEE ALSO 250394324d3SSascha Wildner.Xr vlan 4 , 251394324d3SSascha Wildner.Xr ifconfig 8 252394324d3SSascha Wildner.Sh AUTHORS 253394324d3SSascha WildnerThe 254394324d3SSascha Wildner.Nm 255394324d3SSascha Wildnerdriver was written by 256394324d3SSascha Wildner.An Semihalf . 257