Relax DMA restrictions on ehci(4) and xhci(4) controllers that can do
64-bit DMA.

ok mpi@, mlarkin@

skip Controller Save State (CSS) and Controller Restore State (CRS)
on AMD 17h/1xh xHCI

avoids problem with resume after CRS was introduced in xhci.c rev 1.133
uhub0: device problem, disabling port

skip Controller Save State (CSS) and Controller Restore State (CRS)
on AMD 17h/1xh xHCI

avoids problem with resume after CRS was introduced in xhci.c rev 1.133
uhub0: device problem, disabling port 2
uhub1: device problem, disabling port 1

reported by brynet@ and reproduced on t495, ok kettenis@

show more ...

Upon resume, restore the saved state. Newer Intel xHCI controller (e.g.
those on Meteor Lake) need this, otherwise the ports are dead after
resume.

ok mglocker@, deraadt@

Some Intel xhci(4) controllers don't fully power down unless they've seen
a "save state" command. So use that command when we suspend (and don't
reset the controller at that point such that it doesn

Some Intel xhci(4) controllers don't fully power down unless they've seen
a "save state" command. So use that command when we suspend (and don't
reset the controller at that point such that it doesn't forget about it).
Note that on resume we don't restore the state. Instead we just reset
the controller and bring it up from scratch. There isn't much state to
save anyway since we detach all USB devices when we suspend.

ok mlarkin@, deraadt@

show more ...

remove unneeded includes; ok mpi@

Revert rev 1.129:
When detaching devices when we suspend, we need to continue processing
command completion events. So only return early in xhci_softintr() if
the controller is dead instead of dying

Revert rev 1.129:
When detaching devices when we suspend, we need to continue processing
command completion events. So only return early in xhci_softintr() if
the controller is dead instead of dying. This fixes USB suspend/resume
in Apple M1/M2.

ok mlarkin@, deraadt@
-----

The change does not only allow completion events to be processed but also
events that get processed by the drivers attached to this usb bus. As a
result I see a uvm panic on suspend in urtwn(4) which is triggered by a
bad rx eof.

OK kettenis@

show more ...

When detaching devices when we suspend, we need to continue processing
command completion events. So only return early in xhci_softintr() if
the controller is dead instead of dying. This fixes USB

When detaching devices when we suspend, we need to continue processing
command completion events. So only return early in xhci_softintr() if
the controller is dead instead of dying. This fixes USB suspend/resume
in Apple M1/M2.

ok mlarkin@, deraadt@

show more ...

Clear status bits when we receive a host system error. Otherwise the
interrupt keeps firing even if the hos controller has been declared dead.

ok mpi@

Fix USB hotplug on type-C connectors of Apple Silicon hardware. The USB
controller on these machines does not see connection events. Instead we
need to rely on the USB PD controllers to notify us o

Fix USB hotplug on type-C connectors of Apple Silicon hardware. The USB
controller on these machines does not see connection events. Instead we
need to rely on the USB PD controllers to notify us of a new connection
and reset the USB controller. This diff implements this by adding a new
tipd(4) driver and infrastructure to notify xhci(4) of new connections.

ok patrick@

show more ...

Use a separate varaible to signal a "dead" controller (that has disappeared
from the bus, probably because of hot(un)plugging it). This makes sure
that we still ack interrupts that arrive while the

Use a separate varaible to signal a "dead" controller (that has disappeared
from the bus, probably because of hot(un)plugging it). This makes sure
that we still ack interrupts that arrive while the controller is "dying"
(when we go down into suspend or if an unrecoverable hardware error
occurs). Fixes an interrupt storm when we suspend on Amlogic arm64 boards.

ok deraadt@

show more ...

constify USB host controller method tables and root hub descriptors

ok mglocker@

spelling
feedback and ok tb@ jmc@ ok ratchov@

Align memory allocation for USB device drivers and USB HC drivers:

* USB device drivers use M_USBDEV instead of M_DEVBUF.
* USB HC drivers use M_USBHC instead of M_DEVBUF.

In a vanilla setup, this

Align memory allocation for USB device drivers and USB HC drivers:

* USB device drivers use M_USBDEV instead of M_DEVBUF.
* USB HC drivers use M_USBHC instead of M_DEVBUF.

In a vanilla setup, this enlarges the USB memory pool.

ok anton@

show more ...

Improve unhibernate performance (30% on some machines, another upcoming diff
shows gains up to 50%) by skipping attach of irrelevant devices, which are
tagged CD_SKIPHIBERNATE in the per-driver cfdri

Improve unhibernate performance (30% on some machines, another upcoming diff
shows gains up to 50%) by skipping attach of irrelevant devices, which are
tagged CD_SKIPHIBERNATE in the per-driver cfdriver. In particular, usb devices
are not attached, so they don't need to detach during the suspend-unpack-resume.
New bootblocks are required (which tell the kernel it's job is unhibernate
before configure runs)
tested by various

show more ...

remove unused xhci_cmd_evaluate_ctx()

ok mpi@

Do proper accounting of zero length TDs. Currently a specific number
of zero length TDs can cause our free TRBs to run out, causing xhci(4)
to return USBD_NOMEM to the USB stack.

The issue was repo

Do proper accounting of zero length TDs. Currently a specific number
of zero length TDs can cause our free TRBs to run out, causing xhci(4)
to return USBD_NOMEM to the USB stack.

The issue was reported by
Jonathon Fletcher <jonathon.fletcher () gmail ! com> -- Thanks!

Reviewed/suggestions by patrick@.

ok mpi@

show more ...

In xhci_device_isoc_start() do first check if the transfer is
in-progress before we do check for the pipe being halted.

This fixes some kind of race condition for isoc devices during device
close wh

In xhci_device_isoc_start() do first check if the transfer is
in-progress before we do check for the pipe being halted.

This fixes some kind of race condition for isoc devices during device
close when xp->halted gets set while usbd_start_next() still tries to
dequeue in-progress transfers which will report 'usbd_start_next:
error=13' (USBD_IOERROR).

ok mpi@

show more ...

Don't nest a enum declaration inside a struct declaration, and anyway
since the enum isn't tied to the type (because want smaller storage)
is better to use #define
ok kettenis mglocker

Fix a problem related to isochronous transfers appearing in certain
constellations resulting in bogus frame sizes being returned by xhci(4).
E.g. for uvideo(4) devices erroneous video streams were re

Fix a problem related to isochronous transfers appearing in certain
constellations resulting in bogus frame sizes being returned by xhci(4).
E.g. for uvideo(4) devices erroneous video streams were reported.

The problem occurs when multi-trb TDs are queued and being processed
as a zero-length or short transfer. Those cases were not handled in
the current isochronous code path, which this patch is adding.

Feedback and testing done by many on tech@. Thanks!

ok mpi@

show more ...

When a transfer times out, the TRB should be aborted, too. But still
the completion interrupt may already be pending while aborting.
Hence, in xhci_event_command() ignore events from a TRB that is no

When a transfer times out, the TRB should be aborted, too. But still
the completion interrupt may already be pending while aborting.
Hence, in xhci_event_command() ignore events from a TRB that is not
the expected one. And don't let xhci_abort_command() yield the CPU.

ok mpi@

show more ...

Acknowledge xhci(4) interrupts before calling usb_schedsoftintr().

On powerdown (halt -p), sd(4)'s suspend function tries to powerdown
a USB mass storage using a STOP command. In that case we are a

Acknowledge xhci(4) interrupts before calling usb_schedsoftintr().

On powerdown (halt -p), sd(4)'s suspend function tries to powerdown
a USB mass storage using a STOP command. In that case we are already
cold and splhigh(), so that the xhci is supposed to run in polling-
mode.

usb_schedsoftintr() behaves differently when running in polling-mode.
Instead of scheduling a soft interrupt, it immediately dequeues from
the event queue. But dequeueing means touching the xhci registers.

Apparently we need to acknowledge the interrupts before touching those
registers, the hardware doesn't like it otherwise and we will never get
an interrupt status for the second transfer.

ok gerhard@

show more ...

Move the responsibility of syncing the data buffers from the USB
stack into the USB controller driver, since not all of the xfers
actually use DMA and some invalidations might even be harmful.

The i

Move the responsibility of syncing the data buffers from the USB
stack into the USB controller driver, since not all of the xfers
actually use DMA and some invalidations might even be harmful.

The important part is to sync before handing the buffer to the
controller, and to sync on a successful transfer before handing
it back to the USB stack. For isoc transfers it's easier to sync
the complete length of the transfer, since the buffer to flush is
not filled in a contiguous fashion.

For dwc2 there's a common point which takes care of the start
of transfers from or to devices, where we can make sure that our
buffers are synced. On completion, there's a common point before
handing it off to the USB stack.

For ehci there are three places which take care of the start
of transfers from or to devices, where one already does the sync,
while the two other places still need the sync. There are two
completion handler (isoc and non-isoc), where one already has a
comment asking for the need of a sync. The done handler for intr
xfers does a sync that is not needed anymore after adding the sync
in the completion handler.

For ohci there are three places which take care of the start
of transfers from or to devices, where all of them were still in
need of the sync. For completion, there is one place for isoc
xfers and two places for handling successful generic xfers.

For uhci there are two places which take care of the start
of transfers from or to device, where all of them were still in
need of the sync. For completion, there is one handler for both
isoc and non-isoc xfers where syncs need to occur.

For xhci there are three places which take care of the start
of transfers from or to device, where all of them were still in
need of the sync. For completion, there is one handler for isoc
and one for non-isoc xfers where syncs need to occur.

With this we can revert the workaround that implicitly allocated
buffers are COHERENT, since now control transfers fulfilled by the
driver code (instead of the controller doing DMA) are not flushed
into oblivion anymore.

Tested by Janne Johansson with dwc2 (octeon)
Tested by kettenis@ with xhci(4) (octeon)
Tested by patrick@ with ehci(4) on a Cubox-i (armv7)
Tested by patrick@ with xhci(4) on an i.MX8MQ (arm64)
Tested by tobhe@ with dwc2 on a rPi 3b (arm64)
ok kettenis@

show more ...

Fix endian swapping of trb_flags and trb_status in xhci(4).

This corrects computation of xfer->actlen, allowing xhci(4) to work
again on octeon and other big endian architectures.

Reported by Allen

Fix endian swapping of trb_flags and trb_status in xhci(4).

This corrects computation of xfer->actlen, allowing xhci(4) to work
again on octeon and other big endian architectures.

Reported by Allen Smith

OK patrick@

show more ...

use the UE_GET_XFERTYPE macro where applicable

ok mpi@

Add a zero length TD, rather than adding a zero length TRB to the
original TD, when a transfer is a multiple of the max packet size. The
zero length TD will have a NULL xfer pointer.

As a result "NU

Add a zero length TD, rather than adding a zero length TRB to the
original TD, when a transfer is a multiple of the max packet size. The
zero length TD will have a NULL xfer pointer.

As a result "NULL xfer pointer" situations become perfectly normal. So
change the log_warnx() that issues that verbiage to a log_debug().

Note that the original transfer will complete and report its result up
the USB stack before the zero length transfer is executed.

Fixes (at least) urtwn(4) interfaces.

Feeback, cluebats, fixes and ok patrick@

show more ...