nvme/rdma: Fix reinserting qpair in connecting list after stale state

When a qpair is first created, we add it to a list of connecting qpairs.
If the connection fails, we move the qpair to a stale s

nvme/rdma: Fix reinserting qpair in connecting list after stale state

When a qpair is first created, we add it to a list of connecting qpairs.
If the connection fails, we move the qpair to a stale state and we retry
later. At this point, we should not add the qpair again to the
connecting qpairs list.

Change-Id: If38a8a51d3cb86f4d52d926d1acc349af21a6947
Signed-off-by: Alex Michon <amichon@kalrayinc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25526
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot

show more ...

nvme/rdma: Don't limit max_sge if UMR is used

Since UMR creates a vurtually contig memory buffer,
we can always support up to 16 SGEs regardless of
MSDBD reported by target

Signed-off-by: Alexey Ma

nvme/rdma: Don't limit max_sge if UMR is used

Since UMR creates a vurtually contig memory buffer,
we can always support up to 16 SGEs regardless of
MSDBD reported by target

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: Ibd339f71ad35d355783993f777fcf8009ea68466
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24710
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <ben@nvidia.com>
Reviewed-by: Jim Harris <jim.harris@nvidia.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Community-CI: Mellanox Build Bot

show more ...

nvme/rdma: Register UMR per IO request

If accel sequence is supported, append a copy
task even if there is no accel sequence. NVME RDMA
driver expects that accel framework registers UMR
for the data

nvme/rdma: Register UMR per IO request

If accel sequence is supported, append a copy
task even if there is no accel sequence. NVME RDMA
driver expects that accel framework registers UMR
for the data buffer. This UMR allows to represent
fragmented payload as a virtually contig one.

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I410f991959b08eab033105a7dbb4a9aaba491567
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24709
Reviewed-by: Ben Walker <ben@nvidia.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Reviewed-by: Jim Harris <jim.harris@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

nvme: add poll_group interrupt callback

In interrupt mode, IO completions are processed when waiting on
poll_group's fd_group. But there are some events (qpair disconnection)
that require extra han

nvme: add poll_group interrupt callback

In interrupt mode, IO completions are processed when waiting on
poll_group's fd_group. But there are some events (qpair disconnection)
that require extra handling. Normally, this happens in
spdk_nvme_poll_group_wait(), but when manually doing a
spdk_fd_group_wait() on poll_group's fd_group, we need a notification to
get this done.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I08979e42ff57b53f0c97670e9996b0ce6dad713e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25468
Reviewed-by: Ankit Kumar <ankit.kumar@samsung.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <ben@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@nvidia.com>

show more ...

nvme: add spdk_nvme_poll_group_get_fd_group()

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I0eb30622baf8d1d0ba0af632482570aaaeef52af
Reviewed-on: https://review.spdk.io/gerrit

nvme: add spdk_nvme_poll_group_get_fd_group()

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I0eb30622baf8d1d0ba0af632482570aaaeef52af
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25467
Reviewed-by: Ankit Kumar <ankit.kumar@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@nvidia.com>
Reviewed-by: Ben Walker <ben@nvidia.com>

show more ...

nvme/rdma: Force qp disconnect on pg remove

If a qpair is removed from a poll group and it still
has a poller, we must force qpair disconnect because
otherwise group reference is removed an we wont'

nvme/rdma: Force qp disconnect on pg remove

If a qpair is removed from a poll group and it still
has a poller, we must force qpair disconnect because
otherwise group reference is removed an we wont' be
able to release the poller

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I42699e4a692e6b878a828812328737a729e0295e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25513
Reviewed-by: Jim Harris <jim.harris@nvidia.com>
Reviewed-by: Ben Walker <ben@nvidia.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot

show more ...

nvme/rdma: Flush queued send WRs when disconnecting a qpair

This will prevent staying in lingering state until the disconnection
timeout for no good reason.

Change-Id: Ife01eb2a7dd28e000fee15fba10d

nvme/rdma: Flush queued send WRs when disconnecting a qpair

This will prevent staying in lingering state until the disconnection
timeout for no good reason.

Change-Id: Ife01eb2a7dd28e000fee15fba10dfd8aa7802725
Signed-off-by: Alex Michon <amichon@kalrayinc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25429
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <ben@nvidia.com>

show more ...

nvme/rdma: Prevent submitting new recv WR when disconnecting

If we are in a disconnection process, we may never get WC for these
recv WR and we will have to wait the entire disconnection timeout
bef

nvme/rdma: Prevent submitting new recv WR when disconnecting

If we are in a disconnection process, we may never get WC for these
recv WR and we will have to wait the entire disconnection timeout
before deciding to destroy the qpair.

Change-Id: Ifdd5ed7866dec4c3e8b37b45aea2c95293c0d994
Signed-off-by: Alex Michon <amichon@kalrayinc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25415
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <ben@nvidia.com>

show more ...

nvme/rdma: Support accel sequence

If a request has an accel sequence, we append a copy task
with RDMA memory domain and don't send capsule until
the data_transfer callback is called. In the callback

nvme/rdma: Support accel sequence

If a request has an accel sequence, we append a copy task
with RDMA memory domain and don't send capsule until
the data_transfer callback is called. In the callback
we expect to get a single iov and a memory key which
are sent in NVMF capsule to remote taget. When network
transmission is finished, we finish data tranfer operation.
The reuqest is completed in accel sequence finish_cb.
A request which is executing accel sequence has a special
flag, we don't abort such requests.
Also, we store the data transfer completion callback and
call it in case of network failure.
Added tests for this feature

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I021bd5f268185a5e1b2d77eb098f8daf491aacf9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24702
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <ben@nvidia.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>

show more ...

lib/nvme: destruct controllers that failed init asynchronously

The controller destroy sequence is as follows:
- Set `CC.SHN` to request shutdown
- Wait for `CSTS.SHST` to be set to `0b10` (Shutdown

lib/nvme: destruct controllers that failed init asynchronously

The controller destroy sequence is as follows:
- Set `CC.SHN` to request shutdown
- Wait for `CSTS.SHST` to be set to `0b10` (Shutdown complete)
- Destroy the associated structs when it's done or after a timeout
To do it, two things should be done:
- First, call `nvme_ctrlr_destruct_async`
- Then, poll `nvme_ctrlr_destruct_poll_async`

However, when a controller fails to initialize on probe, this polling is
done synchronously using `nvme_ctrlr_destruct`, which introduces 1ms
sleep between each poll.

This is really bad if a controller does not behave as expected and does
not set its `CSTS.SHST` in a timely manner because it burdens the
probe thread with tons of sleep. If hot-plug is enabled, it makes things
even worse because this operation is retried again and again.

Fix this by doing an asynchronous destruct when the controller fails to
initialize. Add contexts for this operation on the probe context and
poll for controllers destruction in the probe poller function.

Signed-off-by: Nathan Claudel <nclaudel@kalrayinc.com>
Change-Id: Ic072a2b7c3351a229d3b6e5c667b71dca2a84b93
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25414
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Vasuki Manikarnike <vasuki.manikarnike@hpe.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Reviewed-by: Jim Harris <jim.harris@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ankit Kumar <ankit.kumar@samsung.com>
Reviewed-by: Changpeng Liu <changpeliu@tencent.com>
Community-CI: Mellanox Build Bot

show more ...

nvme: Fix discovery loop when target has no entry

When a target returns no discovery entry, we may enter in a loop of
discovery because we don't correctly set start_genctr. So the exit
condition of

nvme: Fix discovery loop when target has no entry

When a target returns no discovery entry, we may enter in a loop of
discovery because we don't correctly set start_genctr. So the exit
condition of the loop `ctx->start_genctr == ctx->end_genctr` is never
achieved.

Change-Id: I1fed834f6a2da1099bddcf7599408f01831288a1
Signed-off-by: Alex Michon <amichon@kalrayinc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25457
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeliu@tencent.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>

show more ...

nvme/nvme: Factor out submit_request function

Move part of code to another function which
will later be called from several places

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I2

nvme/nvme: Factor out submit_request function

Move part of code to another function which
will later be called from several places

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I22b650034b505c9e65b7ec5ea1621a70dc15c576
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25064
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <ben@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>

show more ...

nvme/rdma: Remove qpair::max_recv_sge as unused

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I92cb54e92e93ffccc9bfaa42deab30a5433d336f
Reviewed-on: https://review.spdk.io/gerrit/c

nvme/rdma: Remove qpair::max_recv_sge as unused

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I92cb54e92e93ffccc9bfaa42deab30a5433d336f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24696
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <ben@nvidia.com>

show more ...

nvme/rdma: Add likely/unlikely to IO path

Even though compilers can inline function
implicitly, let's add inline keyword to some functions
which are part of IO path.

Signed-off-by: Alexey Marchuk <

nvme/rdma: Add likely/unlikely to IO path

Even though compilers can inline function
implicitly, let's add inline keyword to some functions
which are part of IO path.

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I54aa24607028dabd70084193b1947049c379f0d9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24695
Reviewed-by: Ben Walker <ben@nvidia.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>

show more ...

nvme/rdma: Factor our contig request preparation

Move the NVMF configuration to dedicated functions, they
are to be used in next patches.
Move rdma_req and cid initialization out of nvme_rdma_req_in

nvme/rdma: Factor our contig request preparation

Move the NVMF configuration to dedicated functions, they
are to be used in next patches.
Move rdma_req and cid initialization out of nvme_rdma_req_init,
that is needed in next patches to support accel sequence

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I9aca26d96c92d44b1b3f6542c3cf00fe9af9cc4b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24694
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <ben@nvidia.com>

show more ...

nvme/rdma: Allocate memory domain in rdma provider

Next patches add data_transfer function to a memory domain,
for mlx5_dv provider, that means we can't use a memory
domain created via rdma_utils. I

nvme/rdma: Allocate memory domain in rdma provider

Next patches add data_transfer function to a memory domain,
for mlx5_dv provider, that means we can't use a memory
domain created via rdma_utils. In future, memory domain
will hold a qpair pointer, to minize changes we create
a memory domain per qpair in this patch
The verbs provider still uses rdma_utils library.

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I53c20b70901c1061c8a067c612dc4ce6b9a3999a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24692
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <ben@nvidia.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>

show more ...

lib/nvme: eventfd to handle disconnected I/O qpair

In poll mode the process completion API checks for the disconnected
queue pair and calls the specified disconnected_qpair_cb function. In
interrupt

lib/nvme: eventfd to handle disconnected I/O qpair

In poll mode the process completion API checks for the disconnected
queue pair and calls the specified disconnected_qpair_cb function. In
interrupt mode we need a way to notify the application in case of
disconnected queue pair.
Register an eventfd to the epoll file descriptor of nvme poll group. This
fd will be written after a queue pair disconnection. Thus the next event
on poll group will handle the queue pair disconnection.

Change-Id: I481eeb847c22e710d8e9705c4ea3e9267da93876
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25387
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

nvme/poll_group: create and manage fd_group for nvme poll group

Create spdk_fd_group within spdk_nvme_poll_group, which manages interrupt
events for all the file descriptors of spdk_nvme_qpair that

nvme/poll_group: create and manage fd_group for nvme poll group

Create spdk_fd_group within spdk_nvme_poll_group, which manages interrupt
events for all the file descriptors of spdk_nvme_qpair that are part of
this poll group.

Two new APIs have been introduced to manage this fd_group

1). spdk_nvme_poll_group_get_fd()
Fetches the internal epoll file descriptor of the poll group.

2). spdk_nvme_poll_group_wait()
Collectively waits for interrupt events on all the I/O queue pair file
descriptors managed by the poll group.
When an interrupt event gets generated, it processes any outstanding
completions on the I/O queue pair with interrupts. These interrupt events
are registered at the the time of I/O queue pair creation.

The nvme_poll_group_connect_qpair() has been modified. Based on the poll
group interrupt support, this now registers an event source for the file
descriptor of queue pair to the internal epoll file descriptor of the
poll group.
Similarly, the nvme_poll_group_disconnect_qpair() unregisters the event
source for file descriptor of the queue pair from the internal epoll file
descriptor of the poll group.

Additional checks are in place to prevent mixing of interrupts enabled
and interrupts disabled I/O queue pairs. The poll group interrupt support
capability is set by the first I/O queue pair added to it.

Change-Id: If40f1ea82051ae598590f5a23ab9ed58bcb4af09
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25080
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

nvme: interface to check disconnected queue pairs

The following commit will enable handling of interrupt events on nvme
poll group. For interrupt mode currently there is no way to check for
disconne

nvme: interface to check disconnected queue pairs

The following commit will enable handling of interrupt events on nvme
poll group. For interrupt mode currently there is no way to check for
disconnected queue pairs at the transport level, whereas in poll mode
its done with every process completion call,

Change-Id: Ic87872d55f99e40a67c6e2e8861722b2cb256295
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25370
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>

show more ...

lib/nvme: add opts_size to spdk_nvme_io_qpair_opts

Add opts_size to spdk_nvme_io_qpair_opts to align it with other opts
structures. Cleanup spdk_nvme_ctrlr_get_default_io_qpair_opts() a bit.

Use nv

lib/nvme: add opts_size to spdk_nvme_io_qpair_opts

Add opts_size to spdk_nvme_io_qpair_opts to align it with other opts
structures. Cleanup spdk_nvme_ctrlr_get_default_io_qpair_opts() a bit.

Use nvme_ctrlr_io_qpair_opts_copy() instead of memcpy.

Change-Id: I6d2f7d16a2f4f6cfb68e3fe5ac0515050e8c36ee
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25246
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>

show more ...

nvme: interface to retrieve fd for a queue

Applications that enable interrupts for completion notification will
register and unregister queue pair file descriptor with event source.
These applicatio

nvme: interface to retrieve fd for a queue

Applications that enable interrupts for completion notification will
register and unregister queue pair file descriptor with event source.
These application also require other necessary details such as event
source and file descriptor type during interrupt registration phase.

Added APIs spdk_nvme_qpair_get_fd() and spdk_nvme_ctrlr_get_admin_qp_fd()
to retrieve the file descriptor of an I/O and the admin queue pair of a
given controller respectively. These APIs accept spdk_event_handler_opts
which if specified, allows the transport layer to fill out event source
and fd type.

Change-Id: I5c24ae15ffc01afea879d0219e8721728cce592e
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24906
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>

show more ...

nvme: enable interrupts for pcie nvme devices

Add an option enable_interrupts to the spdk_nvme_ctrlr_opts structure.
If this is set to true for pcie controllers interrupts may be enabled
during init

nvme: enable interrupts for pcie nvme devices

Add an option enable_interrupts to the spdk_nvme_ctrlr_opts structure.
If this is set to true for pcie controllers interrupts may be enabled
during initialization. Applications are required to check the resulting
value after the attach step to check for success.
Maximum of 256 eventfds can be reserved for I/O queues, but the actual
number can be lower and is based on the minimum requested I/O queues
and number of available I/O queues.
The nvme_pcie_ctrlr_cmd_create_io_cq() interface has been modified to
create I/O completion queues with interrupts. The interrupt vector field
corresponds to the queue identifier in this case.

This is only supported within a primary SPDK process, and if enabled
SPDK will not support any secondary processes.

Change-Id: Iff4e2348a0b77199cabb30c0bf93e0eed920cc93
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24905
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>

show more ...

nvme: Add transport interface to enable interrupts

The following commit will enable interrupts for pcie transport.
So add a new interface for the same.

Change-Id: I5cd87b0bb4ec95d6a9b862b659405cf56

nvme: Add transport interface to enable interrupts

The following commit will enable interrupts for pcie transport.
So add a new interface for the same.

Change-Id: I5cd87b0bb4ec95d6a9b862b659405cf56d8f864a
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24904
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Community-CI: Mellanox Build Bot

show more ...

nvme: Add spdk_nvme_ctrlr_get_id() for multipath identification

ctrlr->cntlid is easy to get. However, there was no public API to
return ctrlr->cntlid yet.

As the following comment in lib/nvme/nvme

nvme: Add spdk_nvme_ctrlr_get_id() for multipath identification

ctrlr->cntlid is easy to get. However, there was no public API to
return ctrlr->cntlid yet.

As the following comment in lib/nvme/nvme_ctrlr.c,

Fabrics controllers should already have CNTLID from the Connect command.

If CNTLID from Connect doesn't match CNTLID in the Identify Controller data,
trust the one from Connect.

ctrlr->cntlid is sufficiently reliable.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Iad7f1d8defcc427b7656e1b5f4f8104439bcfd51
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25297
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>

show more ...

nvme: Add multipath info (controller ID) to NVME_CTRLR_*LOG()

NVME_CTRLR_*LOG() is helpful but all controllers have the same SUBNQN
for fabric transports. We cannot distinguish which path caused an

nvme: Add multipath info (controller ID) to NVME_CTRLR_*LOG()

NVME_CTRLR_*LOG() is helpful but all controllers have the same SUBNQN
for fabric transports. We cannot distinguish which path caused an error.

IP address and port are nice identification information but too long.

For multipath, each controller should have unique ID. Hence, controller
ID is good practical information to add.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Id0739a4caccff93aba5fbdfa3b611c4bf7da3045
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25296
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...