bdev/nvme: Fail reset sequence immediately if ctrlr is already removed.

After a controller was hot-removed, if a reset sequence started to
the controller, spdk_nvme_ctrlr_disconnect() failed and cau

bdev/nvme: Fail reset sequence immediately if ctrlr is already removed.

After a controller was hot-removed, if a reset sequence started to
the controller, spdk_nvme_ctrlr_disconnect() failed and caused core
dump in debug mode.

When implemented, how to cause the failure and how to process the
failure were not clear. Hence assert was added to detect the
failure.

We know how we cause the failure now. Let's handle the failure
appropriately.

If spdk_nvme_ctrlr_disconnect() fails, we are on the nvme_ctrlr->thread.
Hence call bdev_nvme_reset_complete() with failure immediately.

Even if spdk_nvme_ctrlr_disconnect() completes synchronously, the
completion callback is executed asynchronously when polling an adminq.

Hence set the completion callback only if spdk_nvme_ctrlr_disconnect()
succeeds.

Fixes issue #2632

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I11f61853aba9eca2515592f964a291e59def7247
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13892
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

bdev/nvme: Rename check_multipath_params by check_io_error_resiliency_params

These checked parameters are necessary themselves even for single path
configuration.

Signed-off-by: Shuhei Matsumoto <s

bdev/nvme: Rename check_multipath_params by check_io_error_resiliency_params

These checked parameters are necessary themselves even for single path
configuration.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ie1eb2f51eeec1dbc634c6bae462a41d4c209d6ac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12052
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <yidong0635@126.com>
Community-CI: Mellanox Build Bot

show more ...

accel: Do not refer to the "framework" as "engine"

The word engine was both used (interchangeably with module) to refer to
the things that plug into the framework and to the framework itself.
This p

accel: Do not refer to the "framework" as "engine"

The word engine was both used (interchangeably with module) to refer to
the things that plug into the framework and to the framework itself.
This patch eliminates all use of the word engine that meant the
framework. It leaves uses of the word that meant "module".

Change-Id: I6b9b50e2f045ac39f2a74d0152ee8d6269be4bd1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13918
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>

show more ...

bdev/nvme: Set multipath policy correctly when creating nvme_bdev_channel

bdev_nvme_create_bdev_channel_cb() did not initialized the multipath
policy of the newly created channel. 0 was active-passi

bdev/nvme: Set multipath policy correctly when creating nvme_bdev_channel

bdev_nvme_create_bdev_channel_cb() did not initialized the multipath
policy of the newly created channel. 0 was active-passive and hence
multipath policy was always initialized to active-passive.

Fix the bug and add unit tests for verification.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I6e44108740da4b9ff72311ae4b5500558c65c5c3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14225
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>

show more ...

bdev/nvme: Remove admin passthrough retry and failover

Admin passthrough supported retry and failover as same as I/O by
using the bdev_retry_count. However, doing retry or failover for
admin passthr

bdev/nvme: Remove admin passthrough retry and failover

Admin passthrough supported retry and failover as same as I/O by
using the bdev_retry_count. However, doing retry or failover for
admin passthrough may have unexpected side effects and its value
is not clear. The safest way is to limit retry and failover for I/O.
If we need to support retry and failover for admin passthrough,
restore the code and add a new option bdev_admin_retry_count.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I680513a40a80041f6ea6f546c74c672f2a81812d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14227
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>

show more ...

check_format: For C files only, fix return type breaks

In SPDK, declarations have the return type on the same line. Definitions
have the return type on a separate line. Astyle has an option for
enfo

check_format: For C files only, fix return type breaks

In SPDK, declarations have the return type on the same line. Definitions
have the return type on a separate line. Astyle has an option for
enforcing this. Unfortunately, it seems to have two bugs:

1) It doesn't work correctly at all on C++ files.
2) It often fails on functions that return enums, or long type names

Deal with 1) by adjusting the check_format.sh script to only tell astyle
to fix return type line breaks for C files and not C++. Deal with 2) by
adding a few typedefs to work around the problem.

Change-Id: Idf28281466cab8411ce252d5f02ab384166790c6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13437
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>

show more ...

bdev/nvme: replace nn with mnan in ana_log size calculation

Calculation of the ANA log page size should use the
identify ctrl MNAN field
(maximum number of allowed namespaces)
not the NN (maximum va

bdev/nvme: replace nn with mnan in ana_log size calculation

Calculation of the ANA log page size should use the
identify ctrl MNAN field
(maximum number of allowed namespaces)
not the NN (maximum valid nsid value).

An ANA-enabled controller must have a non-zero MNAN value,
see NVMe Base Specification, Figure 251,
therefore nvme_ctrlr_init_ana_log_page() may safely use MNAN.

Since NN might be much higher than MNAN,
ANA log size based on NN may results in a very large
log page and cause a failure to get ANA log,
e.g. if it is larger than the controller's MDTS.

Fix: replace cdata->nn with cdata->mnan
in nvme_ctrlr_init_ana_log_page()

Signed-off-by: Anton Eidelman <anton@lightbitslabs.com>
Change-Id: I2a522dca833a27dddad25848d7688efa23d23091
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13039
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>

show more ...

Replace most BSD 3-clause license text with SPDX identifier.

Many open source projects have moved to using SPDX identifiers
to specify license information, reducing the amount of
boilerplate code in

Replace most BSD 3-clause license text with SPDX identifier.

Many open source projects have moved to using SPDX identifiers
to specify license information, reducing the amount of
boilerplate code in every source file. This patch replaces
the bulk of SPDK .c, .cpp and Makefiles with the BSD-3-Clause
identifier.

Almost all of these files share the exact same license text,
and this patch only modifies the files that contain the
most common license text. There can be slight variations
because the third clause contains company names - most say
"Intel Corporation", but there are instances for Nvidia,
Samsung, Eideticom and even "the copyright holder".

Used a bash script to automate replacement of the license text
with SPDX identifier which is checked into scripts/spdx.sh.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa88ab5e92ea471691dc298cfe41ebfb5d169780
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12904
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: <qun.wan@intel.com>

show more ...

bdev/nvme: Poll adminq more often during ctrlr disconnection

During ctrlr reconnection, spdk_nvme_ctrlr_reconnect_poll_async()
is executed by a non-timed poller.

We should poll adminq more often du

bdev/nvme: Poll adminq more often during ctrlr disconnection

During ctrlr reconnection, spdk_nvme_ctrlr_reconnect_poll_async()
is executed by a non-timed poller.

We should poll adminq more often during ctrlr disconnection too.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ib1f5b41015aed20deda8df6f2c837981ac233c04
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12615
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

bdev/nvme: Reversed orderings for reset between PCIe and NVMe-oF

As described in the NVMe specification, a controller level reset
includes the following actions:
- the controller stops processing an

bdev/nvme: Reversed orderings for reset between PCIe and NVMe-oF

As described in the NVMe specification, a controller level reset
includes the following actions:
- the controller stops processing any outstanding admin or I/O
commands;
- all I/O SQs and CQs are deleted.

In a full controller reset sequence for a PCIe controller, if we do
a controller level reset first, we can abort outstanding commands
after the hardware has actually been stopped.

For NVMe-oF controller, each I/O qpair is an independent network
connection and is disconnected safely. We do not want to change
NVMe-oF controller.

Fixes the issue #2360

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: If05febac74705bfd3df5abd15064c1203126e027
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12447
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

bdev/nvme: Disable automatic failback in multipath mode

By default, failback to the preferred I/O path is done automatically
if it is restored. Some users may want to keep using the backup I/O
path

bdev/nvme: Disable automatic failback in multipath mode

By default, failback to the preferred I/O path is done automatically
if it is restored. Some users may want to keep using the backup I/O
path even if the preferred I/O path is restored. In this case,
bdev_nvme_set_preferred_path can be used to do manual failback.

We may be able to clear/fill I/O path cache more strictly but it will
be complicated and have bugs. This patch does the minimal change,
just skips an apparent case.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I78fe5faee6ff04e88ae3d7c6be6da1c20637c912
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12431
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

bdev/nvme: Add active/active policy for multipath mode

The NVMe bdev module supported active-passive policy for multipath mode
first. By this patch, the NVMe bdev module supports active-active polic

bdev/nvme: Add active/active policy for multipath mode

The NVMe bdev module supported active-passive policy for multipath mode
first. By this patch, the NVMe bdev module supports active-active policy
for multipath node next. Following the Linux kernel native NVMe multipath,
the NVMe bdev module supports round robin algorithm for active-active
policy.

The multipath policy, active-passive or active-active, is managed per
nvme_bdev. The multipath policy is copied to all corresponding
nvme_bdev_channels.

Different from active-passive, active-active caches even non_optimized
path to provide load balance across multiple paths.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ie18b24db60d3da1ce2f83725b6cd3079f628f95b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12001
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>

show more ...

bdev/nvme: Set preferred I/O path in multipath mode

If we specify a preferred path manually for each NVMe bdev, we will
be able to realize a simple static load balancing and make the failover
more c

bdev/nvme: Set preferred I/O path in multipath mode

If we specify a preferred path manually for each NVMe bdev, we will
be able to realize a simple static load balancing and make the failover
more controllable in the multipath mode.

The idea is to move I/O path to the NVMe-oF controller to the head of
the list and then clear the I/O path cache for each NVMe bdev channel.
We can set the I/O path to the I/O path cache directly but it must be
conditional and make the code very complex. Hence, let find_io_path() do
that.

However, a NVMe bdev channel may be acquired after setting the preferred
path. To cover such case, sort the nvme_ns list of the NVMe bdev too.

This feature supports only multipath mode. The NVMe bdev module supports
failover mode too. However, to support the latter, the new RPC needs to
have trid as parameters and the code and the usage will be come very
complex. Add a note for such limitation.

To verify one by one exactly, add unit test.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ia51c74f530d6d7dc1f73d5b65f854967363e76b0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12262
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: <tanl12@chinatelecom.cn>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

bdev/nvme: Fix aborting fuse commands

When sending a fused compare and write command, we pass a callback
bdev_nvme_comparev_and_writev_done that we expect to be called twice
before marking the io as

bdev/nvme: Fix aborting fuse commands

When sending a fused compare and write command, we pass a callback
bdev_nvme_comparev_and_writev_done that we expect to be called twice
before marking the io as completed. In order to detect if a call to
bdev_nvme_comparev_and_writev_done is the first or the second one, we
currently rely on the opcode in cdw0. However, cdw0 may be set to 0,
especially when aborting the command. This may cause use-after-free
issues and this may call the user callbacks twice instead of once.
Use a bit in the nvme_bdev_io instead to keep track of the number of
calls to bdev_nvme_comparev_and_writev_done.

Signed-off-by: Alex Michon <amichon@kalrayinc.com>
Change-Id: I0474329e87648e44b08998d0552b2a9dd5d34ac2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12180
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

bdev/nvme: Factor out ctrlr info json dump into a helper function

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I7f1e08ff13d890cb780e7b66c18a77ab85c82029
Reviewed-on: https://re

bdev/nvme: Factor out ctrlr info json dump into a helper function

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I7f1e08ff13d890cb780e7b66c18a77ab85c82029
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12311
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

bdev/nvme: Handle ANA transition (change or inaccessible state) correctly

Previously, if a namespace is in ANA inaccessible state, I/O had been
queued infinitely. Fix this issue according to the NVM

bdev/nvme: Handle ANA transition (change or inaccessible state) correctly

Previously, if a namespace is in ANA inaccessible state, I/O had been
queued infinitely. Fix this issue according to the NVMe spec.

Add a temporary poller anatt_timer and a flag ana_transition_timedout for
each nvme_ns.

Start anatt_timer if the nvme_ns enters ANA transition. If anatt_timer
is expired, set ana_transition_timedout to true. Cancel anatt_timer or
clear ana_transition_timedout if the nvme_ns exits ANA transition.

nvme_io_path_become_available() returns false if ana_transition_timedout
is true.

Add unit test case to verify these addition.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ic76933242046b3e8e553de88221b943ad097c91c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12194
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>

show more ...

test/bdev: In bdev_nvme_ut, handle spdk_nvme_poll_group_remove when
there is no group

The real implementation handles this by returning -ENOENT, so do the
same in the test.

Signed-off-by: Ben Walke

test/bdev: In bdev_nvme_ut, handle spdk_nvme_poll_group_remove when
there is no group

The real implementation handles this by returning -ENOENT, so do the
same in the test.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I405b6f60bf4dcdb22c57e48bbaf66d57522a49c5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11508
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>

show more ...

test/bdev: In bdev_nvme_ut, count a disconnect as "activity"

Count disconnecting a queue pair as activity so that the unit test
poll_threads() calls don't bail out until the disconnectedd_qpair_cb i

test/bdev: In bdev_nvme_ut, count a disconnect as "activity"

Count disconnecting a queue pair as activity so that the unit test
poll_threads() calls don't bail out until the disconnectedd_qpair_cb is
called at least once.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Idc437d6c589dbf133bfcbb5edba1087f928a718c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11507
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>

show more ...

bdev/nvme: Remove ctrlr from nvme_ctrlr_channel

This was neither set nor used.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I3119135843c5fc0b8724e593db40df46e6b5bdb0
Reviewed-on

bdev/nvme: Remove ctrlr from nvme_ctrlr_channel

This was neither set nor used.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I3119135843c5fc0b8724e593db40df46e6b5bdb0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12097
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

bdev: Add ext_opts in public bdev_io section

Bdev modules must not access internal bdev_io
structure, so add a new pointer in a public
section. Pointer in internal section will be
used in next patch

bdev: Add ext_opts in public bdev_io section

Bdev modules must not access internal bdev_io
structure, so add a new pointer in a public
section. Pointer in internal section will be
used in next patch

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Ib631563015b3e5fa9300d22b7ae59d8db43c8275
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10421
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

bdev/nvme: Check not only I/O qpair but also adminq when finding optimal I/O path

For RDMA transport, adminq will find transport error first because
usually only adminq polls CM events.

Change-Id:

bdev/nvme: Check not only I/O qpair but also adminq when finding optimal I/O path

For RDMA transport, adminq will find transport error first because
usually only adminq polls CM events.

Change-Id: I7b22cc8883bf02198f1a90d2654c1de6f2e736e6
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11331
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

bdev/nvme: Fail fast I/O qpair if poll_group_process_completions() returns negated errno

If qpair is disconnected asynchronously, it takes time from detecting
transport error to actually disconnecte

bdev/nvme: Fail fast I/O qpair if poll_group_process_completions() returns negated errno

If qpair is disconnected asynchronously, it takes time from detecting
transport error to actually disconnected. We should avoid using the
path as soon as possible after detecting any transport error.

Poll group clears I/O path cache if it finds transport error and avoid
using the path which had transport error.

These changes will reduce the failover time.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I00580159a84372a115ed5e62a6ce13eed4368999
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11329
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

bdev/nvme: Reconnect ctrlr after it is disconnected at completion poller

spdk_nvme_ctrlr_disconnect() will be made asynchronous in the
following patches and so we will need to have some changes.

sp

bdev/nvme: Reconnect ctrlr after it is disconnected at completion poller

spdk_nvme_ctrlr_disconnect() will be made asynchronous in the
following patches and so we will need to have some changes.

spdk_nvme_ctrlr_disconnect() disconnects adminq and ctrlr synchronously
now.

If spdk_nvme_ctrlr_disconnect() is made asynchronous,
spdk_nvme_ctrlr_process_admin_completions() will complete to disconnect
adminq and ctrlr, and will return -ENXIO only if adminq is disconnected.

However even now spdk_nvme_ctrlr_process_admin_completions() returns
-ENXIO if adminq is disconnected.

So as a preparation, set a callback before calling spdk_nvme_ctrlr_disconnect()
and call the callback if it is set and spdk_nvme_ctrlr_process_admin_completions()
returns -ENXIO.

Besides, fix the return value of bdev_nvme_poll_adminq() in this patch.

Change-Id: I2559f86bb8cf9a92b5b386ed816c00b08c9832df
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10950
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

bdev/nvme: Disconnect and then free I/O qpair in a ctrlr reset sequence

As we do when deleting ctrlr_channel, disconnect and then free I/O
qpair in a ctrlr reset sequence.

Deleting ctrlr_channel an

bdev/nvme: Disconnect and then free I/O qpair in a ctrlr reset sequence

As we do when deleting ctrlr_channel, disconnect and then free I/O
qpair in a ctrlr reset sequence.

Deleting ctrlr_channel and resetting ctrlr_channel may cause conflicts.
This patch processes such conflicts correctly.

If destroy_ctrlr_channel_cb() is executed between pending and executing
reset_destroy_qpair(), reset_destroy_qpair() is not executed because
ctrlr_channel is not found. In this case, destroy_qpair_channel()
starts disconnecting qpair and deletes ctrlr_channel. Then
disconnected_qpair_cb() releases a reference to poll group.

If destroy_ctrlr_channel_cb() is excuted between executing reset_destroy_qpair()
and disconnected_qpair_cb(), destroy_ctrlr_channel_cb() skips
ctrlr_channel for a reset sequence.

Change-Id: I1f49f74b94aefbea178680aa53ded3a12876c676
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10766
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

nvme: Free I/O qpair now even if it is in poll group completion

spdk_nvme_poll_group has followed spdk_nvme_qpair about how to
process I/O qpair deletion inside of a completion context.

spdk_nvme_q

nvme: Free I/O qpair now even if it is in poll group completion

spdk_nvme_poll_group has followed spdk_nvme_qpair about how to
process I/O qpair deletion inside of a completion context.

spdk_nvme_qpair_process_completions() accesses qpair after
returning from nvme_transport_qpair_process_completions().

So this is reasonable.

On the other hand, if spdk_nvme_poll_group_process_completions()
can execute spdk_nvme_ctrlr_free_io_qpair() inside of a completion
context, the target qpair is ensured to be deleted after returning
from spdk_nvme_ctrlr_free_io_qpair(). Then the target qpair is
not accessed anymore in spdk_nvme_poll_group_process_completions().

Remove two variables, in_completion_context and num_qpairs_to_delete,
of spdk_nvme_transport_poll_group and the related code.

This change is really necessary to support the following case.

In the NVMe bdev module, a nvme_qpair has a qpair and a poll_group
channel. disconnected_qpair_cb calls spdk_nvme_ctrlr_free_io_qpair()
for the qpair and spdk_put_io_channel() to the poll_group_channel.
spdk_nvme_ctrlr_free_io_qpair() is executed after unwinding stack
but spdk_put_io_channel() is executed now. The callback to
spdk_put_io_channel() calls spdk_nvme_poll_group_destroy(). However,
spdk_nvme_ctrlr_free_io_qpair() is not executed. Hence
spdk_nvme_poll_group_destroy() fails.

Update the corresponding stub in unit test together.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Icd1f1daf049c6c7ffb28790fe87989a1060f8952
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11496
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...