bdev/raid1: Support resize when increasing the size of base bdevs

Implement the resize function for RAID1. raid1_resize() calculate the
new raid_bdev's block count and if it is different from the ol

bdev/raid1: Support resize when increasing the size of base bdevs

Implement the resize function for RAID1. raid1_resize() calculate the
new raid_bdev's block count and if it is different from the old block
count, call spdk_bdev_notify_blockcnt_change() with the new block count.

Add a simple functional test for this feature. The test is to create
a raid1 bdev with two null bdevs, resize one null bdev, check if the
raid1 bdev is not resize, resize another null bdev, check if the raid1
bdev is resized.

The commit is an overall copy/adaptation of the work of Shuhei Matsumoto
with raid0.

Change-Id: I0737ba3fe4e1260d35d51e7b022c9cfd1eb6db58
Signed-off-by: Loïc Yavercovski <loic.yavercovski@gmail.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22619
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <ben@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

raid1: handle read errors

If a read fails, try reading from other base bdevs and if that succeeds,
write to the bdev that returned the read error.

Signed-off-by: Artur Paszkiewicz <artur.paszkiewic

raid1: handle read errors

If a read fails, try reading from other base bdevs and if that succeeds,
write to the bdev that returned the read error.

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Iffca8921d0177e16f1ac3d661b8b6131f3dbd39f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22974
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

raid1: handle write errors

Fail base bdev on write error. If write to at least one base bdev
succeeds, complete the raid I/O as successful.

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel

raid1: handle write errors

Fail base bdev on write error. If write to at least one base bdev
succeeds, complete the raid I/O as successful.

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I0315f98937a548648f819bf7e8c5707d7adec991
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22846
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

raid: move the raid I/O wrappers back to the header

Instead of having these in bdev_raid.c, just make
raid_bdev_remap_dix_reftag() public. This allows to have the unit tests
a little simpler and bet

raid: move the raid I/O wrappers back to the header

Instead of having these in bdev_raid.c, just make
raid_bdev_remap_dix_reftag() public. This allows to have the unit tests
a little simpler and better reflecting the real behavior.

This effectively reverts 2c9dfc37c ("module/raid: Move the RAID I/O
wrappers").

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I92de84d7cbc233f5d744aedf6e77d0728ecdaa72
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22828
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

ut/raid: make the common ut functions public

If a file including common.c doesn't use some of those functions it
would cause a "defined but not used" warning.

Signed-off-by: Artur Paszkiewicz <artu

ut/raid: make the common ut functions public

If a file including common.c doesn't use some of those functions it
would cause a "defined but not used" warning.

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: I1bdd1375babd1b400e7ba03de5a2b1a5f0e26f15
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22775
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

ut/raid: refactor setting test params

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Ib06297159ebcb354da5198b9b0c4eeef087cde1c
Reviewed-on: https://review.spdk.io/gerrit/c

ut/raid: refactor setting test params

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Change-Id: Ib06297159ebcb354da5198b9b0c4eeef087cde1c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21681
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot

show more ...

raid1: rebuild support

Change-Id: Ie700c90a787cbc1db1fc858e5d0caf1f6bbedaae
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18

raid1: rebuild support

Change-Id: Ie700c90a787cbc1db1fc858e5d0caf1f6bbedaae
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18740
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot

show more ...

module/raid: make struct raid_bdev_io_channel private to bdev_raid

Define getters for accessing only the necessary raid_ch data from the
raid modules.

Change-Id: I2622b0d219a3893ba3518b3c749695f6a1

module/raid: make struct raid_bdev_io_channel private to bdev_raid

Define getters for accessing only the necessary raid_ch data from the
raid modules.

Change-Id: I2622b0d219a3893ba3518b3c749695f6a1e57857
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18736
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

module/raid: put relevant bdev_io data in raid_bdev_io

Everything the raid modules need will be in struct raid_bdev_io, they
won't access spdk_bdev_io fields directly. This will simplify the raid
mo

module/raid: put relevant bdev_io data in raid_bdev_io

Everything the raid modules need will be in struct raid_bdev_io, they
won't access spdk_bdev_io fields directly. This will simplify the raid
modules' code and expose only what's needed. It will also allow easier
implementation of functionality like raid_io splitting (in the following
patches), which needs to alter the io in some way.

Change-Id: I26abf6d3397109a15251d279926ad28d7ea22105
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18734
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

ut/raid: factor out raid io channel create/destroy

Change-Id: If6259d9798ab5db64871047fce99a031a85a691e
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spd

ut/raid: factor out raid io channel create/destroy

Change-Id: If6259d9798ab5db64871047fce99a031a85a691e
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18712
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

module/raid: remove num_channels from raid_bdev_io_channel

This value is always equal to the number of base bdevs so just use that.

Change-Id: I6b6f2076c28b5f85f7ede212da048f594c54d6a9
Signed-off-b

module/raid: remove num_channels from raid_bdev_io_channel

This value is always equal to the number of base bdevs so just use that.

Change-Id: I6b6f2076c28b5f85f7ede212da048f594c54d6a9
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18711
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

test/raid1: unit tests for read balancing

Change-Id: Ic42bfe8be2d0d76e804a82b4837391384f01f8f8
Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Signed-off-by: Artur Paszkiewicz <ar

test/raid1: unit tests for read balancing

Change-Id: Ic42bfe8be2d0d76e804a82b4837391384f01f8f8
Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15272
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

raid1: read balancing

Allow using all base devices for reads instead of only the first one.
Keep track of the number of outstanding read blocks for every base bdev
and choose the least busy when han

raid1: read balancing

Allow using all base devices for reads instead of only the first one.
Keep track of the number of outstanding read blocks for every base bdev
and choose the least busy when handling a read I/O.

Here are some performance results. The tests were performed using fio on
2 and 3 drive (Intel P4510 2TB) arrays. Random reads, 1 thread, qd=128,
each run was 1 minute.

WITHOUT read balancing:
n=2 bs=4k : IOPS=804k, BW=3141MiB/s
n=3 bs=4k : IOPS=804k, BW=3140MiB/s
n=2 bsrange=4k-1M : IOPS=6421, BW=3223MiB/s
n=3 bsrange=4k-1M : IOPS=6416, BW=3222MiB/s
WITH read balancing:
n=2 bs=4k : IOPS=1291k, BW=5043MiB/s
n=3 bs=4k : IOPS=1572k, BW=6140MiB/s
n=2 bsrange=4k-1M : IOPS=12.2k, BW=6148MiB/s
n=3 bsrange=4k-1M : IOPS=18.0k, BW=9058MiB/s

Change-Id: I7d85411a6421bd7352031efb562ee95f2c612011
Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15271
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

test/unit: use spdk_ut_run_tests()

Replaced direct calls to the CUnit's functions to run the tests with
spdk_ut_run_tests(). That way, each test will have the ability to run
a specific test case.

test/unit: use spdk_ut_run_tests()

Replaced direct calls to the CUnit's functions to run the tests with
spdk_ut_run_tests(). That way, each test will have the ability to run
a specific test case.

The blob.c unit test wasn't changed, because it runs all tests multiple
times with different parameter combinations, so it cannot be easily
converted. In the future, each such combination could be split into a
separate test suite, which would make it compatible with
spdk_ut_run_tests().

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I4463f808f89844e9bf32b5b31eda197c5d729d1d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19288
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

test/unit: move spdk_cunit.h to include/spdk_internal

It'll make it easier to include this file outside of unit tests.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I171ddb864

test/unit: move spdk_cunit.h to include/spdk_internal

It'll make it easier to include this file outside of unit tests.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I171ddb8649f67b5786f08647560e2907603d0574
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19284
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>

show more ...

raid5f: io metadata support

Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Change-Id: I7760b2acc315970fbe1a99412dbd9a858ed17d0c
Signed-off-by: Artur Paszkiewicz <artur.paszkiewic

raid5f: io metadata support

Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Change-Id: I7760b2acc315970fbe1a99412dbd9a858ed17d0c
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15130
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

raid1: io metadata support

Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Change-Id: I53a57f85d94110365492d457715f8a8bc1194628
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/s

raid1: io metadata support

Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Change-Id: I53a57f85d94110365492d457715f8a8bc1194628
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15127
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>

show more ...

test/raid1: unit tests implementation

Implementation of basic unit tests for raid1 module.

Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Change-Id: Ide961b272cfd4997e51634dc6d2

test/raid1: unit tests implementation

Implementation of basic unit tests for raid1 module.

Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Change-Id: Ide961b272cfd4997e51634dc6d298d9ab52d358a
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14437
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...