module/raid: make struct raid_bdev_io_channel private to bdev_raid

Define getters for accessing only the necessary raid_ch data from the
raid modules.

Change-Id: I2622b0d219a3893ba3518b3c749695f6a1

module/raid: make struct raid_bdev_io_channel private to bdev_raid

Define getters for accessing only the necessary raid_ch data from the
raid modules.

Change-Id: I2622b0d219a3893ba3518b3c749695f6a1e57857
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18736
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

module/raid: put relevant bdev_io data in raid_bdev_io

Everything the raid modules need will be in struct raid_bdev_io, they
won't access spdk_bdev_io fields directly. This will simplify the raid
mo

module/raid: put relevant bdev_io data in raid_bdev_io

Everything the raid modules need will be in struct raid_bdev_io, they
won't access spdk_bdev_io fields directly. This will simplify the raid
modules' code and expose only what's needed. It will also allow easier
implementation of functionality like raid_io splitting (in the following
patches), which needs to alter the io in some way.

Change-Id: I26abf6d3397109a15251d279926ad28d7ea22105
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18734
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

module/raid: data offset and data size implementation

When raid bdev is created with superblock parameter then all data on
this bdev should be shifted by some offset. Such space at the beginning
of

module/raid: data offset and data size implementation

When raid bdev is created with superblock parameter then all data on
this bdev should be shifted by some offset. Such space at the beginning
of bdev will be used to store on-disk raid metadata.

Change-Id: I2545a2b00a651ef5332ca1757da0110a63914a43
Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15246
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

module/raid: add wrappers for base bdev io functions

These will be used by the next patch for implementing base bdev data
offset.

Change-Id: Ib225474672a6980b797ab21b2b5a118709406dd2
Signed-off-by:

module/raid: add wrappers for base bdev io functions

These will be used by the next patch for implementing base bdev data
offset.

Change-Id: Ib225474672a6980b797ab21b2b5a118709406dd2
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18151
Reviewed-by: Slawomir Ptak <slawomir.ptak@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

module/raid: remove bdev pointer from base_bdev_info

There is a spdk_bdev_desc pointer in base_bdev_info, which is always set
together with the bdev pointer, so use that instead. It makes checking
w

module/raid: remove bdev pointer from base_bdev_info

There is a spdk_bdev_desc pointer in base_bdev_info, which is always set
together with the bdev pointer, so use that instead. It makes checking
whether a base bdev is present more consistent. Before, in some places
the bdev pointer was checked and in others the desc pointer.

Change-Id: I5ac305b7d4b07c609b759a59f7f0fa540532ea1e
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18286
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

module/raid: specify memory domain support per raid module

Not all raid modules may support memory domains - raid5f currently does
not. Add a parameter to struct raid_bdev_module to specify that.

C

module/raid: specify memory domain support per raid module

Not all raid modules may support memory domains - raid5f currently does
not. Add a parameter to struct raid_bdev_module to specify that.

Change-Id: I3285c118db846d290837606b3f85ac4b5277de97
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17601
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

bdev: remove spdk_bdev_ext_io_opts from spdk_bdev_io

The spdk_bdev_ext_io_opts structure is used to pass extra options when
submitting a bdev IO request, without having to modify/add functions to
ha

bdev: remove spdk_bdev_ext_io_opts from spdk_bdev_io

The spdk_bdev_ext_io_opts structure is used to pass extra options when
submitting a bdev IO request, without having to modify/add functions to
handle new options. Additionally, the structure has a size field to
allow adding new fields without breaking the ABI (and thus having to
bump up the major version of a library).

It is also a part of spdk_bdev_io and there are several reasons for
removing it from that structure:

1. The size field only makes sense in structures that are passed
through pointers. And spdk_bdev_ext_io_opts is indeed passed as a
pointer to spdk_bdev_{readv,writev}_blocks_ext(), however it is
also embedded in spdk_bdev_io (internal.ext_opts_copy), which is
also part of the API. It means that each time a new field is added
to spdk_bdev_ext_io_opts, the size of spdk_bdev_io will also
change, so we will need to bump the major version of libspdk_bdev
anyway, thus making spdk_bdev_ext_io_opts.size useless.
2. The size field also makes internal.ext_opts cumbersome to use, as
each time one of its fields is accessed, we need to check the size.
Currently the code doesn't do that, because all of the existing
spdk_bdev_ext_io_opts fields were present when this structure was
initially introduced, but we'd need to do check the size before
accessing any new fields.
3. spdk_bdev_ext_io_opts has a metadata field, while spdk_bdev_io
already has u.bdev.md_buf, which means that we store the same thing
in several different places in spdk_bdev_io (u.bdev.md_buf,
u.bdev.ext_opts->metadata, internal.ext_opts->metadata).

Therefore, this patch removes all references to spdk_bdev_ext_io_opts
from spdk_bdev_io and replaces them with fields (memory_domain,
memory_domain_ctx) that were missing in spdk_bdev_io. Unfortunately,
this change breaks the API and requires changes in bdev modules that
supported spdk_bdev_io.u.bdev.ext_opts.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I49b7524eb84d1d4d7f12b7ab025fec36da1ee01f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16773
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

concat: io metadata support

Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Change-Id: I4909e8096dbfbeb8bc2f689ab694baf07c76af21
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/

concat: io metadata support

Signed-off-by: Krzysztof Smolinski <krzysztof.smolinski@intel.com>
Change-Id: I4909e8096dbfbeb8bc2f689ab694baf07c76af21
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15129
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>

show more ...

update Intel copyright notices

per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date defa

update Intel copyright notices

per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date default <file> | tail -1

and then pull just the 4 digit year from the result.

Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc). Contribution date used
"--follow -C95%" to get the most accurate date.

Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line. Simply there for
consistency.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id5b7ce4f658fe87132f14139ead58d6e285c04d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15192
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot

show more ...

module/raid: fix async module stopping

If the module stop handler is asynchronous we must wait until it
finishes before unregistering the io_device.

Change-Id: I149b716d9f4b0c1680b3e43b395fc9ec5b90

module/raid: fix async module stopping

If the module stop handler is asynchronous we must wait until it
finishes before unregistering the io_device.

Change-Id: I149b716d9f4b0c1680b3e43b395fc9ec5b90d70c
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14717
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

raid/concat: update bdev readv/writev to ext API.

Other parts have changed these since readv and writev
support ext_opts.
And change corresponding unittest.

Signed-off-by: yidong0635 <dongx.yi@inte

raid/concat: update bdev readv/writev to ext API.

Other parts have changed these since readv and writev
support ext_opts.
And change corresponding unittest.

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I79260a7e6110aa46df2016e579f1da5c241c9844
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13620
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>

show more ...

check_format: For C files only, fix return type breaks

In SPDK, declarations have the return type on the same line. Definitions
have the return type on a separate line. Astyle has an option for
enfo

check_format: For C files only, fix return type breaks

In SPDK, declarations have the return type on the same line. Definitions
have the return type on a separate line. Astyle has an option for
enforcing this. Unfortunately, it seems to have two bugs:

1) It doesn't work correctly at all on C++ files.
2) It often fails on functions that return enums, or long type names

Deal with 1) by adjusting the check_format.sh script to only tell astyle
to fix return type line breaks for C files and not C++. Deal with 2) by
adding a few typedefs to work around the problem.

Change-Id: Idf28281466cab8411ce252d5f02ab384166790c6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13437
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>

show more ...

Replace most BSD 3-clause license text with SPDX identifier.

Many open source projects have moved to using SPDX identifiers
to specify license information, reducing the amount of
boilerplate code in

Replace most BSD 3-clause license text with SPDX identifier.

Many open source projects have moved to using SPDX identifiers
to specify license information, reducing the amount of
boilerplate code in every source file. This patch replaces
the bulk of SPDK .c, .cpp and Makefiles with the BSD-3-Clause
identifier.

Almost all of these files share the exact same license text,
and this patch only modifies the files that contain the
most common license text. There can be slight variations
because the third clause contains company names - most say
"Intel Corporation", but there are instances for Nvidia,
Samsung, Eideticom and even "the copyright holder".

Used a bash script to automate replacement of the license text
with SPDX identifier which is checked into scripts/spdx.sh.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa88ab5e92ea471691dc298cfe41ebfb5d169780
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12904
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: <qun.wan@intel.com>

show more ...

bdev/raid: Add concat module

The concat module can combine multiple underlying bdevs to a single
bdev. It is a special raid level. You can add a new bdev to the end of
the concat bdev, then the conc

bdev/raid: Add concat module

The concat module can combine multiple underlying bdevs to a single
bdev. It is a special raid level. You can add a new bdev to the end of
the concat bdev, then the concat bdev size is increased, and it won't
change the layout of the exist data. This is the major difference
between concat and raid0. If you add a new underling device to raid0,
the whole data layout will be changed. So the concat bdev is extentable.

Change-Id: Ibbeeaf0606ff79b595320c597a5605ab9e4e13c4
Signed-off-by: Peng Yu <yupeng0921@gmail.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11070
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...