blob: remove all references to pages as io_units

When blobstore was first created, everything was in terms of 4KB
pages - this included metadata page size and the unit for I/O
operations.

A bit lat

blob: remove all references to pages as io_units

When blobstore was first created, everything was in terms of 4KB
pages - this included metadata page size and the unit for I/O
operations.

A bit later, we introduced concept of "io_unit". If a blobstore
was put on a bdev with 512 blocksize, then the io_unit could be
512 bytes.

But when this happened, we should have changed all of the blobstore
code such that remaining "page" references only referred to metadata
pages. Instead, we left a bunch of places where we would convert
various values to/from a number of 4KB pages, and then to the number
of io_units. This made the code quite confusing, since direct
conversion to/from io_units would have been much clearer.

This existing problem was exacerbated with the upcoming patch to support
variable metadata page sizes. We need things like spdk_bs_get_page_size()
to return the size of the metadata pages, which may not be 4KB.

So make all of the changes necessary such that all references to
"page" now means "metadata page". This includes removing the
spdk_blob_get_num_pages() function, which no longer makes sense.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I66e93e7a4325a3b032bb16edaf657ef12044e1fd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25368
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <ben@nvidia.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

general: fix misspells and typos

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: Iab206ef526eb7032c6681a3145450010c91705a4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+

general: fix misspells and typos

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: Iab206ef526eb7032c6681a3145450010c91705a4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24120
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

lvol: add lvol set external parent

This function is mostly a wrapper around
spdk_bs_blob_set_external_parent to set the parent external snapshot
of a lvol.
It also perform some checks over input par

lvol: add lvol set external parent

This function is mostly a wrapper around
spdk_bs_blob_set_external_parent to set the parent external snapshot
of a lvol.
It also perform some checks over input parameters and create backing
device of the external snapshot.

Change-Id: Ia08d0a724271ff4ce48510f0e375a325e0c7dc2a
Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21690
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

lvol: add lvol set parent

This function is mostly a wrapper around spdk_bs_blob_set_parent
to set the parent snapshot of a lvol.
It also perform some checks over input parameters.

Change-Id: I0f709

lvol: add lvol set parent

This function is mostly a wrapper around spdk_bs_blob_set_parent
to set the parent snapshot of a lvol.
It also perform some checks over input parameters.

Change-Id: I0f709b6c772f02576f2f42b97e86ca0a82bb5dd7
Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19305
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

lvol: add shallow copy over a given device

This is mostly a wrapper around spdk_bs_blob_shallow_copy(), but it also
performs some checks over lvol, bs_dev and their compatibility.

Change-Id: I57ef2

lvol: add shallow copy over a given device

This is mostly a wrapper around spdk_bs_blob_shallow_copy(), but it also
performs some checks over lvol, bs_dev and their compatibility.

Change-Id: I57ef2d2dd7a298f6ac8749d11ffe4e59612eb22e
Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19249
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

bdev/lvol: use spdk_bs_grow_live() in grow lvs RPC

This patch adds spdk_lvs_grow_live() pipes through to
spdk_bs_grow_live() and use it when
bdev_lvol_grow_lvstore RPC is called.

Growing the size o

bdev/lvol: use spdk_bs_grow_live() in grow lvs RPC

This patch adds spdk_lvs_grow_live() pipes through to
spdk_bs_grow_live() and use it when
bdev_lvol_grow_lvstore RPC is called.

Growing the size of lvs still needs to be explicit
RPC call by the user.

Moved include for blob_bdev.h to header of the whole
bdev module. This way spdk_bdev_update_bs_blockcnt()
can be called when RPC is called.

Please note that there already exists a test with this
RPC - blobstore_grow.sh. Next patch in series
adds test calling grow while I/O is being performed.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I9e896c65a8b017d0d20b77d3758901e588e9e70c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/20322
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>

show more ...

lvol/ut: fix lvs_load ut with proper blob_id

Just a small miss in the UT.
It does come up later in the series when adding verification
of matching blob_id in lvol_free().

Change-Id: If54c80aaf24476

lvol/ut: fix lvs_load ut with proper blob_id

Just a small miss in the UT.
It does come up later in the series when adding verification
of matching blob_id in lvol_free().

Change-Id: If54c80aaf24476a4cd061d0127852451fa3566a8
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18686
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Ben Walker <ben@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lib/lvol: fix action_in_progress

action_in_progress right now determines if lvol is
in process of being closed or deleted.

It should be set only once blobstore functions are called,
and unset if th

lib/lvol: fix action_in_progress

action_in_progress right now determines if lvol is
in process of being closed or deleted.

It should be set only once blobstore functions are called,
and unset if they do encounter an error afterwards.
Failure to close can only occur during close, then the lvol
remains. For destroy it is freed anyway.

This patch moves setting it to true bit later on close/destroy,
and joins success/error paths by setting in to false in both.

While here remove lvol_free() that should not be freed
in the close path, if blob close failed.
This allows to add relevant UT.

Part of work on #3006.

Change-Id: I9dc8b8defb6090b9e40b3520fbdccbd071622ba6
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18684
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <ben@nvidia.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lvol/ut: add negative test cases for lvol_close

Change-Id: I205a15888e8fb98269d3784f517140460b0c86df
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/ge

lvol/ut: add negative test cases for lvol_close

Change-Id: I205a15888e8fb98269d3784f517140460b0c86df
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18683
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <ben@nvidia.com>

show more ...

lvol/ut: reuse ut_cb_res in lvol_close UT

Since the inception of the lvol_close a better way to
pass the callback results was added in form of ut_cb_res.

Re-using this here will make future changes

lvol/ut: reuse ut_cb_res in lvol_close UT

Since the inception of the lvol_close a better way to
pass the callback results was added in form of ut_cb_res.

Re-using this here will make future changes simpler.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I15952ed1c1c0c570e516ba6f03624cefb1481789
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18682
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <ben@nvidia.com>

show more ...

lvol/ut: remove lvol_close_fail UT

lvol_close_success/fail were doing the exact same thing.
So just remove it, and later patches in the series adds
error path test cases.

Change-Id: I0e6973bd31eaad

lvol/ut: remove lvol_close_fail UT

lvol_close_success/fail were doing the exact same thing.
So just remove it, and later patches in the series adds
error path test cases.

Change-Id: I0e6973bd31eaadd4deaaddd2511ed1ecda33108d
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18681
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@solidigm.com>
Reviewed-by: Ben Walker <ben@nvidia.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

test/unit: use spdk_ut_run_tests()

Replaced direct calls to the CUnit's functions to run the tests with
spdk_ut_run_tests(). That way, each test will have the ability to run
a specific test case.

test/unit: use spdk_ut_run_tests()

Replaced direct calls to the CUnit's functions to run the tests with
spdk_ut_run_tests(). That way, each test will have the ability to run
a specific test case.

The blob.c unit test wasn't changed, because it runs all tests multiple
times with different parameter combinations, so it cannot be easily
converted. In the future, each such combination could be split into a
separate test suite, which would make it compatible with
spdk_ut_run_tests().

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I4463f808f89844e9bf32b5b31eda197c5d729d1d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19288
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

test/unit: move spdk_cunit.h to include/spdk_internal

It'll make it easier to include this file outside of unit tests.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I171ddb864

test/unit: move spdk_cunit.h to include/spdk_internal

It'll make it easier to include this file outside of unit tests.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I171ddb8649f67b5786f08647560e2907603d0574
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19284
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>

show more ...

lvol: add spdk_lvol_is_degraded

This is mostly a wrapper around spdk_blob_is_degraded(), but it also
performs a NULL check on lvol->blob. Since an lvol without a blob cannot
perform IO, this conditi

lvol: add spdk_lvol_is_degraded

This is mostly a wrapper around spdk_blob_is_degraded(), but it also
performs a NULL check on lvol->blob. Since an lvol without a blob cannot
perform IO, this condition returns true.

The two callers of spdk_blob_is_degraded() in vbdev_lvol.c have been
updated to use spdk_lvol_is_degraded().

Change-Id: I11dc682a26d971c8854aeab280c8199fced358c3
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17896
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

lvol: esnap clones must end on cluster boundary

When regular lvols are created, their size is rounded up to the next
cluster boundary. This is not acceptable for esnap clones as this means
that the

lvol: esnap clones must end on cluster boundary

When regular lvols are created, their size is rounded up to the next
cluster boundary. This is not acceptable for esnap clones as this means
that the clone may be silently grown larger than external snapshot. This
can cause a variety of problems for the consumer of an esnap clone lvol.

While the better long-term solution is to allow lvol sizes to fall on
any block boundary, the implementation of that needs to be suprisingly
complex to support creation and deletion of snapshots and clones of
esnap clones, inflation, and backward compatibility.

For now, it is best to put in a restriction on the esnap clone size
during creation so as to not hit problems long after creation. Since
lvols are generally expected to be large relative to the cluster size,
it is somewhat unlikely that this restriction will be a significant
limitation.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Id7a628f852a40c8ec2b7146504183943d723deba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17607
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

vbdev_lvol: load esnaps via examine_config

This introduces an examine_config callback that triggers hotplug of
missing esnap devices.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I5ce

vbdev_lvol: load esnaps via examine_config

This introduces an examine_config callback that triggers hotplug of
missing esnap devices.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I5ced2ff26bfd393d2df4fd4718700be30eb48063
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16626
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lvol: add spdk_lvol_get_by_* API

spdk_lvol_get_by_uuid() allows lookup of lvols by the lvol's uuid.

spdk_lvol_get_by_names() allows lookup of lvols by the lvol's lvstore
name and lvol name.

Signed

lvol: add spdk_lvol_get_by_* API

spdk_lvol_get_by_uuid() allows lookup of lvols by the lvol's uuid.

spdk_lvol_get_by_names() allows lookup of lvols by the lvol's lvstore
name and lvol name.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Id165a3d17b76e5dde0616091dee5dff8327f44d0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17546
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lvol: add spdk_lvol_iter_immediate_clones()

Add an interator that calls a callback for each clone of a snapshot
volume. This follows the typical pattern of stopping iteration when the
callback retur

lvol: add spdk_lvol_iter_immediate_clones()

Add an interator that calls a callback for each clone of a snapshot
volume. This follows the typical pattern of stopping iteration when the
callback returns non-zero.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: If88ad769b72a19ba0993303e89da107db8a6adfc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17545
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lvol_ut: test esnap hotplug

This exercises spdk_lvs_esnap_notify_hotplug() under a variety of happy
and not-so-happy paths.

Change-Id: I1f4101a082b113dacc7d03f81ca16069acfb458d
Signed-off-by: Mike

lvol_ut: test esnap hotplug

This exercises spdk_lvs_esnap_notify_hotplug() under a variety of happy
and not-so-happy paths.

Change-Id: I1f4101a082b113dacc7d03f81ca16069acfb458d
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17602
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

lvol: hotplug of missing esnaps

This introduces spdk_lvs_notify_hotplug() to trigger the lvstore to call
the appropriate lvstore's esnap_bs_dev_create() callback for each esnap
clone lvol that is mi

lvol: hotplug of missing esnaps

This introduces spdk_lvs_notify_hotplug() to trigger the lvstore to call
the appropriate lvstore's esnap_bs_dev_create() callback for each esnap
clone lvol that is missing the device identified by esnap_id.

Change-Id: I0e2eb26375c62043b0f895197b24d6e056905aa2
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16428
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

lvol: keep track of missing external snapshots

If an lvol is opened in degraded mode, keep track of the missing esnap
IDs and which lvols need them. A future commit will make use of this
information

lvol: keep track of missing external snapshots

If an lvol is opened in degraded mode, keep track of the missing esnap
IDs and which lvols need them. A future commit will make use of this
information to bring lvols out of degraded mode when their external
snapshot device appears.

Change-Id: I55c16ad042a73e46e225369bfff2631958a2ed46
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16427
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lvol_ut: add test for invalid options

Add unit test for calling spdk_lvs_load_ext()/lvs_load()
with invalid options (opts_size is 0).

Change-Id: I9c48b972066cf977304e3efa936827d1ef1b5250
Signed-off

lvol_ut: add test for invalid options

Add unit test for calling spdk_lvs_load_ext()/lvs_load()
with invalid options (opts_size is 0).

Change-Id: I9c48b972066cf977304e3efa936827d1ef1b5250
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17584
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lvol: do not open esnaps during bs_load

As an lvstore is opening it calls spdk_bs_load(), which briefly opens
each blob and has no use for external snapshots. Since there is no point
in opening them

lvol: do not open esnaps during bs_load

As an lvstore is opening it calls spdk_bs_load(), which briefly opens
each blob and has no use for external snapshots. Since there is no point
in opening them at this time, don't open them. Once the blobstore has
been loaded, update lvs->load_esnaps so that external snapshots are
opened as the lvols open their blobs.

Change-Id: Ib16c8474300ff4b106aad0baa5b8b38332c23b01
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16424
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

lvol: add support for external snapshots

This provides the lib/lvol wrapper around blobstore's external
snapshots. Later commits make this work with vbdev_lvol.

The blobstore external snapshot impl

lvol: add support for external snapshots

This provides the lib/lvol wrapper around blobstore's external
snapshots. Later commits make this work with vbdev_lvol.

The blobstore external snapshot implementation stores an opaque
identifier in an internal xattr. Lvstore uses this to store the
stringified UUID of the bdev that will act as the external snapshot.
This is used by the newly introduced spdk_lvol_create_esnap_clone() to
store the bdev UUID in the blob's metadata.

Change-Id: I58c7b32b656ad1d21a446e3b91e59e655efac7e4
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14977
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot

show more ...