blob: remove all references to pages as io_units

When blobstore was first created, everything was in terms of 4KB
pages - this included metadata page size and the unit for I/O
operations.

A bit lat

blob: remove all references to pages as io_units

When blobstore was first created, everything was in terms of 4KB
pages - this included metadata page size and the unit for I/O
operations.

A bit later, we introduced concept of "io_unit". If a blobstore
was put on a bdev with 512 blocksize, then the io_unit could be
512 bytes.

But when this happened, we should have changed all of the blobstore
code such that remaining "page" references only referred to metadata
pages. Instead, we left a bunch of places where we would convert
various values to/from a number of 4KB pages, and then to the number
of io_units. This made the code quite confusing, since direct
conversion to/from io_units would have been much clearer.

This existing problem was exacerbated with the upcoming patch to support
variable metadata page sizes. We need things like spdk_bs_get_page_size()
to return the size of the metadata pages, which may not be 4KB.

So make all of the changes necessary such that all references to
"page" now means "metadata page". This includes removing the
spdk_blob_get_num_pages() function, which no longer makes sense.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I66e93e7a4325a3b032bb16edaf657ef12044e1fd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25368
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <ben@nvidia.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

blob: add blob set external parent

Implemented a new function to set an external snapshot as the parent
of a blob. The previous parent of the blob, if any, can be another
external snapshot or a snap

blob: add blob set external parent

Implemented a new function to set an external snapshot as the parent
of a blob. The previous parent of the blob, if any, can be another
external snapshot or a snapshot; if the blob is not a clone, it must
be thin-provisioned.

Change-Id: Ib9eda0fba04428d058109a1eacdec084ef050a65
Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21689
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

blob: add blob set parent

Implemented a new function to set a snapshot as the parent of a blob.
The previous parent of the blob, if any, can be another snapshot or
an external snapshot; if the blob

blob: add blob set parent

Implemented a new function to set a snapshot as the parent of a blob.
The previous parent of the blob, if any, can be another snapshot or
an external snapshot; if the blob is not a clone, it must be
thin-provisioned.

Change-Id: I00eeefd352f437c587433d56f1988e7a607d2bf8
Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19304
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot

show more ...

blob: keep count of allocated clusters number

A new variable has been added in spdk_blob_mut_data to store the
number of allocated clusters in clusters array. Also a new method
to get this value has

blob: keep count of allocated clusters number

A new variable has been added in spdk_blob_mut_data to store the
number of allocated clusters in clusters array. Also a new method
to get this value has been added.

Change-Id: Ibac9344bcf7e4c5e6e12cf78a6eae0a4d6755acb
Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19712
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot

show more ...

blob: add shallow copy over a given device

A shallow copy will copy over the destination device only the
cluster allocated to the blob discarding those belonging
to the blob's parent snapshot, if an

blob: add shallow copy over a given device

A shallow copy will copy over the destination device only the
cluster allocated to the blob discarding those belonging
to the blob's parent snapshot, if any.

Change-Id: I763ba9d952b74bce2d5827abe1fc3f41b8ebd209
Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19247
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

lib/blob: add functionality to grow bs in run-time

spdk_bs_grow() requires unloading the blobstore,
therefore it is basically combining bs load and init
in a way that allows to change the size of bl

lib/blob: add functionality to grow bs in run-time

spdk_bs_grow() requires unloading the blobstore,
therefore it is basically combining bs load and init
in a way that allows to change the size of blobstore.

This provides a clean way to make sure that recreation
of bs_dev (with new block count) results in proper
updates in on-disk metadata and run-time.

It has the obvious drawback of closing all blobs.

This patch introduced spdk_bs_grow_live() that
allows for run-time change of blobstore size.

It has the following caveats:
- bs_dev size has to be changed before calling it
- relies on clean unload for updates of used clusters map

When bs is not unloaded, then recovery path will handle
increasing the size of used clusters map.

Change-Id: I81f8d1ce5652601a7b6109379e319c1231b8415d
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19454
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <ben@nvidia.com>

show more ...

blob: add spdk_blob_is_degraded()

In preparation for supporting degraded lvols, spdk_blob_is_degraded() is
added. To support this, bs_dev gains an optional is_degraded() callback.
spdk_blob_is_degra

blob: add spdk_blob_is_degraded()

In preparation for supporting degraded lvols, spdk_blob_is_degraded() is
added. To support this, bs_dev gains an optional is_degraded() callback.
spdk_blob_is_degraded() returns false so long as no bs_dev that the blob
depends on is degraded. Depended upon bs_devs include the blobstore's
device and the blob's back_bs_dev.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ib02227f5735b00038ed30923813e1d5b57deb1ab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17516
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blob: add spdk_blob_get_esnap_bs_dev()

While getting memory domains, vbdev_lvol will need to be able to access
the bdev that acts as the lvol's external snapshot. The introduction of
spdk_blob_get_e

blob: add spdk_blob_get_esnap_bs_dev()

While getting memory domains, vbdev_lvol will need to be able to access
the bdev that acts as the lvol's external snapshot. The introduction of
spdk_blob_get_esnap_bs_dev() facilitates this access.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I604c957a468392d40b824c3d2afb00cbfe89cd21
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16429
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

blob: hotplug new back_bs_dev

When an esnap clone blob's external snapshot arrives after the blob is
opened, it can now be hot-added to the blob. Presumably the new device
replaces a place-holder de

blob: hotplug new back_bs_dev

When an esnap clone blob's external snapshot arrives after the blob is
opened, it can now be hot-added to the blob. Presumably the new device
replaces a place-holder device that did not really atteempt IO.

Change-Id: I622feb84efa66628debf44f7e7cb88b6a012db6d
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16232
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blob: add spdk_blob_is_esnap_clone

Add an API to easily determine if a blob is an esnap clone, similar to
what already exists for snapshot, clone, and thin_provisioned.

Signed-off-by: Mike Gerdts <

blob: add spdk_blob_is_esnap_clone

Add an API to easily determine if a blob is an esnap clone, similar to
what already exists for snapshot, clone, and thin_provisioned.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ie07cd09b30513893e82f1c85e94a24a93c79d71e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16862
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot

show more ...

blob: clones of external snapshots

This is the beginning of support for external snapshots. An external
snapshot is a read-only blobstore device (struct spdk_bs_dev) that can
be used as a blob's bac

blob: clones of external snapshots

This is the beginning of support for external snapshots. An external
snapshot is a read-only blobstore device (struct spdk_bs_dev) that can
be used as a blob's back device. Normally a blob will have no back
device (a normal blob), a zeroes back device (a thin provisioned blob),
or a blob back device (a clone blob). When a blob has an external
snapshot ("esnap") as its back device, it is called an esnap clone.

With this patch, esnap clones can be created but they are not yet
useful. Subsequent patches in the series will plumb the IO path, enable
various features, and allow lvol bdevs to be esnap clones.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I29206b628a2b03b6386a88532565e228df988e0e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14969
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blob: Add functions to find [un]allocated io_unit

These functions start from a given offset and seek for first
io_unit belonging to an allocated cluster or first io_unit
belonging to an unallocated

blob: Add functions to find [un]allocated io_unit

These functions start from a given offset and seek for first
io_unit belonging to an allocated cluster or first io_unit
belonging to an unallocated cluster

Signed-off-by: Damiano Cipriani <damiano.cipriani@suse.com>
Change-Id: I0c632e2b3dfd2e96aa22e21796e25a36f2f55f9f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14360
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot

show more ...

blobstore: implement spdk_bs_grow and bdev_lvol_grow_lvstore RPC

The bdev_lvol_grow_lvstore will grow the lvstore size if the undering
bdev size is increased. It invokes spdk_bs_grow internally. The

blobstore: implement spdk_bs_grow and bdev_lvol_grow_lvstore RPC

The bdev_lvol_grow_lvstore will grow the lvstore size if the undering
bdev size is increased. It invokes spdk_bs_grow internally. The
spdk_bs_grow will extend the used_clusters bitmap. If there is no
enough space resereved for the used_clusters bitmap, the api will
fail. The reserved space was calculated according to the num_md_pages
at blobstore creating time.

Signed-off-by: Peng Yu <yupeng0921@gmail.com>
Change-Id: If6e8c0794dbe4eaa7042acf5031de58138ce7bca
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9730
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

blob: Add readv/writev_ext functions

These function accept optional spdk_blob_ext_io_opts
structure. If this structure is provided by the user
then readv/writev_ext ops of base dev will be used
in d

blob: Add readv/writev_ext functions

These function accept optional spdk_blob_ext_io_opts
structure. If this structure is provided by the user
then readv/writev_ext ops of base dev will be used
in data path

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I370dd43f8c56f5752f7a52d0780bcfe3e3ae2d9e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11371
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>

show more ...

lib/blob: add a map file.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I72da725f82dfa3eeb2c0ec6614ecc9ffb0b852a8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1682
Tested

lib/blob: add a map file.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I72da725f82dfa3eeb2c0ec6614ecc9ffb0b852a8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1682
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom CI

show more ...