Revert "blobstore: Add a blob_bs_dev that provides back_bs_dev for clones"

This change wasn't correctly rebased and needs to be updated to compile
against the current blobstore.

This reverts commit

Revert "blobstore: Add a blob_bs_dev that provides back_bs_dev for clones"

This change wasn't correctly rebased and needs to be updated to compile
against the current blobstore.

This reverts commit c1174e6895ca44242d12b2e5f0742f84ccbf0b8f.

Change-Id: I529608bee7323cb626d8c36dff15adc9ba24ad26
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/402352
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blobstore: Add a blob_bs_dev that provides back_bs_dev for clones

Unit tests implemented in following patches.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Ib18c9060f527b

blobstore: Add a blob_bs_dev that provides back_bs_dev for clones

Unit tests implemented in following patches.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Ib18c9060f527bd22bfdbed74e96871a6e0551ead
Reviewed-on: https://review.gerrithub.io/396648
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>

show more ...

blob: add option to iterate all blobs during spdk_bs_load

blobfs and lvol can now use this to automatically iterate
all existing blobs during spdk_bs_load. Changes to blobfs
and lvol will come in f

blob: add option to iterate all blobs during spdk_bs_load

blobfs and lvol can now use this to automatically iterate
all existing blobs during spdk_bs_load. Changes to blobfs
and lvol will come in future patches.

This will also be used in some upcoming patches which need
to iterate through blobs during load to determine
snapshot/clone relationships.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic7c5fac4535ceaa926217a105dda532517e3e251

Reviewed-on: https://review.gerrithub.io/400177
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

blob: fix _spdk_bs_load_ctx_fail ordering

Finish the sequence first, before calling _spdk_bs_free().
Otherwise synchronous bs_devs (like we use in the unit
tests) cause the sequence memory to get fr

blob: fix _spdk_bs_load_ctx_fail ordering

Finish the sequence first, before calling _spdk_bs_free().
Otherwise synchronous bs_devs (like we use in the unit
tests) cause the sequence memory to get freed via
_spdk_bs_free() and then we try to finish the sequence.

This eliminates the need for g_scheduler_delay and
_bs_flush_scheduler() in the blob unit tests. But don't
remove them - they will be useful in upcoming unit tests
for queued persist operations.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I09aac3ae4d3a56ff8e04a5b822fcd6746f13afc3

Reviewed-on: https://review.gerrithub.io/401267
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

blob: add _spdk_blob_persist_start

No functional change - this just separates out the
code that creates the persist ctx from the code that
actually performs the persist operation.

Part of series to

blob: add _spdk_blob_persist_start

No functional change - this just separates out the
code that creates the persist ctx from the code that
actually performs the persist operation.

Part of series to enable queuing persist operations -
this will be useful for starting a previously queued
persist operation.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie1966ff2a477f3075c36f90560010d036658f803

Reviewed-on: https://review.gerrithub.io/401255
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

blob: use _spdk_blob_persist_complete for all persist error paths

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaef32731b05a53ac0707524d78086eedc89d6af6

Reviewed-on: https://revi

blob: use _spdk_blob_persist_complete for all persist error paths

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaef32731b05a53ac0707524d78086eedc89d6af6

Reviewed-on: https://review.gerrithub.io/401254
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

blob: clean up some blob->state checks

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I096fb24dd2fe2fc4dd97d80c957c328d960fb867
Reviewed-on: https://review.gerrithub.io/401073
Teste

blob: clean up some blob->state checks

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I096fb24dd2fe2fc4dd97d80c957c328d960fb867
Reviewed-on: https://review.gerrithub.io/401073
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

blob: always use _spdk_blob_persist to check for CLEAN state

spdk_blob_close() and spdk_blob_sync_md() currently do their
own CLEAN state. To consolidate the state checking code,
have both function

blob: always use _spdk_blob_persist to check for CLEAN state

spdk_blob_close() and spdk_blob_sync_md() currently do their
own CLEAN state. To consolidate the state checking code,
have both functions rely on the check in _spdk_blob_persist()
instead.

This will reduce code but more importantly is needed for
some upcoming changes for queuing persist operations.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I38118624b4fad6f18c4b7466d9ddfa0915c3fce0

Reviewed-on: https://review.gerrithub.io/401065
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

blob: add _spdk_blob_verify_md_op

These are common functions that can be called from
any function that reads or modifies a blob's metadata to
perform necessary asserts.

This will also fix several p

blob: add _spdk_blob_verify_md_op

These are common functions that can be called from
any function that reads or modifies a blob's metadata to
perform necessary asserts.

This will also fix several places where blob metadata
functions were asserting the calling thread context,
but not the current state of the blob.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9e16c082a27c439311f8ff214335adadfa715497

Reviewed-on: https://review.gerrithub.io/401053
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

blob: remove SPDK_BLOB_STATE_SYNCING

All metadata operations are now done on the metadata
thread, so we no longer have to worry about one thread
updating in-memory metadata structures while another

blob: remove SPDK_BLOB_STATE_SYNCING

All metadata operations are now done on the metadata
thread, so we no longer have to worry about one thread
updating in-memory metadata structures while another
thread is transferring the in-memory structures to
on-disk structures.

This does not protect against multiple sync operations
outstanding at once - that will be coming in an
upcoming path.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ibf33edf4d41d867c96a38df017737e9ceb87fa58

Reviewed-on: https://review.gerrithub.io/401056
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

blob: add asserts for md operations on md thread

The md (metadata) thread is always the thread that
initialize/loaded the blobstore. Metadata operations may
only be performed from this thread. Thi

blob: add asserts for md operations on md thread

The md (metadata) thread is always the thread that
initialize/loaded the blobstore. Metadata operations may
only be performed from this thread. This patch adds some
more asserts in metadata functions that were previously
missed.

While here, also update some of the blobstore documentation
related to this.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5cafdb3ba402ceb6c3ccb6fdd9d36e7768f59f39

Reviewed-on: https://review.gerrithub.io/400885
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

blob: change spdk_bs_io_xxx_blob functions to spdk_blob_io_xxx

These new names are much more clear and are aligned with other
functions such as spdk_blob_close.

Keep the old names around for now bu

blob: change spdk_bs_io_xxx_blob functions to spdk_blob_io_xxx

These new names are much more clear and are aligned with other
functions such as spdk_blob_close.

Keep the old names around for now but deprecate them. We will
remove them in next release.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Idc60fd0b19fa2a8b0247a1f5835774d342e721f9

Reviewed-on: https://review.gerrithub.io/400884
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>

show more ...

blob: change spdk_blob_set_read_only to return int

This is needed for an upcoming change which will
prevent metadata functions from being called on
threads other than the metadata thread. Without
t

blob: change spdk_blob_set_read_only to return int

This is needed for an upcoming change which will
prevent metadata functions from being called on
threads other than the metadata thread. Without
this change, there was no way for this function
to return an error if it was called from the wrong
thread.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I67e591140194ff6ad250878168f6b166a1ff2282

Reviewed-on: https://review.gerrithub.io/400883
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>

show more ...

blob: revert spdk_blob_data changes

There was some thinking that we would need to allocate
I/O channels on a per-blob basis to handle dynamic
resizing during I/O. Making spdk_blob an opaque handle,

blob: revert spdk_blob_data changes

There was some thinking that we would need to allocate
I/O channels on a per-blob basis to handle dynamic
resizing during I/O. Making spdk_blob an opaque handle,
with the existing spdk_blob structure renamed to
spdk_blob_data was a first step towards making that
happen. But more recent work on blobstore has
simplified the resizing approach, so this spdk_blob_data
is no longer needed. So revert it.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I22e07008faceb70649ee560176ebe5e014d5f1a3

Reviewed-on: https://review.gerrithub.io/400881
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>

show more ...

blob: add _spdk_bs_load_complete

This reduces some code duplication and ensures all
successful load operations (whether or not it included
recovery after power fail) through the same function.

Sign

blob: add _spdk_bs_load_complete

This reduces some code duplication and ensures all
successful load operations (whether or not it included
recovery after power fail) through the same function.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia463ce6ebe976ab2420525d86962e8fe54ad04d7

Reviewed-on: https://review.gerrithub.io/400164
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

blobstore: allow xattrs to be set internally only for blobstore

Patch adds internal version of xattr functions to allow
operations on internal xattrs, which are not visible to
upper layers.
When the

blobstore: allow xattrs to be set internally only for blobstore

Patch adds internal version of xattr functions to allow
operations on internal xattrs, which are not visible to
upper layers.
When there is at least one internal xattr set, also
SPDK_BLOB_INTERNAL_XATTR flag is set in invalid_flags to prevent
loading this blob in previous spdk versions.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Iec918ec858f069f7cd9f36d5e8f0495ffa4a42d8
Reviewed-on: https://review.gerrithub.io/395122
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>

show more ...

blobstore: move free xattr's to separate function

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I8b570802b05b8e03802d3c2b68a1e7644ea548ac
Reviewed-on: https://review.gerrit

blobstore: move free xattr's to separate function

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I8b570802b05b8e03802d3c2b68a1e7644ea548ac
Reviewed-on: https://review.gerrithub.io/396572
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

blobstore: move xattr serialization to separate function

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I277f7288427788e7a107b143331753fd5b23f16f
Reviewed-on: https://review

blobstore: move xattr serialization to separate function

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I277f7288427788e7a107b143331753fd5b23f16f
Reviewed-on: https://review.gerrithub.io/396571
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

blobstore: move xattrs parameters passed as options to separate structure

This change will allow reusing this structure for both internal
and external xattrs as well as in functions having optional

blobstore: move xattrs parameters passed as options to separate structure

This change will allow reusing this structure for both internal
and external xattrs as well as in functions having optional xattr,
but missing other options (i.e. snapshot, clone implemented in next patches)

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Ia6619a75efa0a100168a6f8317be274823af04ab
Reviewed-on: https://review.gerrithub.io/396417
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

blobstore: separate deserializing xattrs to common function

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Icba20351c9ca76397393064b41013c527084853e
Reviewed-on: https://rev

blobstore: separate deserializing xattrs to common function

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Icba20351c9ca76397393064b41013c527084853e
Reviewed-on: https://review.gerrithub.io/396385
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>

show more ...

blob: print error when dma allocation fails

For thin provisioned blobs we allocate dma memory
required for copying cluster from backing device.
When cluster size is too big dma allocation may fail
s

blob: print error when dma allocation fails

For thin provisioned blobs we allocate dma memory
required for copying cluster from backing device.
When cluster size is too big dma allocation may fail
silently (only IO error).

Use PRIu32 to print out the cluster size, and while
here, fix two other places that were using %d to print
cluster size instead of PRIu32.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I098b1a58aee2f0d3f4ead7aa326ecdb63a5b53d8

Reviewed-on: https://review.gerrithub.io/397563
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>

show more ...

blob: make _spdk_bs_allocate_cluster thread safe

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I3c7d2096f549a88b4a9884c0026d15d3bcd8dc67

Reviewed-on: https://review.gerrithub.io/3

blob: make _spdk_bs_allocate_cluster thread safe

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I3c7d2096f549a88b4a9884c0026d15d3bcd8dc67

Reviewed-on: https://review.gerrithub.io/396387
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

blob: support for thin provisioned reads and writes

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ibc9609ad36188006e9454e5c799bccd8a92d7991
Reviewed-on: https://review.gerrithub.io

blob: support for thin provisioned reads and writes

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ibc9609ad36188006e9454e5c799bccd8a92d7991
Reviewed-on: https://review.gerrithub.io/391422
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blob: allow inserting cluster from non metadata thread

This will be needed for thin provisioning, since a write
I/O may result in needing to insert a cluster into the
blob and that write I/O may not

blob: allow inserting cluster from non metadata thread

This will be needed for thin provisioning, since a write
I/O may result in needing to insert a cluster into the
blob and that write I/O may not have been performed
on the metadata thread.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I84b0cb6e7af87b1f9c6cab4e2c24fa26b12e2c06
Reviewed-on: https://review.gerrithub.io/396737
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

blob: add _spdk_blob_sync_md

This enables some code reuse for future patches.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I296a6c5c0915da4a77a1ab43e8f10a335b7d16d0

Reviewed-on:

blob: add _spdk_blob_sync_md

This enables some code reuse for future patches.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I296a6c5c0915da4a77a1ab43e8f10a335b7d16d0

Reviewed-on: https://review.gerrithub.io/396736
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>

show more ...