ut/blob: simplify error handling for UT

Failure to prepare CUnit for tests is always a fatal error,
and application exits.
Most test managment functions in CUnit set some CU_ErrorCode,
that can be r

ut/blob: simplify error handling for UT

Failure to prepare CUnit for tests is always a fatal error,
and application exits.
Most test managment functions in CUnit set some CU_ErrorCode,
that can be retrived by return code from a function or CU_get_error().

Thus usual structure of UT in SPDK checked every output from every
test managment function.
There is a helper function CU_set_error_action() that when set to
CUEA_ABORT, will abort the application run with proper error return code.
Along with nice error message.

For example if one were to add same UT twice:
"Aborting due to error #32: Test having this name already in suite."
Application return code set to 32.

Using CU_set_error_action(CUEA_ABORT), removes the need for error checking
on test managment functions in CUnit.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ida7de5e040b714509e79957d47a006ee518a42b8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/930
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

ut/blob: issue to consecutive md syncs

Expands blob_simultaneous_operations to cover case
when second md sync is perfomed without waiting for the first
one to complete.

The blob state is forced in

ut/blob: issue to consecutive md syncs

Expands blob_simultaneous_operations to cover case
when second md sync is perfomed without waiting for the first
one to complete.

The blob state is forced in UT, but in normal usage could be
changed by modifying xattr, allocating thin clusters or other.

Related to #1170

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I880d289cdf957d4513c080efdcb7a5d8c6b3b447
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/775
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

ut/blob: add tests for persiting metadata

This new UT is checking validity of md sync being run
along another md sync with changed metadata.

At this time added case where first md sync is being
int

ut/blob: add tests for persiting metadata

This new UT is checking validity of md sync being run
along another md sync with changed metadata.

At this time added case where first md sync is being
interrupted by removal of xattr with second md sync.
This interruption is perfomed at increasing number of
poller executions, until the number is enough to
complete first md sync.

There are two expected states of used_md_pages array,
either with xattr or without. The state is verified
after md syncs.

This UT will be expaned in similar manner by other
operations (than xattr) that casue changes in persisted metadata.

Related to #960

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I251dca92ffc3080d8dc503a7f1ff342aa59adef9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/774
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

ut/blob: use poll_thread_times()

Making use of recently added utility to poll exact number of times.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I23a9dc702c899285626522ace

ut/blob: use poll_thread_times()

Making use of recently added utility to poll exact number of times.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I23a9dc702c899285626522ace8ec77a9ba0f7757
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/773
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

lib/thread: Assert if spdk_put_io_channel() is called on the wrong thread

spdk_put_io_channel() was designed to be called on the same thread
that called spdk_get_io_channel(). spdk_put_io_channel()

lib/thread: Assert if spdk_put_io_channel() is called on the wrong thread

spdk_put_io_channel() was designed to be called on the same thread
that called spdk_get_io_channel(). spdk_put_io_channel() sends a
message to its own thread, to allow the context to unwind before
releasing the resources. This had the side effect to allow an
incorrect thread to call spdk_put_io_channel(). This patch will fix
that.

Bdevperf tool had a design flaw that needed the side effect, but
it was fixed recently. We do not know if we have any other case.

Hence add assert to spdk_put_io_channel() to find other case.

We found that unit test for blobstore had called
spdk_put_io_channel() and fix it together in this patch.

Besides, correct the comment for spdk_put_io_channel() in
include/spdk/thread.h not to create any other case in future.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6ec7bf074818abef43b23ca40bc9385adac70a75
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479390
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>

show more ...

lib/blob: set default use_extent_table to true

Extent table and extent page descriptors are now
set to be default way clusters are serialized on disk.

With this patch UT are ran with and without
ex

lib/blob: set default use_extent_table to true

Extent table and extent page descriptors are now
set to be default way clusters are serialized on disk.

With this patch UT are ran with and without
extent table.

Changed two asserts in test, since amount is dependent on
which type of serialization is used.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ica58fce6a4effd014d7dd40ee26edd0fa3196d0f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481901
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>

show more ...

ut/blob: redefine spdk_blob_opts_init()

This will be used to add another run of whole UT suite
with extent pages on/off.

Next patch in series will be enabling both types of
extent serialization for

ut/blob: redefine spdk_blob_opts_init()

This will be used to add another run of whole UT suite
with extent pages on/off.

Next patch in series will be enabling both types of
extent serialization for all UT.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia8b4b8822edefb90ffc13cf777885f9af95e4545
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482170
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>

show more ...

ut/blob: replace all blob creation with extended version

All the non-ext version of the call is doing, is calling
ext with NULL as opts. Then default opts are used in
its place.

This change was fac

ut/blob: replace all blob creation with extended version

All the non-ext version of the call is doing, is calling
ext with NULL as opts. Then default opts are used in
its place.

This change was facilitated by next on in series,
where all blob opts will be initalized in UT
with parameter use_extent_table either set to
true or false.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I62b642c1808b38a5f7c94a5900f25f4978a4ec39
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482859
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>

show more ...

ut/blob: assure bs and blobs have expected state during power failure

After creation of blobs in both tests, only clusters indexed from
0 to 10 are supposed to be used. Index 0 for md and 1-10 for d

ut/blob: assure bs and blobs have expected state during power failure

After creation of blobs in both tests, only clusters indexed from
0 to 10 are supposed to be used. Index 0 for md and 1-10 for data
of single blob since it was create thick provisoned.
Cluster allocations are done in order so if there was a bug for
overflow amount of clusters claimed, first in order would be
one with index 11.
This patch adds asserts after each bs load for first data cluster
that is supposed to be used and for first data cluster that is
not.
During the tests those should remain constant.

When creating/deleting snapshots, the blobs are affected
by changing their type to/from thin_provisioned.
Added asserts to verify their state at every blob open.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I38418da55850d5b8468e578b3c42c5b817ae8045
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482661
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

ut/blob: do not check for success on power failure UT

g_bserrno from blob deletion or snapshot creation,
should not be checked. It is implementation
dependent whether the error (or success) from tho

ut/blob: do not check for success on power failure UT

g_bserrno from blob deletion or snapshot creation,
should not be checked. It is implementation
dependent whether the error (or success) from those
calls actually means that enough data was persisted
on disk.
This test case should work even if we set the threshold
high enough that no failed opperations occur.

On the other hand some parts of those calls do cleanup
in them, meanwhile there is enough metadata data on disk already.
Such as cleaning up unused clusters or pages issue
writes, but at that point the blobs already are in expected
state.

Thus removed assert for g_bserrno, as failure is not indicative
of impossibility to recover.

While here, removed the spdk_bs_unload(). This UT are for
testing power fail safety. Never should it be the case that
enough writes occured in create/delete, but blobs are not
in the expected state.
When such bug would be introduced, it could be covered up
by spdk_bs_unload() cleanly closing up the blobstore.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic69c3061f2cc1fe04bf895632cdb11efb2fe6912
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482660
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lib/blob: pass Extent Page offset on cluster allocation

Extent Pages claim and insertion can be asynchronous
when cluster allocation happens due to writing to a new cluster.

In such case lowest fre

lib/blob: pass Extent Page offset on cluster allocation

Extent Pages claim and insertion can be asynchronous
when cluster allocation happens due to writing to a new cluster.

In such case lowest free cluster and lowest free md page
is claimed, and message is passed to md_thread.
Where inserting both into the arrays and md_sycn happens.

This patch adds parameters to pass the Extent Page offset
in such case.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I46d8ace9cd5abc0bfe48174c2f2ec218145b9c75
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479849
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

lib/blob: store clear_method in per blob metadata

Accept a clear method option on blob create by adding clear_method
to the opts structure passed in to _spdk_bs_create_blob(). Store
these 2 bits in

lib/blob: store clear_method in per blob metadata

Accept a clear method option on blob create by adding clear_method
to the opts structure passed in to _spdk_bs_create_blob(). Store
these 2 bits in md_ro_flags so that earlier versions without an
understanding of these bits can not alter metadata.

The new metadata values will be used later in the series.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I5440645ca20b426778d13b2e544b65dc2b3b83c7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472204
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

ut/blob: check for clone count in blob_snapshot UT

Previously this UT was only checking the snapshot counts.
This was correct, but since the blobs in this tests are
clones of each other - it would m

ut/blob: check for clone count in blob_snapshot UT

Previously this UT was only checking the snapshot counts.
This was correct, but since the blobs in this tests are
clones of each other - it would make sense to check it as well.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id9dec7f0a292cbb52da8196bfc77679fc0f49d8b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478976
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

ut/blob: added poll_threads() after deleting blob

For deletion to actually deliver results, poll_threads()
has to be called.

g_bserrno came from previous API call, not delete itself.

This patch fi

ut/blob: added poll_threads() after deleting blob

For deletion to actually deliver results, poll_threads()
has to be called.

g_bserrno came from previous API call, not delete itself.

This patch fixes it so that g_bserrno is from the delete itself.

Next patch in series adds checks for number of clones.
Having such checks in first place,
would allow to notice lack of poll_threads() here before.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I3477f5c5b99106b70ee3d5706906a4dc94283180
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478975
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

ut/blob: fix freeing io_channels

This patch fixes minor unit test deallocations around io_channels.
It has not exposed any issues in blobstore code, but should always be right
to prevent covering up

ut/blob: fix freeing io_channels

This patch fixes minor unit test deallocations around io_channels.
It has not exposed any issues in blobstore code, but should always be right
to prevent covering up any issues later.

1) Two spdk_bs_free_io_channel() were missing
2) Dirty shutdown should still free bs resources with _spdk_bs_free()

Sample log at the end of UT, before this patch:
thread.c: 200:_free_thread: *ERROR*: thread 0x617000000080 still has channel for io_device blobstore
thread.c: 200:_free_thread: *ERROR*: thread 0x617000000080 still has channel for io_device blobstore
thread.c: 200:_free_thread: *ERROR*: thread 0x617000000080 still has channel for io_device blobstore
thread.c: 180:spdk_thread_lib_fini: *ERROR*: io_device blobstore not unregistered
thread.c: 180:spdk_thread_lib_fini: *ERROR*: io_device blobstore not unregistered

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I187bc61bb6e094c9c740a987e7d14760551a0503
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475872
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

ut/blob: test cluster selection using _spdk_bs_allocate_cluster()

Previously blob_insert_cluster_msg UT specified the cluster
which will be used to store LBA. So for blob the cluster [1],
was always

ut/blob: test cluster selection using _spdk_bs_allocate_cluster()

Previously blob_insert_cluster_msg UT specified the cluster
which will be used to store LBA. So for blob the cluster [1],
was always stored as cluster 0xF in blobstore.

This patch changes _spdk_bs_claim_cluster() with preselected
cluster in blobstore to _spdk_bs_allocate_cluster() that
will choose first free one in the map.

It will help in further patches that add more logic
on extent table and extent pages in _spdk_bs_allocate_cluster().
Otherwise this patch 'hard coded' values might get more complex
than needed.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I238329b2191e7ebd3f73c1fe85d7e1da0249979e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475493
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

blob: fix sequentially allocated clusters starting from 0

When serializing extents, run-length encoding is supposed to
1) RLE all sequential LBAs
2) RLE zero LBAs (unallocated)

There is one special

blob: fix sequentially allocated clusters starting from 0

When serializing extents, run-length encoding is supposed to
1) RLE all sequential LBAs
2) RLE zero LBAs (unallocated)

There is one special case, with sequential LBAs that start
with 0 LBA. This is RLE as 1) case, but results in descriptor
matching case 2). Which causes loss of allocated clusters.

This requires following conditions to be met:
- blobstore has just a single cluster reserved for MD
- blob is thin provisioned
- first allocation occurs on cluster_num=1

For last part to be true, very first write for blob has to be
issued to LBA between cluster_size and 2*cluster_size.
Causing allocation of second cluster in blobstore and assiging
it LBA equal to number of LBAs per cluster.

To fix this, case 1) disallows to RLE zeroes.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I136282407966310c882ca97c960e9a71c442c469
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475494
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

ut/cunit: added missing semicolon on CU_ASSERT macros

This was inconsistent on all CU_ASSERT macros,
so now it is fixed.

Change-Id: I6d20f98528c4041baf02cb419ab9644a45b807c4
Signed-off-by: Tomasz Z

ut/cunit: added missing semicolon on CU_ASSERT macros

This was inconsistent on all CU_ASSERT macros,
so now it is fixed.

Change-Id: I6d20f98528c4041baf02cb419ab9644a45b807c4
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459760
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

lib/blob: only validate blobid of first page during bs_load

Blob id only is matched to the very first page of md for
that particular blob.

During loading blobstore, we shouldn't verify
further page

lib/blob: only validate blobid of first page during bs_load

Blob id only is matched to the very first page of md for
that particular blob.

During loading blobstore, we shouldn't verify
further pages in chain against the blobid.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ifc7863ddcb403aedc264c14e6b4c3915bd30dc41
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460607
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>

show more ...

lib/blob: do not allow xattr to exceed maximum descriptor length

Length of xattr descriptor is equal to length of xattr struct,
xattr name and the len of stored value.

There is no limit to how much

lib/blob: do not allow xattr to exceed maximum descriptor length

Length of xattr descriptor is equal to length of xattr struct,
xattr name and the len of stored value.

There is no limit to how much can be stored in memory for xattr.
On disk xattr size is limited to single page and within that to
max descriptors that can fit in it.
This size is known at compile time.

Before this patch it was possible to add xattr exceeding
what was possible to be written to disk. This caused issues
when serializing the metadata during spdk_blob_sync_md()
or spdk_blob_close(). Making those fail without specific info
to the user and not actually writting such descriptor.

Since maximum length of xattr descriptor is known at compile time,
this patch compares against this value when setting the xattr.
It will immediately report back to user with error, and will
not store xattr in memory (thus not serialize it).

This patch should not affect any backward compatibility for blobs.
Too large xattrs weren't written to disk before,
API for blobstore stays the same - only reporting ENOMEM
when it should.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6f4af4d079e47f084e20d7a4969d9a78ec1f8610
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460450
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>

show more ...

blobstore: Cleanup after power failure while creating snapshot

Currently we are missing cleanup routine for case when
power failure interrupts creating snapshot. This patch
add such routine.

For th

blobstore: Cleanup after power failure while creating snapshot

Currently we are missing cleanup routine for case when
power failure interrupts creating snapshot. This patch
add such routine.

For the case where we find blob with a parent snapshot ID
matching newly created snapshot we can finish whole process
during recovery by processing forward with setting snpashot
as read only, removing xattr and syncing. We should remove
snapshot only if there is no blob with parent pointing at
snapshot.

Fixes github issue #760

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I2f0e298164e07a2b4dfa5367e8878facef640702
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455216
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

blobstore/ut: Delete snapshot and power failure unit test

This is unit test for power failure event during snapshot
deletion. At the same time this is an example how to
use new power failure event f

blobstore/ut: Delete snapshot and power failure unit test

This is unit test for power failure event during snapshot
deletion. At the same time this is an example how to
use new power failure event functionality in unit tests.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I9212392f665576fa16edd28c609199d0e02dc434
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454608
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

blobstore: Make possible to remove snapshot if there is only one clone

Starting with this patch it is possible to remove a
snapshot if there is only a one clone created from it.

In such case snapsh

blobstore: Make possible to remove snapshot if there is only one clone

Starting with this patch it is possible to remove a
snapshot if there is only a one clone created from it.

In such case snapshot can be removed without any data
copying. This is achieved with following steps (case
with only one clone):
1. Open snapshot (Snapshot1) that shall be removed
2. Check if the Snapshot1 has no more than 1 clone (Clone1)
3. Remove Clone1 entry from Snapshot1
4. If the Snapshot1 has a parent snapshot (Snapshot2):
4a. Add Clone1 entry to the Snapshot2 clones list
4b. Remove Snapshot1 entry from Snapshot2 clones list
5. Open Clone1 blob
6. Freeze I/O operations on Clone1
7. Temporarily override md_ro flag for Snapshot1 and Clone1
for MD modification
8. Merge Snapshot1 and Clone1 clusters maps into Clone1
clusters map
9a. If Snapshot2 is present switch parent ID and backing
bs_dev on Clone1
9b. If Snapshot2 is not present set parent ID to
SPDK_BLOBID_INVALID and backing bs_dev to zeroes_dev
10. Sync MD on Clone1
11. Sync MD on Snapshot1
12. Restore MD flags for Clone1 and Snapshot1
13. Unfreeze I/O on Clone1
14. Close Clone1 blob
15. Remove Snapshot1

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I800724b981af894e01e1912d0077c5b34a2ae634
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445576
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>

show more ...

blobstore: Remove blob on blobstore load when required

In some cases user may want to flag blob for removal
then do some operations (before removing it) and while
it happens there might be power fai

blobstore: Remove blob on blobstore load when required

In some cases user may want to flag blob for removal
then do some operations (before removing it) and while
it happens there might be power failure. In such cases
we should remove this blob on next blobstore load.
Example of such usage is delete snapshot functionality
that will be introduced in upcoming patch.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I85f396b73762d2665ba8aec62528bb224acace74
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453835
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

ut/blobstore: write and allocate clusters from multiple threads

This patch verifies that functionality of only releasing cluster
that was originaly claimed in race thread condition.

Change-Id: I4d6

ut/blobstore: write and allocate clusters from multiple threads

This patch verifies that functionality of only releasing cluster
that was originaly claimed in race thread condition.

Change-Id: I4d60ffdb4fd6612d61a761835aae2fd1bf81b7fd
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452626
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...