bit_array: return UINT32_MAX if no cleared bits found

spdk_bit_array_find_first_set() returns UINT32_MAX if no
set bits are found. But spdk_bit_array_find_first_clear()
would return the size of the

bit_array: return UINT32_MAX if no cleared bits found

spdk_bit_array_find_first_set() returns UINT32_MAX if no
set bits are found. But spdk_bit_array_find_first_clear()
would return the size of the bit array instead in this case.
(Note: the comments say size of the bit array + 1 which was
incorrect)

So this patch makes spdk_bit_array_find_first_clear()
consistent with spdk_bit_array_find_first_set() and returns
UINT32_MAX if no cleared bit is found.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I44c1e674149f8c2e87122800d5db45a2851f0bef

Reviewed-on: https://review.gerrithub.io/428225
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>

show more ...

blobstore: allow I/O operations to use io unit size smaller than page size.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I994b5d46faffd34430cb39e66225929c4cba90ba
Reviewed

blobstore: allow I/O operations to use io unit size smaller than page size.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I994b5d46faffd34430cb39e66225929c4cba90ba
Reviewed-on: https://review.gerrithub.io/414935
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

blobstore: Introduce io_unit size to blobstore.

This patch just adds the call, but doesn't change behaviour of blobstore.
io unit size remains same as page size.

Signed-off-by: Piotr Pelplinski <pi

blobstore: Introduce io_unit size to blobstore.

This patch just adds the call, but doesn't change behaviour of blobstore.
io unit size remains same as page size.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Idcd1b7d5126fc7cacf12c996188bd41e2c9a744d

Reviewed-on: https://review.gerrithub.io/425355
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

thread: Add a name parameter to spdk_register_io_device

This is a string name used for debugging only.

Change-Id: I9827f0e6c83be7bc13951c7b5f0951ce6c2a1ece
Signed-off-by: Ben Walker <benjamin.walke

thread: Add a name parameter to spdk_register_io_device

This is a string name used for debugging only.

Change-Id: I9827f0e6c83be7bc13951c7b5f0951ce6c2a1ece
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424127
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>

show more ...

blob: Make sure that our IOVs match the io size

This is in response to a Scan-build error with Clang 6.0 but is a real
bug. If we don't match up our IOVs properly with the lenght we provide,
we coul

blob: Make sure that our IOVs match the io size

This is in response to a Scan-build error with Clang 6.0 but is a real
bug. If we don't match up our IOVs properly with the lenght we provide,
we could jump over the iov list into invalid memory in
_spdk_rw_iov_split_next.

Change-Id: I472a7aa53027af0a18ebeee8226e0b008447cce4
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/424248
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lib: fix typos in the lib directory

Change-Id: Idcb60b79d2902bb316facc6f60e0a81e5cf847ed
Signed-off-by: Chen Wang <chenx.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/423372
Reviewed-by:

lib: fix typos in the lib directory

Change-Id: Idcb60b79d2902bb316facc6f60e0a81e5cf847ed
Signed-off-by: Chen Wang <chenx.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/423372
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

blobstore: recalculate total clusters after reading super block

Blobstore uses 1MB cluster size by default, but logical volumes
override it to 4MB by default. When an existing lvolstore is loaded,

blobstore: recalculate total clusters after reading super block

Blobstore uses 1MB cluster size by default, but logical volumes
override it to 4MB by default. When an existing lvolstore is loaded,
all cluster calculations were being done based on the 1MB size - not
the 4MB cluster size read from the superblock. That would result
in asserts (due to mismatched used cluster mask size) and all kinds of
other possible weirdness with subsequent operations.

Fixes: 2c91e91907e2 ("blobstore: Save the original size of the disk.")

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5d57c2b64aba791903e69560b9fe5684a72669df

Reviewed-on: https://review.gerrithub.io/420582
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

blobstore: silence false error message if device is too small

It is not an error if bdev is smaller than cluster size so convert this
error to infolog. This fixes false error message dring examine p

blobstore: silence false error message if device is too small

It is not an error if bdev is smaller than cluster size so convert this
error to infolog. This fixes false error message dring examine process.

To return proper error message when creating blobstore using RPC the
_spdk_bs_alloc was adjusted to return errocode that is propagated up and
converted to "No space left on device".

Fixes #316

Change-Id: Ic9803720a55125fcfa34263346f2d9e1aae03a53
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/420054
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

blobstore: no copy write in thin-provisioning

This patch prevents to copy cluster data when there is not
backing blob to improve cluster allocation performance
in thin provisioned blobs.

Change-Id:

blobstore: no copy write in thin-provisioning

This patch prevents to copy cluster data when there is not
backing blob to improve cluster allocation performance
in thin provisioned blobs.

Change-Id: Ie766d2e5274daa74c2b13b2198a20205e3417467
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/417938
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

blob: fix unallocated clusters to run-length encode

Before this patch when serializing extents,
unallocated clusters were treated as separate lba.
This caused metadata to grow without need.

Change-

blob: fix unallocated clusters to run-length encode

Before this patch when serializing extents,
unallocated clusters were treated as separate lba.
This caused metadata to grow without need.

Change-Id: I5d66466dda5f5e6d4d53f4ed5bd0bac18c74be96
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/419180
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

blobstore: Save the original size of the disk.

Save the original size of the disk to metadata when it is first created.
On load verify that the disk did not change size.

Signed-off-by: Piotr Pelpli

blobstore: Save the original size of the disk.

Save the original size of the disk to metadata when it is first created.
On load verify that the disk did not change size.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I535940ee188425ee3b394effd99653cc073d541e

Reviewed-on: https://review.gerrithub.io/410896
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

blobstore: postpone all superblock writes to first metadata sync

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I49d23d2af40ff909757a5fd15b80a7a8cbbff399

Reviewed-on: https

blobstore: postpone all superblock writes to first metadata sync

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I49d23d2af40ff909757a5fd15b80a7a8cbbff399

Reviewed-on: https://review.gerrithub.io/416922
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>

show more ...

blobstore: add decouple parent function

This patch adds an API to decouple blobs parent removing dependency
on it. Blob stays thin after this operation.

Also unit tests for blobstore inflate are im

blobstore: add decouple parent function

This patch adds an API to decouple blobs parent removing dependency
on it. Blob stays thin after this operation.

Also unit tests for blobstore inflate are improved and reused with
decouple parent functionality.

Change-Id: I96dfee467c78cf4f4d929ec7bc05263f7a23a8aa
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/410829
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

blobstore: fix parent for snapshot of clone

When snapshot is created from a clone, clones parent is not
inherited.

This patch also updates unit tests covering this case.

Change-Id: I42eb00fe2f3350

blobstore: fix parent for snapshot of clone

When snapshot is created from a clone, clones parent is not
inherited.

This patch also updates unit tests covering this case.

Change-Id: I42eb00fe2f33504c5b5c5beded20c4ea65eaff67
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/414804
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>

show more ...

blob: change lba to uint64_t in serialize_extent

Make sure we don't truncate the LBA when using it to serialize the
cluster array into an extent list.

We also need to add an explicit cast in _spdk_

blob: change lba to uint64_t in serialize_extent

Make sure we don't truncate the LBA when using it to serialize the
cluster array into an extent list.

We also need to add an explicit cast in _spdk_bs_cluster_to_lba
to ensure the conversion doesn't get truncated. While here, do
the same cast for _spdk_bs_cluster_to_page.

Change-Id: If4e65ed86550e39dfa39826930dfafac158d519c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-on: https://review.gerrithub.io/416231
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

blob: factor out mask loading into a function

This is duplicated in three places; combine them into a shared helper
function.

Change-Id: I47682da8fa8b13134a6422c6a0e8a4d68f12ee36
Signed-off-by: Dan

blob: factor out mask loading into a function

This is duplicated in three places; combine them into a shared helper
function.

Change-Id: I47682da8fa8b13134a6422c6a0e8a4d68f12ee36
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/416257
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>

show more ...

blob: use bitarray function to count free clusters

Change-Id: I4830bcd2342551a6a02afd6e4551acbf463eb44c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub

blob: use bitarray function to count free clusters

Change-Id: I4830bcd2342551a6a02afd6e4551acbf463eb44c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/416256
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>

show more ...

blob: fix load of non-multiple-of-8 masks

Previously, the blobstore load code was iterating over the masks (blob
IDs, clusters) byte by byte, then bit by bit in a nested loop, but it
was rounding in

blob: fix load of non-multiple-of-8 masks

Previously, the blobstore load code was iterating over the masks (blob
IDs, clusters) byte by byte, then bit by bit in a nested loop, but it
was rounding incorrectly and skipping any bits set in the last byte if
the total size was not a multiple of 8.

Replace the nested loops with a single loop iterating over bits to
simplify the code and avoid the bug.

Change-Id: Ib365421bf3ba1002d2e5634b34c2bcf9ef7d1267
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/416230
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>

show more ...

blobstore: check return code in IO freeze completion

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I744948dd98cb210b41e59bfed1ef8a448f77ff07
Reviewed-on: https://review.gerrithub.i

blobstore: check return code in IO freeze completion

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I744948dd98cb210b41e59bfed1ef8a448f77ff07
Reviewed-on: https://review.gerrithub.io/415254
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Piotr Pelpliński <piotr.pelplinski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blob: add metadata dump capability

Add spdk_bs_dump which dumps low level blobstore metadata
information to a specified FILE.

Also add a corresponding -D option to blobcli which
utilizes this new f

blob: add metadata dump capability

Add spdk_bs_dump which dumps low level blobstore metadata
information to a specified FILE.

Also add a corresponding -D option to blobcli which
utilizes this new functionality.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iad018b70f8caa4f950d55dd308b9000d55d885ae

Reviewed-on: https://review.gerrithub.io/414479
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>

show more ...

thread: Replace #include of io_channel.h with thread.h

Change-Id: I6babd4cf990bf19b510db88bdfb0ca81e29d9252
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithu

thread: Replace #include of io_channel.h with thread.h

Change-Id: I6babd4cf990bf19b510db88bdfb0ca81e29d9252
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/414700
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Madhu Pai <mpai@netapp.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>

show more ...

blob: mark internal function static

_spdk_blob_insert_cluster_on_md_thread() is defined and used in the same
file, so it doesn't need to be exposed outside of the compilation unit.

Change-Id: Ifc32

blob: mark internal function static

_spdk_blob_insert_cluster_on_md_thread() is defined and used in the same
file, so it doesn't need to be exposed outside of the compilation unit.

Change-Id: Ifc327a3d80cae47b28fa2a9ba77471a9b80c83e0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/414702
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blobstore: freeze I/O during resize

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I23c34d4dcb542aa9ab3fa8cb734cf9cc0e0fc5da

Reviewed-on: https://review.gerrithub.io/409144

blobstore: freeze I/O during resize

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I23c34d4dcb542aa9ab3fa8cb734cf9cc0e0fc5da

Reviewed-on: https://review.gerrithub.io/409144
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

blobstore: freeze I/O during snapshoting.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I6182eb3a77d23db7088703492d71349e3a4b6460
Reviewed-on: https://review.gerrithub.io/3

blobstore: freeze I/O during snapshoting.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I6182eb3a77d23db7088703492d71349e3a4b6460
Reviewed-on: https://review.gerrithub.io/399366
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blobstore: Change behaviour of dirty bit

The patch disables writing dirty bit during blobstore loading.
Instead, dirty bit is written prior to the first metadata update.

Signed-off-by: Piotr Pelpli

blobstore: Change behaviour of dirty bit

The patch disables writing dirty bit during blobstore loading.
Instead, dirty bit is written prior to the first metadata update.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I7be81009a99f09048bf23749c8f6ef5e9f7b3751

Reviewed-on: https://review.gerrithub.io/410884
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>

show more ...