blobfs: refactor the I/O device free functions.

Some functions should be executed togther, so
group it into a single function which will avoid
the potential issues.

Change-Id: Ie57be20aa76d9bece82b

blobfs: refactor the I/O device free functions.

Some functions should be executed togther, so
group it into a single function which will avoid
the potential issues.

Change-Id: Ie57be20aa76d9bece82b5534c3c5fc6c9faeebad
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/420233
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blobfs: Check fs error info in __do_blob_read

We should check the value of error info. If it
is error, we should not do the read anymore.

Change-Id: Ie3527766779a1f4ed98dde7d9881fe66720badc4
Signed

blobfs: Check fs error info in __do_blob_read

We should check the value of error info. If it
is error, we should not do the read anymore.

Change-Id: Ie3527766779a1f4ed98dde7d9881fe66720badc4
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/420335
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blobfs: add the check for buf allocation

If it fails to allocate the buf, we should conduct
the error handling.

Change-Id: If36b2dd84236052c777f5007ecbf7295e1c1d3eb
Signed-off-by: Ziye Yang <optimi

blobfs: add the check for buf allocation

If it fails to allocate the buf, we should conduct
the error handling.

Change-Id: If36b2dd84236052c777f5007ecbf7295e1c1d3eb
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/420334
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blobfs: change the name macro

The two values are all default values.

Change-Id: I117eb217b71c3f079a4450c6f716f9bb70762399
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://revie

blobfs: change the name macro

The two values are all default values.

Change-Id: I117eb217b71c3f079a4450c6f716f9bb70762399
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/420333
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

blobfs: add some check for req allocation

Reason:It will be helpful for the debug, and
for some places we need to handle no memory
case.

Change-Id: Id221a856b1f65fe7b946dec246bb53532a29db78
Signed-

blobfs: add some check for req allocation

Reason:It will be helpful for the debug, and
for some places we need to handle no memory
case.

Change-Id: Id221a856b1f65fe7b946dec246bb53532a29db78
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/419923
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>

show more ...

thread: Replace #include of io_channel.h with thread.h

Change-Id: I6babd4cf990bf19b510db88bdfb0ca81e29d9252
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithu

thread: Replace #include of io_channel.h with thread.h

Change-Id: I6babd4cf990bf19b510db88bdfb0ca81e29d9252
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/414700
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Madhu Pai <mpai@netapp.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>

show more ...

blobfs: sync length if append occurs after cache eviction

Intermittent failures with the RocksDB tests pointed to
corruption in the MANIFEST file. Further debug showed that
the MANIFEST file would

blobfs: sync length if append occurs after cache eviction

Intermittent failures with the RocksDB tests pointed to
corruption in the MANIFEST file. Further debug showed that
the MANIFEST file would be corrupted when its cache was
evicted. Blobfs was partly handling append after cache
eviction - it would write the data to the correct position
within the blob. But the sync path would not write the
updated file length xattr when the cache had been evicted.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie5cf273f8d6511548146a756a7b63c73fee12101

Reviewed-on: https://review.gerrithub.io/407232
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

blobfs: Remove all uses of strncpy

strncpy is going to be added to the list of banned functions because
it does not guarantee strings are null terminated.

Change-Id: I14e2ce65401bbdb77260627e94aa93

blobfs: Remove all uses of strncpy

strncpy is going to be added to the list of banned functions because
it does not guarantee strings are null terminated.

Change-Id: I14e2ce65401bbdb77260627e94aa936797c20ce0
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/407020
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blob: make spdk_blob_resize an async operation

To support resize operations during I/O, we will need
to send messages to each thread to quiesce I/O while
the resize operation is in progress to guard

blob: make spdk_blob_resize an async operation

To support resize operations during I/O, we will need
to send messages to each thread to quiesce I/O while
the resize operation is in progress to guard against
the cluster map memory changing while another thread
is accessing the cluster map.

Therefore, spdk_blob_resize needs to be asynchronous.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ida037334739b4b80a1dbc76e8f1c70bca8b73582

Reviewed-on: https://review.gerrithub.io/404616
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>

show more ...

blobfs: cleanup and add comments in deleted file cleanup path

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I30675083dcb7dda47ac62eaf124614033ff9c587

Reviewed-on: https://review.g

blobfs: cleanup and add comments in deleted file cleanup path

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I30675083dcb7dda47ac62eaf124614033ff9c587

Reviewed-on: https://review.gerrithub.io/401702
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>

show more ...

blobfs: use new blobstore iter_cb_fn option

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5f9a999e63231f54ba2a2116ec2d6403d9587a3d

Reviewed-on: https://review.gerrithub.io/400185

blobfs: use new blobstore iter_cb_fn option

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5f9a999e63231f54ba2a2116ec2d6403d9587a3d

Reviewed-on: https://review.gerrithub.io/400185
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>

show more ...

blob: change spdk_bs_io_xxx_blob functions to spdk_blob_io_xxx

These new names are much more clear and are aligned with other
functions such as spdk_blob_close.

Keep the old names around for now bu

blob: change spdk_bs_io_xxx_blob functions to spdk_blob_io_xxx

These new names are much more clear and are aligned with other
functions such as spdk_blob_close.

Keep the old names around for now but deprecate them. We will
remove them in next release.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Idc60fd0b19fa2a8b0247a1f5835774d342e721f9

Reviewed-on: https://review.gerrithub.io/400884
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>

show more ...

blobfs: return the correct file length.

If the file have cache data and not flushed, the file length will be
not correct.

Change-Id: Idde23e158c1d010e67579b5d6a8d87d3cfbfed2d
Signed-off-by: Cunyin

blobfs: return the correct file length.

If the file have cache data and not flushed, the file length will be
not correct.

Change-Id: Idde23e158c1d010e67579b5d6a8d87d3cfbfed2d
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
Reviewed-on: https://review.gerrithub.io/393784
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>

show more ...

blobfs: only delete the "removed" file when ref count == 0.

Change-Id: I7bbbed5e62715fbbec381ce4226c5273ebef1a0a
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
Reviewed-on: https://review.gerr

blobfs: only delete the "removed" file when ref count == 0.

Change-Id: I7bbbed5e62715fbbec381ce4226c5273ebef1a0a
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
Reviewed-on: https://review.gerrithub.io/393729
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>

show more ...

blobfs: remove unnecessary opts.

Change-Id: I53c7f242c7c74b47f01568649a0a0e9ab82052ce
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
Reviewed-on: https://review.gerrithub.io/393716
Reviewed-by

blobfs: remove unnecessary opts.

Change-Id: I53c7f242c7c74b47f01568649a0a0e9ab82052ce
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
Reviewed-on: https://review.gerrithub.io/393716
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>

show more ...

blobfs: make the cluster size of blobfs configurable.

Change-Id: I26738d71316b8509cf8e98fee549a0745bb09bfa
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
Reviewed-on: https://review.gerrithub.

blobfs: make the cluster size of blobfs configurable.

Change-Id: I26738d71316b8509cf8e98fee549a0745bb09bfa
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
Reviewed-on: https://review.gerrithub.io/393715
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

blobfs: release the lock before send the flush request when try to write file.

This makes spdk_file_write consistent with _file_sync.
The lock should not be held across calls to send_request.

Chang

blobfs: release the lock before send the flush request when try to write file.

This makes spdk_file_write consistent with _file_sync.
The lock should not be held across calls to send_request.

Change-Id: I2a2987bbfe2688b520901267aa5a197bd55565a8
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
Reviewed-on: https://review.gerrithub.io/393599
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blob: change spdk_bs_iter_next parameter to spdk_blob *

Similar to previous change, the ** paradigm is a bit
problematic for asynchronous routines that could fail.

Signed-off-by: Jim Harris <james.

blob: change spdk_bs_iter_next parameter to spdk_blob *

Similar to previous change, the ** paradigm is a bit
problematic for asynchronous routines that could fail.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ife7748280482356c4c51a796817b71cd7bc7e479

Reviewed-on: https://review.gerrithub.io/391483
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>

show more ...

blob: change spdk_blob_close parameter to spdk_blob *

Using the ** paradigm is a bit problematic for asynchronous
routines that could fail. Currently we were inconsistent in
that some error paths w

blob: change spdk_blob_close parameter to spdk_blob *

Using the ** paradigm is a bit problematic for asynchronous
routines that could fail. Currently we were inconsistent in
that some error paths would zero the pointer while others
did not. So make this just a plain pointer, which simplifies
the API and its implementation.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I67147931c6e8350896a4505022a6a314655de3d3

Reviewed-on: https://review.gerrithub.io/391482
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>

show more ...

blob: rename bs_md functions

As part of clarifying the API and preparing for some
future changes, rename the following functions:

spdk_bs_md_create_blob => spdk_bs_create_blob
spdk_bs_md_open_blob

blob: rename bs_md functions

As part of clarifying the API and preparing for some
future changes, rename the following functions:

spdk_bs_md_create_blob => spdk_bs_create_blob
spdk_bs_md_open_blob => spdk_bs_open_blob
spdk_bs_md_delete_blob => spdk_bs_delete_blob
spdk_bs_md_iter_first => spdk_bs_iter_first
spdk_bs_md_iter_next => spdk_bs_iter_next

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I86bf792717b68379484a6108396bb891fe1c221e

Reviewed-on: https://review.gerrithub.io/391031
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>

show more ...

blob: rename spdk blob metadata functions

spdk_bs_md_resize_blob => spdk_blob_resize
spdk_bs_md_sync_blob => spdk_blob_sync_md
spdk_bs_md_get_xattr_value => spdk_blob_get_xattr_value
spdK_bs_md_get_

blob: rename spdk blob metadata functions

spdk_bs_md_resize_blob => spdk_blob_resize
spdk_bs_md_sync_blob => spdk_blob_sync_md
spdk_bs_md_get_xattr_value => spdk_blob_get_xattr_value
spdK_bs_md_get_xattr_names => spdk_blob_get_xattr_names
spdk_blob_md_set_xattr => spdk_blob_set_xattr
spdk_blob_md_remove_xattr => spdk_blob_remove_xattr
spdk_bs_md_close_blob => spdk_blob_close

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I99b593c03bc61354ba6aa5a446b0fc82df1e9ce2

Reviewed-on: https://review.gerrithub.io/391006
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>

show more ...

astyle: enforce braces around single-line statements

Require braces around all conditional statements, e.g.:

if (cond)
statement();

becomes:

if (cond) {
statement();
}

astyle: enforce braces around single-line statements

Require braces around all conditional statements, e.g.:

if (cond)
statement();

becomes:

if (cond) {
statement();
}

This is the style used through most of the SPDK code, but several
exceptions crept in over time. Add the astyle option to make sure we
are consistent.

Change-Id: I5a71980147fe8dfb471ff42e8bc06db2124a1a7f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/390914
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

log: rename SPDK_TRACE_* to SPDK_LOG_*

Disambiguate the log components from the trace functionality
(include/spdk/trace.h).

The internal spdk_trace_flag structure and related functions will be
rena

log: rename SPDK_TRACE_* to SPDK_LOG_*

Disambiguate the log components from the trace functionality
(include/spdk/trace.h).

The internal spdk_trace_flag structure and related functions will be
renamed in a later commit - this is just a find and replace on
SPDK_TRACE_* and SPDK_LOG_REGISTER_TRACE_FLAG().

Change-Id: I617bd5a9fbe35ffb44ae6020b292658c094a0ad6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/376421
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

blobfs: Add conf parse for blobfs.

This patch make the cache buffer shift of blobfs configurable. We
can configure the cache buffer shift according workload, it will
help to improve the performance.

blobfs: Add conf parse for blobfs.

This patch make the cache buffer shift of blobfs configurable. We
can configure the cache buffer shift according workload, it will
help to improve the performance.

Change-Id: Ied1c2e5d6fd9eaa7aa0759c03c654fcf2e77aa23
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
Reviewed-on: https://review.gerrithub.io/383381
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

blobfs: fix bstype debug prints

Add newlines to the debug logs.

Also dump the mismatched bstype via SPDK_TRACEDUMP() rather than as a
string, since it may not be null terminated.

Change-Id: Icad56

blobfs: fix bstype debug prints

Add newlines to the debug logs.

Also dump the mismatched bstype via SPDK_TRACEDUMP() rather than as a
string, since it may not be null terminated.

Change-Id: Icad567373c56f24aea550903a1370c3734465472
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/382855
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>

show more ...