blobfs: fix the wrong parameter usage for truncate API

fs_truncate_complete_cb() takes spdk_fs_request as
the input parameter.

Change-Id: I0413fecd30a1f7cb6528b502848fa999454ba638
Signed-off-by: Ch

blobfs: fix the wrong parameter usage for truncate API

fs_truncate_complete_cb() takes spdk_fs_request as
the input parameter.

Change-Id: I0413fecd30a1f7cb6528b502848fa999454ba638
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450891
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blobfs: add iov data structure to request

For existing filesystem APIs, SPDK can only support
1 data buffer, we add the iovec data structure here
so that it can support multiple buffer vectors in
fo

blobfs: add iov data structure to request

For existing filesystem APIs, SPDK can only support
1 data buffer, we add the iovec data structure here
so that it can support multiple buffer vectors in
following patches.

Change-Id: I26984f3ea985f349a5016060e0801e3989ce2fc6
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450722
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

blobfs: remove unused parameter in spdk_fs_cb_args

For now all the spdk_fs_cb_args used in blobfs are in the
spdk_fs_request context, so the field 'from_request' in
spdk_fs_cb_args can be removed.

blobfs: remove unused parameter in spdk_fs_cb_args

For now all the spdk_fs_cb_args used in blobfs are in the
spdk_fs_request context, so the field 'from_request' in
spdk_fs_cb_args can be removed.

Change-Id: I9b40cffeac8a673a87406ee89ad248939593806b
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450721
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blobfs: change read to use spdk_fs_request as input parameter

Change-Id: I778f054c82e786c8010d5fa9d4082af739855337
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.

blobfs: change read to use spdk_fs_request as input parameter

Change-Id: I778f054c82e786c8010d5fa9d4082af739855337
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450720
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blobfs: change readahead to take spdk_fs_request as input parameter

Change-Id: I00639a711a4f1637d24d58afb087e9d667d34c95
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://r

blobfs: change readahead to take spdk_fs_request as input parameter

Change-Id: I00639a711a4f1637d24d58afb087e9d667d34c95
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450719
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

blobfs: take the spdk_fs_request parameter for flush

Data structure spdk_fs_request has internal spdk_fs_cb_args
for each request, and for some APIs such as truncate they
all use spdk_fs_request as

blobfs: take the spdk_fs_request parameter for flush

Data structure spdk_fs_request has internal spdk_fs_cb_args
for each request, and for some APIs such as truncate they
all use spdk_fs_request as the parameter, so here we also
change flush to use the request parameter too.

Change-Id: Ic70b98cafe53d42c234f74b3703dc7797f210569
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450718
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blobfs: allocate flush request before appending to cache buffer

After the write data has been appended to cache buffer, the
allocation of flush request can return error, so we move this
allocation b

blobfs: allocate flush request before appending to cache buffer

After the write data has been appended to cache buffer, the
allocation of flush request can return error, so we move this
allocation before appending to the cache buffer.

Change-Id: Ia8baae8f3d88596561633bd8a5c4f83e554c72d7
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450981
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blobfs: spdk_fs_thread_ctx is no longer a channel

This also means that the RocksDB threads don't need
to be SPDK threads any more.

Change-Id: Icadd6dd446958ebf470ad8ab8239f5390942eb87
Signed-off-by

blobfs: spdk_fs_thread_ctx is no longer a channel

This also means that the RocksDB threads don't need
to be SPDK threads any more.

Change-Id: Icadd6dd446958ebf470ad8ab8239f5390942eb87
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449465
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

blobfs: Change channel type for synchronous operations

In the next step, this won't be a channel at all.

Change-Id: Ia8fe4da5b0b283e8dfc5c6477b84cfdd346d89a0
Signed-off-by: Ben Walker <benjamin.wal

blobfs: Change channel type for synchronous operations

In the next step, this won't be a channel at all.

Change-Id: Ia8fe4da5b0b283e8dfc5c6477b84cfdd346d89a0
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449464
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

blobfs: switch to spdk_*malloc()

spdk_dma_*malloc() is about to be deprecated.

Change-Id: Ib9fb94463a69ffd9660ab2c7fbe02826a7e06741
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Revi

blobfs: switch to spdk_*malloc()

spdk_dma_*malloc() is about to be deprecated.

Change-Id: Ib9fb94463a69ffd9660ab2c7fbe02826a7e06741
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/449796
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blobfs: Add trace info for request allocation.

Our blobfs allocates fixed number of reqs on three different channels
(the default size is 512), if the req cannot be allocated, our blobfs
related fun

blobfs: Add trace info for request allocation.

Our blobfs allocates fixed number of reqs on three different channels
(the default size is 512), if the req cannot be allocated, our blobfs
related function will do nothing.

With this debug info added, we can easily know the issues.
If the fixed number of requests are not enough, we need to change the
design, using malloc/free together with the fixed number of requests.

During high presure test, I think that if the reqs are not allocated,
it will cause the sync, flush related issues.

Change-Id: Ied8cd5197c4f87b861bd7dc09edd5a095fdcc802
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448596
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

blobfs: Add the return value check for calling cache_append_buffer

Reason: We need to detect this issue. Otherwise, it will
cause null pointer issue later

Change-Id: I4d9aec0570e0b46274ebf3b9642b47

blobfs: Add the return value check for calling cache_append_buffer

Reason: We need to detect this issue. Otherwise, it will
cause null pointer issue later

Change-Id: I4d9aec0570e0b46274ebf3b9642b4727049a4a3b
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448594
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>

show more ...

blobfs: remove the assert.

The caller function needs to judge whether the buf is
allocated. Putting assert here is easy for debugging,
but not be friendly for real use.

Change-Id: I3c8e66bc95cbd2f9

blobfs: remove the assert.

The caller function needs to judge whether the buf is
allocated. Putting assert here is easy for debugging,
but not be friendly for real use.

Change-Id: I3c8e66bc95cbd2f9bc87f26335f249c32e339da4
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448697
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blobfs: fix the length value of file.

In function spdk_fs_file_stat_async, the
stat.size = f->append_pos >= f->length ? f->append_pos : f->length;

but in spdk_file_get_length, we return f->length.

blobfs: fix the length value of file.

In function spdk_fs_file_stat_async, the
stat.size = f->append_pos >= f->length ? f->append_pos : f->length;

but in spdk_file_get_length, we return f->length.

So generally, it should all use the same method to return the file
length, and this patch will fix this issue.

Change-Id: Idb9aa9e737711fcd48ac0075c7f7ffed825fe3b0
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/439627
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

log: remove "trace" from internal API

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8b1c0d4b00d5d41aae89d3b33f18d1ae957567dc

Reviewed-on: https://review.gerrithub.io/435344
Teste

log: remove "trace" from internal API

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8b1c0d4b00d5d41aae89d3b33f18d1ae957567dc

Reviewed-on: https://review.gerrithub.io/435344
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

blobfs: align rw buffer to lba of device underneath

Previously the alignment was to 4k due to page size of blobstore being minmal
size to write to blobs. Now that io_unit is used, alignment can be c

blobfs: align rw buffer to lba of device underneath

Previously the alignment was to 4k due to page size of blobstore being minmal
size to write to blobs. Now that io_unit is used, alignment can be changed to that.

Change-Id: Ib48182e299dad9d9587ae20b97afe683d53933cb
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/425788
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

blobfs: modify blobfs to use io_unit size instead of constant 4096 as blocklen

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I40d8532494f75f8175aa4730e723c4d0905a486c

Revi

blobfs: modify blobfs to use io_unit size instead of constant 4096 as blocklen

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I40d8532494f75f8175aa4730e723c4d0905a486c

Reviewed-on: https://review.gerrithub.io/425356
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

thread: Add a name parameter to spdk_register_io_device

This is a string name used for debugging only.

Change-Id: I9827f0e6c83be7bc13951c7b5f0951ce6c2a1ece
Signed-off-by: Ben Walker <benjamin.walke

thread: Add a name parameter to spdk_register_io_device

This is a string name used for debugging only.

Change-Id: I9827f0e6c83be7bc13951c7b5f0951ce6c2a1ece
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424127
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>

show more ...

blobfs: cleanup file based resources when unload blobfs

Change-Id: Iea3067d62cd0955c385ce6112f83ef7c793edfb2
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrit

blobfs: cleanup file based resources when unload blobfs

Change-Id: Iea3067d62cd0955c385ce6112f83ef7c793edfb2
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/423607
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

blobfs: remove redundant function __file_close_done

Change-Id: I5264467da9b3ae52c7cb4d2060f0f24551546c15
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.

blobfs: remove redundant function __file_close_done

Change-Id: I5264467da9b3ae52c7cb4d2060f0f24551546c15
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/423600
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

blobfs: add a new API to return file's unique ID

Change-Id: I066085a0606d64a0d95ab2d28340aa35d83efdf7
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/

blobfs: add a new API to return file's unique ID

Change-Id: I066085a0606d64a0d95ab2d28340aa35d83efdf7
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/423504
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

blobfs: return the error code in file callbacks

Change-Id: I9fe730e5a34608dbc7def3a5c19a891fbcc0d9e9
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/4

blobfs: return the error code in file callbacks

Change-Id: I9fe730e5a34608dbc7def3a5c19a891fbcc0d9e9
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/423195
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

blobfs: exit the app after the failure of memory allocation

When testing blobfs, the allocation of cache buffer may fail
without any error information, users may get segment fault
error after that.

blobfs: exit the app after the failure of memory allocation

When testing blobfs, the allocation of cache buffer may fail
without any error information, users may get segment fault
error after that.

Change-Id: I6dc8ee4b93c6a9109aba193f599197caa4c4c383
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/422943
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blobfs: add the check for cache_buffer

Though the buffer can be allocated in the normal case,
however we still need to check it.

Change-Id: I483c7bd083c24590295e44473674c7b8ed30e9da
Signed-off-by:

blobfs: add the check for cache_buffer

Though the buffer can be allocated in the normal case,
however we still need to check it.

Change-Id: I483c7bd083c24590295e44473674c7b8ed30e9da
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/420462
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

blobfs: change the return type of spdk_file_truncate

Change-Id: I29347d7b7be0237d7d604c7cd7363fa08a0cd025
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/42

blobfs: change the return type of spdk_file_truncate

Change-Id: I29347d7b7be0237d7d604c7cd7363fa08a0cd025
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/420133
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>

show more ...