lib/util: Make md5 API public

Make md5 realted functions in iscsi lib public.
Interface remained unchaged except of spdk_ prefix
added to func names. Implementation slightly updated
with unlikely()

lib/util: Make md5 API public

Make md5 realted functions in iscsi lib public.
Interface remained unchaged except of spdk_ prefix
added to func names. Implementation slightly updated
with unlikely() hints.

Signed-off-by: Alexey Marchuk <alexeymar@nvidia.com>
Change-Id: I8bff008aad0c44b639b9249e1b48f3a6f9421f74
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23133
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Ben Walker <ben@nvidia.com>

show more ...

env: add SPDK_ENV_NUMA_ID_ANY and replace socket_id with numa_id

We will try to avoid further proliferation of "SOCKET_ID" to refer
to a NUMA socket ID moving forward, and just use "NUMA_ID" to avoi

env: add SPDK_ENV_NUMA_ID_ANY and replace socket_id with numa_id

We will try to avoid further proliferation of "SOCKET_ID" to refer
to a NUMA socket ID moving forward, and just use "NUMA_ID" to avoid
confusion with TCP sockets.

Change all of the existing in-tree SPDK_ENV_SOCKET_ID_ANY uses to
SPDK_ENV_NUMA_ID_ANY, but keep the old #define around, at least for
now. Also change all 'socket_id' parameters to 'numa_id'.

We still have spdk_env_get_socket_id(), we will need to keep this
but next patch will add spdk_env_get_numa_id().

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: Idc31c29e32b708c24d88f9c6fecaf9a99e34ba1e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24607
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <ben@nvidia.com>
Community-CI: Mellanox Build Bot

show more ...

iscsi: add trace owner support

Note: spdk_trace_record() in the fuzzing app really shouldn't be
there, so this patch removes it.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I4b0da

iscsi: add trace owner support

Note: spdk_trace_record() in the fuzzing app really shouldn't be
there, so this patch removes it.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I4b0dad56775beee3563d15786c4d0ccd41142997
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22892
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

fuzz/llvm_vfio_fuzz: limit length of generated data to `bytes_per_cmd`

Change-Id: I716de9e11ccb417ab493880c5a75467373117ad7
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Reviewe

fuzz/llvm_vfio_fuzz: limit length of generated data to `bytes_per_cmd`

Change-Id: I716de9e11ccb417ab493880c5a75467373117ad7
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22715
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

fuzz/llvm_vfio_fuzz: fix `fuzz_vfio_user_irq_set` incorrect data length

`fuzz_vfio_user_irq_set` requries 13 bytes instead of 12.

Change-Id: I3f041110591c5c0e5e29a96224d89293a7745234
Signed-off-by:

fuzz/llvm_vfio_fuzz: fix `fuzz_vfio_user_irq_set` incorrect data length

`fuzz_vfio_user_irq_set` requries 13 bytes instead of 12.

Change-Id: I3f041110591c5c0e5e29a96224d89293a7745234
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22714
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

fuzz/llvm_vfio_fuzz: terminate fuzzer driver with SIGSEGV

Use `spdk_app_start_shutdown`to ensure that fuzzer driver is
terminated when SPDK app exits.

Change-Id: I07029ac42c0e22a7888873cfba248b367

fuzz/llvm_vfio_fuzz: terminate fuzzer driver with SIGSEGV

Use `spdk_app_start_shutdown`to ensure that fuzzer driver is
terminated when SPDK app exits.

Change-Id: I07029ac42c0e22a7888873cfba248b367ec29eb5
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22309
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

apps: set NULL as RPC address in some apps

Set opts.rpc_addr to NULL in SPDK applications that usually do not
benefit from starting RPC interface. These include programs in app,
examples and test di

apps: set NULL as RPC address in some apps

Set opts.rpc_addr to NULL in SPDK applications that usually do not
benefit from starting RPC interface. These include programs in app,
examples and test directories.

Some of the tests now require that we explicitly set RPC address,
because some apps no longer use the default SPDK address.

Change-Id: I417a3a756581f44c711257e6d2408d933b40386a
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22634
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot

show more ...

llvm/vfio: Suppress checking leaks for `spdk_nvme_ctrlr_alloc_io_qpair`

LSAN has problems detecting references stored in DPDK-managed memory,
causing leak sanitizer to reports leaking memory incorre

llvm/vfio: Suppress checking leaks for `spdk_nvme_ctrlr_alloc_io_qpair`

LSAN has problems detecting references stored in DPDK-managed memory,
causing leak sanitizer to reports leaking memory incorrectly.

This behavior is specific to fuzzers, since
`__lsan_do_recoverable_leak_check` is called in runtime after each
`TestOneInput`, and not only at the end of the program.

Fixes issue #3258.

Change-Id: I13cc9315f0c26a0f03bb9cf6cf6f0128943f11da
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21897
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

test/fuzz: include env_dpdk_rpc library

This patch includes env_dpdk_rpc library when vfio and
nvmf fuzzer apps are built, so it is possible to use
certain RPCs from this library.
We can now use cal

test/fuzz: include env_dpdk_rpc library

This patch includes env_dpdk_rpc library when vfio and
nvmf fuzzer apps are built, so it is possible to use
certain RPCs from this library.
We can now use call to env_dpdk_get_mem_stats to get memory
usage information from DPDK.

Example:
sudo scripts/rpc.py -s /var/tmp/spdk0.sock env_dpdk_get_mem_stats
{
"filename": "/tmp/spdk_mem_dump.txt"
}

This RPC call stores memory information in /tmp/spdk_mem_dump.txt
file. Tracking changes in e.g. memory allocated on huge pages
can be useful during memory leaks debugging.

Change-Id: Ic0623c9edaeea97a6a256f502b1a022a89d58cfc
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21895
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

nvme_fuzz: add call to spdk_app_stop(-1) on errors

When a fuzzer is started, to properly detect errors when
connecting to a controller, 'return -1' is not enough.
We also need to call spdk_app_stop(

nvme_fuzz: add call to spdk_app_stop(-1) on errors

When a fuzzer is started, to properly detect errors when
connecting to a controller, 'return -1' is not enough.
We also need to call spdk_app_stop(-1) when an error
is detected to signal the fuzzer we have failed.

I have encountered such a situation when due to memory
leak, setting up of the controller queues failed
(no memory for structures). Return -1 didin't caused
the fuzzer to stop the test, it was keep trying to call
TestOneInput function for the whole scheduled test
time (16 hrs).
Adding spdk_app_stop(-1) allows the fuzzer to detect the
error so the Jenkins job can fail early, just after the error
is detected.

Change-Id: I08ca4de7d21b13df3bcf93a1ccffbbec32ad9f4a
Signed-off-by: Marcin Spiewak <marcin.spiewak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21894
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

util: use spdk_posix_file_load_from_name

Use new function accross SPDK, instead of reading file contents
locally, so users calling this function will be freed from handling
opening and closing the f

util: use spdk_posix_file_load_from_name

Use new function accross SPDK, instead of reading file contents
locally, so users calling this function will be freed from handling
opening and closing the files.

Change-Id: I7a7a9aa1151500f6abd67e894c1ba2b6891cc966
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21829
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

fuzz/llvm_vfio_fuzz: Adjust array index to avoid overflow

fuzz_vfio_user_set_msix test uses 9 bytes for a single test, this
patch fixes `data` array indexes to match the requested size.

Change-Id:

fuzz/llvm_vfio_fuzz: Adjust array index to avoid overflow

fuzz_vfio_user_set_msix test uses 9 bytes for a single test, this
patch fixes `data` array indexes to match the requested size.

Change-Id: I19762de62173f885ee28a5a4b2a8e500c8c6dd81
Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21820
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Marcin Spiewak <marcin.spiewak@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

include: Remove duplicate includes in source files

Change-Id: I7dd6ae6fa11603a956c3d178b9b23d2c755913d1
Signed-off-by: Pierre Lestringant <plestringant@kalray.eu>
Reviewed-on: https://review.spdk.io

include: Remove duplicate includes in source files

Change-Id: I7dd6ae6fa11603a956c3d178b9b23d2c755913d1
Signed-off-by: Pierre Lestringant <plestringant@kalray.eu>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/20106
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

app/fuzz: enable detect leaks in LLVM fuzzer runtime

Since ASan can now recognize memory allocated on hugepages, false
leaks should not be reported when reference is store on hugepage.

Signed-off-b

app/fuzz: enable detect leaks in LLVM fuzzer runtime

Since ASan can now recognize memory allocated on hugepages, false
leaks should not be reported when reference is store on hugepage.

Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: I56954ab770994f423765f17d70dd2b56672258af
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/20639
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

llvm_vfio_fuzz: add fuzzing of msi-x interupts registers

Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: Idfc77476fb48cd5b23e369efea4f99ada9556147
Reviewed-on: https://

llvm_vfio_fuzz: add fuzzing of msi-x interupts registers

Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: Idfc77476fb48cd5b23e369efea4f99ada9556147
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18501
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Ben Walker <ben@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

test/llvm_vfio_fuzz: fuzz vfio-user message command

This patch introduce fuzzing following message command:
- fuzz_vfio_user_get_region_info,
- fuzz_vfio_user_dma_map,
- fuzz_vfio_user_dma_unmap.

test/llvm_vfio_fuzz: fuzz vfio-user message command

This patch introduce fuzzing following message command:
- fuzz_vfio_user_get_region_info,
- fuzz_vfio_user_dma_map,
- fuzz_vfio_user_dma_unmap.

Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: I61f0b8ee738298b194d45d7635bbdbdd2aa810b3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18500
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <ben@nvidia.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>

show more ...

app/fuzz: disable detect_leaks during fuzzer runtime

Detecting leaks while fuzzing cause asan to report incorrect memory
leaks, this behavior is cause by asan being disable for dpdk so every
memory

app/fuzz: disable detect_leaks during fuzzer runtime

Detecting leaks while fuzzing cause asan to report incorrect memory
leaks, this behavior is cause by asan being disable for dpdk so every
memory that is allocated on heap and then store on hugepages is
reported as leaking even if there are correctly freed at the end.

This is a temporary solution until asan is enabled in dpdk

Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: I2deecf39a2ec4c5345f626746280454e19088055
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18411
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

llvm_vfio_fuzz: check if app thread still exist before spdk_app_stop

Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: Ia9ffc8580174999366b49b22122e3561cf643395
Reviewed-

llvm_vfio_fuzz: check if app thread still exist before spdk_app_stop

Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: Ia9ffc8580174999366b49b22122e3561cf643395
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17886
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

app/fuzz: discard randoms of insufficient length

LLVMFuzzerRunDriver does not allow to specify minimum input length,
return immediately when data insufficient.

Signed-off-by: Sebastian Brzezinka <s

app/fuzz: discard randoms of insufficient length

LLVMFuzzerRunDriver does not allow to specify minimum input length,
return immediately when data insufficient.

Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: I306e1774b17b04108f2454b2fdaadb4d912bd274
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17884
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

fuzz/llvm: provide a prefix to use when saving artifacts

Save crash files and other artifacts in `$output_dir/llvm/`

Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: I2

fuzz/llvm: provide a prefix to use when saving artifacts

Save crash files and other artifacts in `$output_dir/llvm/`

Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: I2ff82b414592cc492b79c9178b7257b2e87440b5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15827
Reviewed-by: Michal Berger <michal.berger@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

llvm_vfio_fuzz: handle thread create failure

In case of `pthread_create` or `spdk_thread_create` failed stop
spdk app with `-1` error code

Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@in

llvm_vfio_fuzz: handle thread create failure

In case of `pthread_create` or `spdk_thread_create` failed stop
spdk app with `-1` error code

Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: Id5d0f6716917f42e06fbda7db9285deb320e309a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16338
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

llvm_vfio_fuzz: detach io ctrlr in new thread

Undetached poller cause timeout when `spdk_thread_exit` has
been called and detaching it in same thread make poller to stuck
on `spdk_nvme_detach_async

llvm_vfio_fuzz: detach io ctrlr in new thread

Undetached poller cause timeout when `spdk_thread_exit` has
been called and detaching it in same thread make poller to stuck
on `spdk_nvme_detach_async`.

`spdk_nvme_detach_async` call `nvme_pcie_ctrlr_delete_io_qpair`
which is synchronous making it to wait for response indefinitly.

Fixes #2798.

Signed-off-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Change-Id: Id500841f9c8fd9847e64805864cb136c74b003f1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15650
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...