init: remove spdk_subsystem_init_from_json_config

This was deprecated and marked for removal in v24.09 release, so it
is time to remove it.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change

init: remove spdk_subsystem_init_from_json_config

This was deprecated and marked for removal in v24.09 release, so it
is time to remove it.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: Ib76e7acd61c4067711be1a7c22aa0b43aba04f4f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24664
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot

show more ...

general: fix misspells and typos

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: Iab206ef526eb7032c6681a3145450010c91705a4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+

general: fix misspells and typos

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: Iab206ef526eb7032c6681a3145450010c91705a4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24120
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

rpc: read config space in two steps

spdk_pci_device_cfg_read() will fail if trying to read beyond the
available config space, so this ensures that devices that don't have
extended config space are d

rpc: read config space in two steps

spdk_pci_device_cfg_read() will fail if trying to read beyond the
available config space, so this ensures that devices that don't have
extended config space are dumped in the RPC.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ib625aaf5d457305d19cf68396e78a344a429304b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23362
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz@tzawadzki.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

lib/init: do not fail if missing RPC's subsystem in JSON config doesn't exist in app

This allows us to run configs generated from spdk_tgt in less featured
apps like bdevperf. It is fine to just ign

lib/init: do not fail if missing RPC's subsystem in JSON config doesn't exist in app

This allows us to run configs generated from spdk_tgt in less featured
apps like bdevperf. It is fine to just ignore these sections of the JSON
config file - if the associated subsystem isn't linked into the application,
it wouldn't be used anyways so we can just ignore those parts of the config
file. That is a lot different than if the subsystem does exist, and one of
its RPCs fails.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: Ie9fe4b3471488aa1534844ad73a0932c81110760
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22780
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot

show more ...

init: add spdk_subsystem_exists()

We can use this to determine if any given subsystem has been linked into
the application.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: Ib2af6f0b54

init: add spdk_subsystem_exists()

We can use this to determine if any given subsystem has been linked into
the application.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: Ib2af6f0b5480091476c44e85404320ada213e251
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22779
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

lib/event: make SPDK app exit upon RPC server launch failure

Whenever an application cannot start a server, it will print an error,
but not exit upon detecting that condition, so we implicitly disab

lib/event: make SPDK app exit upon RPC server launch failure

Whenever an application cannot start a server, it will print an error,
but not exit upon detecting that condition, so we implicitly disable
the server by just ignoring its inability to launch.

We should not allow RPC servers to fail their initialization while
allowing SPDK app to continue working. Instead, we should exit the
app, whenever sever failed to start on a valid RPC address.

Change-Id: I10a779c71ed5101d12d69383d646f44bad6d9565
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22153
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot

show more ...

lib/init: no longer accept NULL listen_address

With addition of skipping RPC server initialization mechanism,
applications may decide to opt out of running RPC interface.
This feature allows us to b

lib/init: no longer accept NULL listen_address

With addition of skipping RPC server initialization mechanism,
applications may decide to opt out of running RPC interface.
This feature allows us to be more explicit, so implicit skip
via NULL socket address is no longer necessary.

Change-Id: Ic1365a9807670472ab9d4c7a1bbc6a846e02b5fb
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22636
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

lib/init: do not fail on missing RPC when stop_on_error is not set

SPDK apps support --json-ignore-init-errors option, to allow
continuing past RPCs that might fail.

But this was only working for R

lib/init: do not fail on missing RPC when stop_on_error is not set

SPDK apps support --json-ignore-init-errors option, to allow
continuing past RPCs that might fail.

But this was only working for RPCs that actually exist, and
then fail execution. It did not work for RPCs that didn't exist
at all. For example, if you save a JSON config from a running
spdk_tgt, it will include a lot of RPCs (nvmf, iscsi) that
don't exist in an app like bdevperf.

So fix up app_json_config_load_subsystem_config_entry() so that it
just continues to the next RPC if it finds an RPC call that doesn't
exist.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I60e6b55c7eda088725bf7486d3f67eeb5ddc3538
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22776
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot

show more ...

lib/init: recurrence => recursion

"recursion" more accurately describes why we are using spdk_thread_send_msg()
in these places.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I70663

lib/init: recurrence => recursion

"recursion" more accurately describes why we are using spdk_thread_send_msg()
in these places.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I7066345ad4720efe18e166a1e107b15c0255bd4d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22778
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot

show more ...

util: use spdk_posix_file_load_from_name

Use new function accross SPDK, instead of reading file contents
locally, so users calling this function will be freed from handling
opening and closing the f

util: use spdk_posix_file_load_from_name

Use new function accross SPDK, instead of reading file contents
locally, so users calling this function will be freed from handling
opening and closing the files.

Change-Id: I7a7a9aa1151500f6abd67e894c1ba2b6891cc966
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21829
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

lib/init: deprecate spdk_subsystem_init_from_json_config

Change-Id: If03fe50ad45a923c129e8113c753fbe5b3a4db82
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.s

lib/init: deprecate spdk_subsystem_init_from_json_config

Change-Id: If03fe50ad45a923c129e8113c753fbe5b3a4db82
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21322
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

lib/init: Add new API to load JSON config

Add a new function to carry out loading JSON configuration per rpc
state. This API takes raw JSON data and its size to avoid reading
the same file multiple

lib/init: Add new API to load JSON config

Add a new function to carry out loading JSON configuration per rpc
state. This API takes raw JSON data and its size to avoid reading
the same file multiple times inside lib/init, which is problematic
in cases where subsystem configuration is piped to SPDK
(e.g. program substitution), because reading bytes from such a file
object consumes them. The user calling this function will be
responsible for managing JSON configuration file.

Change-Id: I6e369b8dba1cb8fe7449142e5028bba841477d2d
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21194
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

init: add pausing and resuming RPC server polling

Add APIs to pause and resume polling on specified server address.
This will be useful in following patches, where one RPC server is
launched for use

init: add pausing and resuming RPC server polling

Add APIs to pause and resume polling on specified server address.
This will be useful in following patches, where one RPC server is
launched for user interaction (e.g. via rpc.py script) and other
will start to configure subsystems from JSON config.

Change-Id: I08ce0107a59e50c5c83a49fcd34109584819fd85
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21193
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

lib/init: manage multiple RPC servers

RPC library now encapsulates RPC server specific data, so it is
possible to run multiple servers at the same time. To utilize that
in init poll each server that

lib/init: manage multiple RPC servers

RPC library now encapsulates RPC server specific data, so it is
possible to run multiple servers at the same time. To utilize that
in init poll each server that was launched through this library and
tie registering/unregistering of the poller to the existence of at
least one RPC server.

Modifications to existing APIs' behavior:
* spdk_rpc_initialize() launches a new server instead of
overwriting existing one,
* spdk_rpc_finish() stops all existing servers instead of
stopping a single, global server instance,

Addition of new API:
* spdk_rpc_server_finish() will stop a server listening
on a given address (if it exists).

Change-Id: I9c794e31f7ee27eb5607925b6c7a8446387670b7
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21080
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot

show more ...

lib/init: split spdk_rpc_initialize into smaller functions

Next patch will introduce a new API to handle initializing multiple
instances of servers (instead of a single instance per application).
Mo

lib/init: split spdk_rpc_initialize into smaller functions

Next patch will introduce a new API to handle initializing multiple
instances of servers (instead of a single instance per application).
Most of the new function will overlap with the existing code, so we
should move it to smaller functions to avoid duplication.

This also reduces the number of operations performed by
spdk_rpc_initialize().

Change-Id: Ib898af5e48399dbe5799e11446ac02f536a13fd2
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21320
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

so_ver: increase all major versions after SPDK 24.01

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to preve

so_ver: increase all major versions after SPDK 24.01

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.

Increasing major so version after LTS, allows the future releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.

This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components

Due to patch below being introduced after v24.01 code freeze,
bdev lib version should not be increased:
61623c5ca9 bdev: add spdk_bdev_io padding in place of io_submit_ch

Short reference to how the versions were changed:
MAX=$(git grep "SO_VER := " | cut -d" " -f 3 | sort -ubnr | head -1)
for((i=$MAX;i>0;i-=1)); do find . -name "Makefile" -exec \
sed -i -e "s/SO_VER := $i\$/SO_VER := $(($i+1))/g" {} +; done
find . -name "Makefile" -exec \
sed -i -e "s/SO_MINOR := .*/SO_MINOR := 0/g" {} +

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I0166d6be6c17bbbf985d48b5dfcb36f1d4af1b48
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21582
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

rpc: initialize state to startup

Set lib/rpc g_rpc_state to SPDK_RPC_STARTUP
upon declaration. This renders spdk_rpc_set_state()
in spdk_rpc_initialize() unnecessary and reduces
number of operations

rpc: initialize state to startup

Set lib/rpc g_rpc_state to SPDK_RPC_STARTUP
upon declaration. This renders spdk_rpc_set_state()
in spdk_rpc_initialize() unnecessary and reduces
number of operations it performs.

This change also opens our code to possibility
of calling spdk_rpc_initialize multiple times
in the future without resetting RPC state to
startup each time (spdk_rpc_initialize will be
called multiple times in the following patches).

This is the start of a series of changes that
implement SPDK application initialization with
both JSON config file and subsystem initialiation
delay (--wait-for-rpc).
To achieve this goal the series focuses on a number
of key changes:
a) introducing new APIs to manage server objects
instead of working on a single global instance,
b) deprecating current lib/rpc and lib/init server
related functions,
c) moving server instances from lib/rpc to a list
in lib/init.

Change-Id: I766f0149b9c12940faef3562f2795f5f1e1381db
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21079
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

init: Add options to spdk_rpc_initialize() for log_file/level params

Add log_file and log_level parameters to spdk_rpc_initialize() via a
new options structure spdk_rpc_opts.

spdk_rpc_initialize()

init: Add options to spdk_rpc_initialize() for log_file/level params

Add log_file and log_level parameters to spdk_rpc_initialize() via a
new options structure spdk_rpc_opts.

spdk_rpc_initialize() calls spdk_jsonrpc_set_log_file() and
_set_log_level() with these parameters.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I084ed5b76849e06ab3419a115234d2d9b44f820b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18830
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot

show more ...

thread: add spdk_thread_is_app_thread()

This simply returns true if the current spdk_thread
is the app thread.

This is a simpler replacement for:

spdk_thread_get_app_thread() == spdk_get_thread()

thread: add spdk_thread_is_app_thread()

This simply returns true if the current spdk_thread
is the app thread.

This is a simpler replacement for:

spdk_thread_get_app_thread() == spdk_get_thread()

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib8754d4ecb91ee8bfb024e8c0b74c42bb8282b32
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18355
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

init: rewrite subsystem_sort

Commit aaba5d introduced a build warning with some
compilers. While fixing it, I realized the function was
difficult to immediately understand. So in addition to fixing

init: rewrite subsystem_sort

Commit aaba5d introduced a build warning with some
compilers. While fixing it, I realized the function was
difficult to immediately understand. So in addition to fixing
the build warning, I also made the following changes:

* Improved names for local variables
* Use TAILQ_INIT for local TAILQ instead of TAILQ_HEAD_INITIALIZER.
* Add comments explaining more clearly what the nested loops are
doing.
* Use TAILQ_SWAP instead of a FOREACH + REMOVE + INSERT.

Fixes: aaba5d ("subsystem: Gather list changed conditions.")
Fixes issue #2978.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic8740b5706537938d62a0acfac62625b2424b85f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17496
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Mike Gerdts <mgerdts@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>

show more ...

subsystem: Gather list changed conditions.

Just remove the duplicated Code and make the
Conditions for g_subsystems list to subsystems_list
together.

Signed-off-by: yidong0635 <dongx.yi@intel.com>

subsystem: Gather list changed conditions.

Just remove the duplicated Code and make the
Conditions for g_subsystems list to subsystems_list
together.

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I011b550b83d32580bfd25130dab9e44bcbdc1daf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13753
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

so_ver: increase all major versions

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where

so_ver: increase all major versions

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.

Increasing major so version after LTS, allows the future releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.

Disabled test for increasing SO version without ABI change, as
that is goal of this patch. This check shall be removed with SPDK 23.05
release.
Looks like this was left over from prior LTS, to avoid that
make sure it is only skipped when running against v23.01.x as latest
release.

This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components
- removes suppressions for ABI tests


Short reference to how the versions were changed:
MAX=$(git grep "SO_VER := " | cut -d" " -f 3 | sort -ubnr | head -1)
for((i=$MAX;i>0;i-=1)); do find . -name "Makefile" -exec \
sed -i -e "s/SO_VER := $i\$/SO_VER := $(($i+1))/g" {} +; done
find . -name "Makefile" -exec \
sed -i -e "s/SO_MINOR := .*/SO_MINOR := 0/g" {} +

Change-Id: I3e5681802c0a5ac6d7d652a18896997cd07cc8bf
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16419
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...