dpdk: add adjustments for recent rte_power changes

There were a couple of changes to the rte_power library:
1) Hardware-specific code was moved out of rte_ower into individual
drivers under the

dpdk: add adjustments for recent rte_power changes

There were a couple of changes to the rte_power library:
1) Hardware-specific code was moved out of rte_ower into individual
drivers under the drivers/power directory.
2) The header file was renamed rte_power.h -> rte_power_cpufreq.h.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Iac8aeb99423a21905a8bea744ccf662539b26da0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25424
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Michal Berger <michal.berger@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>

show more ...

gscheduler: fix divide-by-zero error

If gscheduler is activated and gets a chance to balance() before all
cores have an spdk_thread assigned, the total_tsc for that core would
be 0, and calculate_bu

gscheduler: fix divide-by-zero error

If gscheduler is activated and gets a chance to balance() before all
cores have an spdk_thread assigned, the total_tsc for that core would
be 0, and calculate_busy_pct() would divide by zero causing a SIGFPE.

Fixes issue #3550.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I5b21bbbd3fb7448edd6d929a36a77b9fbd9d0e43
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25200
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Alliswell <hisunzhenliang@outlook.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Changpeng Liu <changpeliu@tencent.com>

show more ...

lib/event: Add support for core isolation in scheduling

Added new 'scheduler_set_options' RPC.
1. isolated_core_mask: Users are allowed to select cores to isolate during scheduling
2. scheduling_cor

lib/event: Add support for core isolation in scheduling

Added new 'scheduler_set_options' RPC.
1. isolated_core_mask: Users are allowed to select cores to isolate during scheduling
2. scheduling_core: Users are allowed to select main core of scheduling
This RPC can only be called before SPDK subsystems have been initialized.
These parameters can only be set once.

The following options are for general mechanisms.
For example, dynamic scheduler now allows specific cores to be isolated from scheduling
through the newly added isolated_core_mask. This means that no threads can enter
or leave the cores designated as isolated cores.

Core isolation can be useful in the following situation:
The user may want the main core to be dedicated to orchestration tasks(such as RPCs, etc.)
and does not want any IO work to be assigned to it while running dynamic scheduler.
These changes allow user to dedicate main thread for RPC, assign nvmf poll groups to
non-main thread cores, and then ensure that dynamic(or any other) scheduler will
not place any other spdk_threads on the main core.

Change-Id: Idb5e564150d014cec7106d48d609d83490561a94
Signed-off-by: SeungYeon Shin <syeon.shin@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23156
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>

show more ...

scheduler/gscheduler: decide freq on highest util of SMT siblings

The CPU frequency governor can only take effect if all SMT siblings
agree on the same frequency. Assume there are two SMT siblings,

scheduler/gscheduler: decide freq on highest util of SMT siblings

The CPU frequency governor can only take effect if all SMT siblings
agree on the same frequency. Assume there are two SMT siblings,
one is 80% busy, the other is 20% busy, we want to make the decision
on frequency for both cores based on the 80% busy. This ensures the
work on that 80% core is not impacted by the other core trying to set
20%.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: Ifffffaa90dd04285467235b55a3009c21fe45d57
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23724
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

scheduler/gscheduler: break out busy_pct calc to a helper function

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I0d87a31a6f12387c94aa315d6623230bcbd096eb
Reviewed-on: https://review

scheduler/gscheduler: break out busy_pct calc to a helper function

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I0d87a31a6f12387c94aa315d6623230bcbd096eb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23701
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

scheduler/dynamic: adjust frequency when core goes to sleep or wakes

When all spdk_threads have been moved to the scheduling core because
they are all idle, we will try to reduce frequency on the sc

scheduler/dynamic: adjust frequency when core goes to sleep or wakes

When all spdk_threads have been moved to the scheduling core because
they are all idle, we will try to reduce frequency on the scheduling
core if it is still idle. But the frequency changes don't take effect
if it has an SMT sibling which is still set to max frequency.

So when we have moved the last thread off of a core, set its frequency
to min. And conversely, when we move the first thread onto a core,
set its frequency to max.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I034194d553aa678d387b1b5448631ba4ffcbb3d3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23649
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

dpdk_governor: don't load if app core mask has subset of SMT siblings

SPDK schedulers and governors don't understand SMT yet, but let's at
least disable the case where the app core mask contains one

dpdk_governor: don't load if app core mask has subset of SMT siblings

SPDK schedulers and governors don't understand SMT yet, but let's at
least disable the case where the app core mask contains one SMT core
but not its sibling. Upcoming patches will teach schedulers about
SMT to make better decisions, but we can never really make
governor changes if SPDK only is managing one core out of a set of
siblings.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: Ie2a7403e2b5a7c3ab47023f01c2edec5de020ab6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23528
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

dpdk_governor: use rte_power_set_env() to reduce noisy log messages

If an application doesn't set the power env explicitly, DPDK will
just iterate through them one at a time until it finds one that

dpdk_governor: use rte_power_set_env() to reduce noisy log messages

If an application doesn't set the power env explicitly, DPDK will
just iterate through them one at a time until it finds one that
works. This clutters the log with error messages about the ones that
didn't work. So use the rte_power_check_env_supported() API to
iterate through them ourselves (just like DPDK does) and set the
first one that we find.

From what I can see, only one can ever work. So SPDK iterates through
the DPDK enum in order, even though DPDK does them out of order when
it needs to find one because the user didn't set one.

While here, set the DPDK rte_power log level to WARNING, which also
eliminates a ton of noisy messages when enabling the governor. While
adding this, we noticed that cryptodev flag here is setting WARNING
but the comment says ERR, so fix that comment while here.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I5cb2552d3b41235884577669bd574f88ca944f2a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23408
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

gscheduler: add percentage variables

Currently the up/down decision is based on comparing idle v. busy,
which implicitly means comparing against 50%.

Likewise the min/max decisions are based on com

gscheduler: add percentage variables

Currently the up/down decision is based on comparing idle v. busy,
which implicitly means comparing against 50%.

Likewise the min/max decisions are based on comparing idle v. busy/100
(or vice versa), which implicitly means comparing against 99% (or 1%).

Upcoming patches will allow for configuring these values, so in
preparation for that, create variables for these percentages and do
calculations based on those.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I9f8253356a9b8fec417519cca38fa0b625ae4f1d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23384
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

gscheduler: use 1% instead of 0.1% for min/max calculations

Upcoming patches will allow for configuring gscheduler parameters.
There doesn't seem to be a need to allow for .1% granularity, so change

gscheduler: use 1% instead of 0.1% for min/max calculations

Upcoming patches will allow for configuring gscheduler parameters.
There doesn't seem to be a need to allow for .1% granularity, so change
the 1000 values to 100, to make the calculations based on 99% busy or
99% idle.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I2a8297a2d629cedef66e79a7b427dab187362416
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23383
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

dpdk_governor: predicate PM_ENV_AMD_PSTATE_CPUFREQ on DPDK >= 23.11

Fixes issue #3420.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I6b2c4673f8741192eb5232dc922f13b249abcbb3
Review

dpdk_governor: predicate PM_ENV_AMD_PSTATE_CPUFREQ on DPDK >= 23.11

Fixes issue #3420.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I6b2c4673f8741192eb5232dc922f13b249abcbb3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23922
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Marek Chomnicki <marek.chomnicki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

module/scheduler: Added dump_info_json function pointer

Implemented a function dump_info_json to
output governor-specific information to a JSON stream.

Change-Id: Ifbd4c9e7a0d6dea0e6aa0c565eb088a07

module/scheduler: Added dump_info_json function pointer

Implemented a function dump_info_json to
output governor-specific information to a JSON stream.

Change-Id: Ifbd4c9e7a0d6dea0e6aa0c565eb088a07a0e2826
Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23639
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

module/scheduler: Added get_core_avail_freqs

To get the available frequencies of a specific Core ID.

Change-Id: I9187a1643c22fb0cff76ba069f99eac1a577e2ad
Signed-off-by: Krishna Kanth Reddy <krish.r

module/scheduler: Added get_core_avail_freqs

To get the available frequencies of a specific Core ID.

Change-Id: I9187a1643c22fb0cff76ba069f99eac1a577e2ad
Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23638
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

module/scheduler: macro for max lcore frquencies

Added a new macro for max lcore frequencies instead of using
local variables to hold this value. This matches the DPDK macro
RTE_MAX_LCORE_FREQS.

Ch

module/scheduler: macro for max lcore frquencies

Added a new macro for max lcore frequencies instead of using
local variables to hold this value. This matches the DPDK macro
RTE_MAX_LCORE_FREQS.

Change-Id: Id8a893c9826d9f2df4b029bbb24e8ed813e44653
Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23637
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

event: enforce 1 second default scheduling period

Currently the scheduling period was an optional parameter for
the framework_set_scheduler RPC. The 'dynamic' scheduler
enforced its own default by e

event: enforce 1 second default scheduling period

Currently the scheduling period was an optional parameter for
the framework_set_scheduler RPC. The 'dynamic' scheduler
enforced its own default by explicitly setting the period
to 1 second during init, if no value had been specified.

But other schedulers (gscheduler) didn't have that same code.
We could add it, but it's better to just enforce this when
the scheduler is first set - if user didn't pass a period,
set it to 1s.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: Ie2e1b3326438b34ebe0f3f65bf568e16621a288b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22593
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

so_ver: increase all major versions after SPDK 24.01

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to preve

so_ver: increase all major versions after SPDK 24.01

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.

Increasing major so version after LTS, allows the future releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.

This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components

Due to patch below being introduced after v24.01 code freeze,
bdev lib version should not be increased:
61623c5ca9 bdev: add spdk_bdev_io padding in place of io_submit_ch

Short reference to how the versions were changed:
MAX=$(git grep "SO_VER := " | cut -d" " -f 3 | sort -ubnr | head -1)
for((i=$MAX;i>0;i-=1)); do find . -name "Makefile" -exec \
sed -i -e "s/SO_VER := $i\$/SO_VER := $(($i+1))/g" {} +; done
find . -name "Makefile" -exec \
sed -i -e "s/SO_MINOR := .*/SO_MINOR := 0/g" {} +

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I0166d6be6c17bbbf985d48b5dfcb36f1d4af1b48
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21582
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

scheduler/dynamic: Add assert to check for operation success

spdk_reactor_get() may return NULL if it's unable to get the reactor. It is a critical error, so we can safely add assert there.

Change-

scheduler/dynamic: Add assert to check for operation success

spdk_reactor_get() may return NULL if it's unable to get the reactor. It is a critical error, so we can safely add assert there.

Change-Id: Iba10178b45891ece2e71626b16c75d956902a78d
Signed-off-by: Wojciech Panfil <wojciech.panfil@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18926
Reviewed-by: Kamuda Szymon <szymon.kamuda@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>

show more ...

module/scheduler: Silence warning about rte_power under clean target

This warning is returned regardless if the rte_power libs were
present or not as the clean target always removes them prior this

module/scheduler: Silence warning about rte_power under clean target

This warning is returned regardless if the rte_power libs were
present or not as the clean target always removes them prior this
check.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I45bd350d434ec1fbb6504c7df05c4d27946d4f9b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13562
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

so_ver: increase all major versions

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where

so_ver: increase all major versions

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.

Increasing major so version after LTS, allows the future releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.

Disabled test for increasing SO version without ABI change, as
that is goal of this patch. This check shall be removed with SPDK 23.05
release.
Looks like this was left over from prior LTS, to avoid that
make sure it is only skipped when running against v23.01.x as latest
release.

This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components
- removes suppressions for ABI tests


Short reference to how the versions were changed:
MAX=$(git grep "SO_VER := " | cut -d" " -f 3 | sort -ubnr | head -1)
for((i=$MAX;i>0;i-=1)); do find . -name "Makefile" -exec \
sed -i -e "s/SO_VER := $i\$/SO_VER := $(($i+1))/g" {} +; done
find . -name "Makefile" -exec \
sed -i -e "s/SO_MINOR := .*/SO_MINOR := 0/g" {} +

Change-Id: I3e5681802c0a5ac6d7d652a18896997cd07cc8bf
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16419
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...