test/scheduler: Read PID's status file only once

There's some peculiar issue where reading this file line by line can
be tainted by sudden updates to the file resulting in given fields
being unrecog

test/scheduler: Read PID's status file only once

There's some peculiar issue where reading this file line by line can
be tainted by sudden updates to the file resulting in given fields
being unrecognized by the get_proc_cpu_affinity(). To avoid that,
make sure this file is read once in full before parsing.

Change-Id: Ied4f03ada3b354b666df71f4d5f5d2467a874d98
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25407
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

test/scheduler: Account for multiple cpus in the affinity mask

get_proc_cpu_affinity() can return an actual list, depending on how
the affinity is set for given thread. To make sure this check doesn

test/scheduler: Account for multiple cpus in the affinity mask

get_proc_cpu_affinity() can return an actual list, depending on how
the affinity is set for given thread. To make sure this check doesn't
suddenly fail, iterate over the entire list.

Change-Id: I83c176445992e0495ef5ada56679d7316cb2ff9b
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25405
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Community CI Samsung <spdk.community.ci.samsung@gmail.com>

show more ...

event: add new 'mappings' parameter to static scheduler

This is useful for testing purposes, to place specific spdk_threads
on specific cores. Specified core must be in the thread's cpumask.

Usage:

event: add new 'mappings' parameter to static scheduler

This is useful for testing purposes, to place specific spdk_threads
on specific cores. Specified core must be in the thread's cpumask.

Usage:

scripts/rpc.py framework_set_scheduler --mappings='2:0,5:1'

This will move thread 2 to core 0 and thread 5 to core 1.

Note: checking for failure when setting core-limit parameter on
static scheduler has been removed. The whole mechanism for determine
which optional parameters are accepted for different schedulers needs
to be rethought, the current mechanism is not really working.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I16000805b8e2512f7ec6d84ee812d32bfcb2fd6d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24766
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Changpeng Liu <changpeliu@tencent.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Anisa Su <anisa.su@samsung.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>

show more ...

event: enable changing back to static scheduler

This requires saving the original core for each thread. Thankfully
it is very easy to just store it in the spdk_lw_thread. Then we
just run ->balance(

event: enable changing back to static scheduler

This requires saving the original core for each thread. Thankfully
it is very easy to just store it in the spdk_lw_thread. Then we
just run ->balance() once after the static scheduler is re-installed
to put all of the threads back to their original lcores, and take
all cores out of interrupt mode.

Note: 82c466 required bumping SO_VER due to adding trace_id to
struct spdk_reactor. But that didn't change the size of the structure.
For some unexplained reason, adding initial_lcore to the private
struct lw_thread in this patch results in abidiff no longer reporting
the trace_id change, and check_so_deps.sh fails saying the SO_VER
bump is no longer needed. A patch that simply adds the initial_lcore
member, with no other code changes, triggers the same error.

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: I9d1ace726617d95f89f42959c9c2c36602fb7f77
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24765
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeliu@tencent.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

test/scheduler: Add a system level test for the scheduler_set_option RPC

Change-Id: Ie9896285b7a037d6f17b18854485dea59d894729
Signed-off-by: SeungYeon Shin <syeon.shin@samsung.com>
Reviewed-on: http

test/scheduler: Add a system level test for the scheduler_set_option RPC

Change-Id: Ie9896285b7a037d6f17b18854485dea59d894729
Signed-off-by: SeungYeon Shin <syeon.shin@samsung.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23145
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

scripts/setup.sh: Use HUGE_EVEN_ALLOC logic by default

To that end, remove it altogether and allow setup.sh to always split
requested amount of hugepages across all available nodes. Custom
setups pe

scripts/setup.sh: Use HUGE_EVEN_ALLOC logic by default

To that end, remove it altogether and allow setup.sh to always split
requested amount of hugepages across all available nodes. Custom
setups per node are still available through the HUGENODE var.

Adjust some of the hugepages.sh tests to adhere to the new default.
This change allows us to remove the per_node_1G_alloc() test since
its flow is now covered by the custom_alloc().

By default, autotest.sh sets HUGEMEM=4096 and by spreading it across
all the nodes, for the minimal scenario, we get only 2GB on a single
node. For vhost tests, the default alloc per VM via vm_setup() is set
to 1GB so sharing that 2GB between qemu and SPDK spreads it a bit too
thin.

Case in point, for vhost.vhost_blk_packed_ring_integrity test roughly
> 600 hugepages is used for vhost. Since it slurps everything from
node0, only < 512 remains. Since VMs are assigned per node, the most
basic setups keep VM_*_qemu_numa_node set to 0. This is then used by
qemu to bind to a specific node. With 1G, < 512 hugepages is simply
not enough.

With that in mind, for vhost tests, keep all allocations on a single
node instead of trying to figure out right amount of memory per node
to keep the old behavior in (this may change in the future when
SPDK/vhost becomes numa-aware).

Change-Id: I83e18bfa4cc6de0a777804b354de083ae6ae9d8c
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24176
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz@tzawadzki.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

test/scheduler: fix cpu mask for rpc governor tests

1) When ran through isolate_cores.sh, only specific cores are possible
to use due to cgroup settings. "0x$spdk_main_core" was incorrectly
assignin

test/scheduler: fix cpu mask for rpc governor tests

1) When ran through isolate_cores.sh, only specific cores are possible
to use due to cgroup settings. "0x$spdk_main_core" was incorrectly
assigning "0x1" which is core 0. Rather than core 1 as intended.

2) After recent changes gscheduler requires that all SMT cores
are included in CPU mask, so this test wouldn't pass with just one.
Simply use the same spdk_cpumask as in other tests and modify
the remainder of the test to match that.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ieda3e48cb3871a22f45062089daf7aed7b670d1a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24303
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

test/scheduler: test scheduling period with dynamic scheduler

Static scheduler sets the scheduling period to 0 when it is
initialized, as no scheduling will take place.

Instead use dynamic schedule

test/scheduler: test scheduling period with dynamic scheduler

Static scheduler sets the scheduling period to 0 when it is
initialized, as no scheduling will take place.

Instead use dynamic scheduler to test scheduling period.
Changed the test order since it is not possible to change
back to static.

Fixes #3454

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic6098c42861255617471fdc4948d183c56944895
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24301

show more ...

test/governor: add tests for framework_get_governor RPC

Add basic tests for framework_get_governor RPC.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6db15a2476faa700690d7c

test/governor: add tests for framework_get_governor RPC

Add basic tests for framework_get_governor RPC.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6db15a2476faa700690d7c977895ad353260b9a0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23959
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

event/scheduler: do not accept params for scheduler without set_opts

Scheduler should not accept any parameters if set_opts is not implemented.
This looks awful, as the decode is done just to check

event/scheduler: do not accept params for scheduler without set_opts

Scheduler should not accept any parameters if set_opts is not implemented.
This looks awful, as the decode is done just to check if more opts were
provided. Yet it does work.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I31c2385244557ac08de281553cad82eb3b3228ba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23958
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

event/scheduler: allow framework_get_scheduler before initialization

It was always possible to configure scheduler before subsystem
initialization, but never possible to retrieve its configuration
a

event/scheduler: allow framework_get_scheduler before initialization

It was always possible to configure scheduler before subsystem
initialization, but never possible to retrieve its configuration
at that time.
Reason for this mismatch might have been omission or 'null'
scheduler pointing to the default one until subsystems are initialized.

Leave it as it is for now, but allow to retrieve scheduler configuration
at the same time as it is possible to configure it.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id5a896b2dc329e891690099e9ad45bf3f01a09bd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23955
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

test/scheduler: add tests for changes with static scheduler

There are assumptions which scheduler is the default,
or that static can only be switched from, never to.
Verify that.

Signed-off-by: Tom

test/scheduler: add tests for changes with static scheduler

There are assumptions which scheduler is the default,
or that static can only be switched from, never to.
Verify that.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I5c8ac44dc8fe145078d13fe776fdfd5f4f1c5119
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23957
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

test/scheduler: add tests for scheduler opts

Make sure that fix added in b238468 is tested.
As observed in issue #3421, this can be easily missed.

While here added any relevant tests that make sens

test/scheduler: add tests for scheduler opts

Make sure that fix added in b238468 is tested.
As observed in issue #3421, this can be easily missed.

While here added any relevant tests that make sense.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib53370dd18f9e1f3722c7949d24c2c2fd79e0848
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23956
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

test: Drop superfluous calls to print_backtrace()

Since we use errtrace, the ERR trap is inherited and preserved in
each subshell, cmd substitution, etc. that we execute along the
way. This means th

test: Drop superfluous calls to print_backtrace()

Since we use errtrace, the ERR trap is inherited and preserved in
each subshell, cmd substitution, etc. that we execute along the
way. This means that each failure inside such subshell triggers
call to print_backtrace(). Depending on the context, this may not
be desired. Consider simple example:

# foo() { return 1; }
# [[ $(foo) == no_output_so_fail_me ]]

This assumes that from the test perspective we don't care if foo(),
internally, fails, we just look for a specific string it should
return. If it doesn't, errexit will be triggered at the [[ level
and fail the test. This is fine, but before this happens, extra
print_backtrace() call will be made from the $() context.

Changing logic above and we may start spamming these calls:

# foo() { return 1; }
# [[ $(foo) -eq 0 ]]

Here failure still triggers the print_backtrace() but logically
the [[ test succeeds so we end up with a useless call in the log
which may be downright confusing.

Cherry on top is the fact that we don't inherit errexit inside
the subshell. So this print_backtrace() call is entirely foobar
as it won't do anything at all.

Best case would be to enable inherit_errexit from autotest_common.sh
across all the test suites. This, however, will require proper
discipline in maintaining the tests as use of subshell environment
would have to be more explicit - in other words the ambiguity from
the first example (i.e. failure != no output) would not be allowed.

For now, refactor code a bit to get rid of these extra calls, next
step would be to test behavior of inherit_errexit.

Change-Id: Ice3f072008162c1713e78a6da99a833919bc716c
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23078
Reviewed-by: Ben Walker <ben@nvidia.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

test/scheduler: Meassure utime of $spdk_pid threads as a fallback

Normally these tests look at a business of given cpu as a whole but
this is a bit problematic when some external entity starts to st

test/scheduler: Meassure utime of $spdk_pid threads as a fallback

Normally these tests look at a business of given cpu as a whole but
this is a bit problematic when some external entity starts to steal
its time. This causes discrepancies in tests where given cpu is
suddenly marked as busy where in fact SPDK was idle.

To mitigate this, after all per-cpu checks are done, and in case given
cpu is not idle where we expect it should be, check utime of the SPDK
thread pinned to that cpu.

Fixes issue #3362.

Change-Id: Ia105d1c45a9d0fa4c01a167a6c39790d68fb73ac
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23074
Reviewed-by: Ben Walker <ben@nvidia.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

test/scheduler: Calculate median of the cpu load samples

Done for debug purposes to clearly see how the outliers can
(sometimes heavily) influence the average.

Related to #3362.

Change-Id: I56db14

test/scheduler: Calculate median of the cpu load samples

Done for debug purposes to clearly see how the outliers can
(sometimes heavily) influence the average.

Related to #3362.

Change-Id: I56db14a817607c3ae13172fb41eaee3481c33ea8
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23073
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

test/scheduler: Enhance lookup of the $old_cgroup in move_proc()

Change-Id: I47fc3462550f03ada3e1b1517aa89c4fe4e2bca0
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://revi

test/scheduler: Enhance lookup of the $old_cgroup in move_proc()

Change-Id: I47fc3462550f03ada3e1b1517aa89c4fe4e2bca0
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23012
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

test/scheduler: Drop SILENT_CGROUP_DEBUG

Change-Id: I3475f8a722c029cb855ebecf6d886b79d2649985
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk

test/scheduler: Drop SILENT_CGROUP_DEBUG

Change-Id: I3475f8a722c029cb855ebecf6d886b79d2649985
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22968
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

test/scheduler: Make sure stderr is not O_TRUNCated in move_proc()

Since we are not dup'ing stdout in this case but rather explicitly
point at a file we want to write to, we need to make sure said
f

test/scheduler: Make sure stderr is not O_TRUNCated in move_proc()

Since we are not dup'ing stdout in this case but rather explicitly
point at a file we want to write to, we need to make sure said
file is not suddenly truncated.

In case foo.sh attempts to call to move_proc() and is executed
like so:

foo.sh >bar.log 2>&1

the redirection to /dev/stderr will cause bar.log to be truncated.
This is actually seen under CI after 8571999d82 got merged as it
removed the SILENT_CGROUP_DEBUG silencer - it causes the build.log
to suddenly drop the trace until remaining tests are finished.
(this happens as jenkins redirects both std{out,err} to a regular
file in a similar fashion).

This can be simply missed when running these tests with std{out,err}
attached to a terminal (as O_TRUNC is ignored there), and so it
unfortunately was while working on the aforementioned commit.

Change-Id: I9d2d5ab87151c3653b0e162aa72b1409418f6fc3
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22967
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

test/scheduler: Stop moving all processes between cgroups

Initially, this was done in order to reduce jitter on the cpus the
SPDK processes would execute on - every other process was being
moved to

test/scheduler: Stop moving all processes between cgroups

Initially, this was done in order to reduce jitter on the cpus the
SPDK processes would execute on - every other process was being
moved to a dedicated "all" cgroup which would limit cpus and mem
nodes to those SPDK was being moved away from.

This was done at early stages of the dynamic scheduler where it
was not known were potential discrepancies in tests are coming
from. Most of these issues had been already resolved, hence there
is no need to perform such an invasive action anymore.

In fact, under newer kernels (6.8.x) this cgroup dance causes a
peculiar issues where some of the processes become unresponsive
when suddenly moved from their designated cgroup.

Instead, move only processes associated with a test (this would
include SPDK processes executed along the way) to a separate
cgroup with appropriate setup.

While at it, enhance remove_cgroup() to make sure all lingering
leaf cgroups are removed as well - this is actually needed since
our cleanup() was leaving unattended cgroups around - /cpuset/all
and /cpuset in particular. Since this patch now tries to rely only
on /cpuset we need to make sure we can properly remove it without
hitting -EBUSY.

Change-Id: I450535959e323980341ffbc9723d38092aa1051d
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22859
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

scripts/perf: Align date's format across the PM tools

vmstat uses 24h timestamp and it cannot be changed so let other
tools follow suit.

Change-Id: I34d2d0d0b6ec551ef71d9e24539f62eb368a6346
Signed-

scripts/perf: Align date's format across the PM tools

vmstat uses 24h timestamp and it cannot be changed so let other
tools follow suit.

Change-Id: I34d2d0d0b6ec551ef71d9e24539f62eb368a6346
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21187
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

scripts/perf: Tag PM data with current run_test() instance

Add signal handler to catch USR1 and drive re-tagging of the PM data.
This should allow for a quick cross-reference while troubleshooting
t

scripts/perf: Tag PM data with current run_test() instance

Add signal handler to catch USR1 and drive re-tagging of the PM data.
This should allow for a quick cross-reference while troubleshooting
target test and its potential impact on the system.

Change-Id: I6eb7ff0ed0fc4d0162db0166543a070a4ad90de4
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21179
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

scripts/perf: Allow to define interval for collect-cpu-load

Change-Id: I35d53128dff8cbd0483b9c972f6baf356e46d221
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.sp

scripts/perf: Allow to define interval for collect-cpu-load

Change-Id: I35d53128dff8cbd0483b9c972f6baf356e46d221
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21177
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

scripts/perf: Add tool for collecting cpu load

As we are at it, remove bottleneck in get_cpu_time() - avoid reading
/proc/stat line by line to find target cpu as in case of systems with
hundreds of

scripts/perf: Add tool for collecting cpu load

As we are at it, remove bottleneck in get_cpu_time() - avoid reading
/proc/stat line by line to find target cpu as in case of systems with
hundreds of cpus this increases the get_cpu_time()'s runtime quite
significantly. Instead, slurp all cpu stats with a single read and
expose them as _cpuN[].

On a system with 144 cpus single get_cpu_time()'s iteration dropped
from ~5s to ~1s.

Note that this tool uses plain integer arithmetic so the results per
cpu time may be a bit off due to lacking precision. Nevertheless,
they should serve nicely as an approximation.

Change-Id: I64fecb593c5c38e3db79f75dcc4f23a9069c701a
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21173
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

test/scheduler: measure ticks since last check

get_thread_stats() gathers busy/idle from start of
the application, directly from RPC.
For some tests it is not required to measure state
from app star

test/scheduler: measure ticks since last check

get_thread_stats() gathers busy/idle from start of
the application, directly from RPC.
For some tests it is not required to measure state
from app start, but only changes during test.

idle.sh assumes that second check would make all
threads idle. Yet if initialization takes long enough,
app_thread will not be idle longer than it was busy by then.
This became apparent when adding ASAN to all jobs,
which considerably increased start up time.

To fix that, only measure ticks between consecutive RPC calls.
Added get_thread_stats_current() that returns busy/idle
difference, since last time it was called.

It allows to remove the workaround for first sample,
per removed comment.

While here:
- added print with load/busy/idle for each thread
- fixed unlikely condition for idle[thread] == busy[thread]

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Iaacce31aa7d4bcc22a51dbd7e71a4c7a4d274862
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/20884
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Ben Walker <ben@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...