scripts/perf: Dump cpu-temp data to stdout

This was missed and the actual output was being dumped only to a
file upon termination. As with any other perf/pm/ tool, make sure
each reading is dumped t

scripts/perf: Dump cpu-temp data to stdout

This was missed and the actual output was being dumped only to a
file upon termination. As with any other perf/pm/ tool, make sure
each reading is dumped to stdout by default as well.

Change-Id: I3ec6199ca7c998573cdec07d1d5b344f2b00e4d6
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23743
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

scripts/perf: Remove vhost/common.sh source from run_vhost_test.sh

It's slurped only to get default $VM_IMAGE which is already
parametrized and quite dependent on user's input.

Change-Id: If98698e1

scripts/perf: Remove vhost/common.sh source from run_vhost_test.sh

It's slurped only to get default $VM_IMAGE which is already
parametrized and quite dependent on user's input.

Change-Id: If98698e1d32244711c594316d4d6a2076d978524
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21060
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

scripts/perf: Include per-node hugepages stats in collect-vmstat

This may help in debugging how hugepages are distributed across
different numa nodes during tests. Example:

# old
... ----extra

scripts/perf: Include per-node hugepages stats in collect-vmstat

This may help in debugging how hugepages are distributed across
different numa nodes during tests. Example:

# old
... ----extra info----
... avail shmem ahp_total hp_free hp_size test
... 46938928 5997040 2048 1778 2048 N/A

# new
... ----extra info----
... avail shmem hp_total hp_free hp_size hp_per_node test
... 46938928 5997040 2048 1778 2048 N/A N0-1048576kB_f=0,N0-1048576kB_t=0,N0-2048kB_f=1778,N0-2048kB_t=2048,N1-1048576kB_f=0,N1-1048576kB_t=0,N1-2048kB_f=0,N1-2048kB_t=0 N/A

Granted it's a mouthful, but it does improve view of the system's
resources.

Change-Id: Ib7737752c9df9a67c32b656a5df294ce32b65ef6
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24180
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz@tzawadzki.com>

show more ...

general: fix misspells and typos

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: Iab206ef526eb7032c6681a3145450010c91705a4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+

general: fix misspells and typos

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: Iab206ef526eb7032c6681a3145450010c91705a4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24120
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

scripts/nvmf_perf: set all NIC RX queues at once

Use wildcards to set and read NIC RX queues values.
Saves a few seconds on every run.

Additionally log appropriate error message if setting
rps_flow

scripts/nvmf_perf: set all NIC RX queues at once

Use wildcards to set and read NIC RX queues values.
Saves a few seconds on every run.

Additionally log appropriate error message if setting
rps_flow_cnt was not successful.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: If79f39bd9b335c322b19076ef7de8a0f667526c1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23473
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

scripts/nvmf_perf: indent multi-line strings

I just really couldn't watch at the lack of
indentation in these lines.

Change-Id: I25447d585e364e779b1f647fbc2bc3def699cb9b
Signed-off-by: Karol Lateck

scripts/nvmf_perf: indent multi-line strings

I just really couldn't watch at the lack of
indentation in these lines.

Change-Id: I25447d585e364e779b1f647fbc2bc3def699cb9b
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17434
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

scripts/nvmf_perf: shorten gen_fio_config

Change existing get_core_list_from_mask() method
into Server class static method and use it to
count initiator CPU cores when cpus_allowed is
set.

Signed-o

scripts/nvmf_perf: shorten gen_fio_config

Change existing get_core_list_from_mask() method
into Server class static method and use it to
count initiator CPU cores when cpus_allowed is
set.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ice00f1fa9a8c47c3758df0cb90b50142dbfd7cfb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17435
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Marek Chomnicki <marek.chomnicki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>

show more ...

scripts/nvmf_perf: silence PCM utility

Redirect stdout and stderr streams so they're not
printed to screen. This utility generates a lot of
(mostly) useless output, so write everything to
files.

Ch

scripts/nvmf_perf: silence PCM utility

Redirect stdout and stderr streams so they're not
printed to screen. This utility generates a lot of
(mostly) useless output, so write everything to
files.

Change-Id: I2bd5392f04d7e421b133dcf1cba1d7fc789953ec
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19065
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

scripts/nvmf_perf: shorten dict path in results parsing

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I2838cc2902ecc520a36be743ae31b708e87c8349
Reviewed-on: https://review.spdk.i

scripts/nvmf_perf: shorten dict path in results parsing

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I2838cc2902ecc520a36be743ae31b708e87c8349
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23670
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

scripts/nvmf_perf: round results

Round results to 3 decimal places. More seems
unnecessary and makes reading log ouptut harder.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I21

scripts/nvmf_perf: round results

Round results to 3 decimal places. More seems
unnecessary and makes reading log ouptut harder.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I21819c7b1105c4c2da19795fb056a2d49a40d5db
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23669
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

scripts/nvmf_perf: improve rdma driver related functions

Refactor functions a bit to improve readability.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Iba9a1ca1f306a03abfd72f02

scripts/nvmf_perf: improve rdma driver related functions

Refactor functions a bit to improve readability.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Iba9a1ca1f306a03abfd72f02b6758e6c16affd63
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22321
Reviewed-by: Marek Chomnicki <marek.chomnicki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>

show more ...

scripts/nvmf_perf: use a more pythonic approach for condition checking

As all of those variables are boolean type we can change
condition checking and drop 'is True/False'.

Signed-off-by: Kamil God

scripts/nvmf_perf: use a more pythonic approach for condition checking

As all of those variables are boolean type we can change
condition checking and drop 'is True/False'.

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: I9efae7178be335c25be52f76bb208acb644dc303
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19746
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Marek Chomnicki <marek.chomnicki@intel.com>

show more ...

scripts/nvmf_perf: add validate_allowlist_settings function

Added the 'validate_allowlist_settings' function.
This function is responsible for detecting potential
irregularities in the provided data

scripts/nvmf_perf: add validate_allowlist_settings function

Added the 'validate_allowlist_settings' function.
This function is responsible for detecting potential
irregularities in the provided data and arguments
regarding use of NVMe drives in test.

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: I76ddbe9c44dae06dff3c087cf141dd39a329c40e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19734
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>

show more ...

scripts/nvmf_perf: simplify fio parameters collecting

Use .get() instead of ifs and use dict to collect all
neccessary parameters.

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id:

scripts/nvmf_perf: simplify fio parameters collecting

Use .get() instead of ifs and use dict to collect all
neccessary parameters.

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Change-Id: I0cd373629b94662f844b178aa7c12ac8f31ae780
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19730
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

scripts/check_format: Adjust bash code to adhere to shfmt v3.8.0

This bump ensures some of the obscure formatting bugs are finally
squashed. All the adjustments introduced here adhere to what we
alr

scripts/check_format: Adjust bash code to adhere to shfmt v3.8.0

This bump ensures some of the obscure formatting bugs are finally
squashed. All the adjustments introduced here adhere to what we
already try to enforce across the entire repo, but due to said bugs
could not be implemented.

Remove SHFMT_NO_EDITORCONFIG since it's not supported in latest
shfmt releases.

This patch doesn't introduce any functional change in the updated
files.

Change-Id: Ia2b011b86c8fe12376c4a8455ee638473a9f7179
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23088
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot

show more ...

scripts/perf: Use sudo only when needed

Most of the monitoring tools can be executed as non-root so use
sudo only for those that explicitly require it.

Change-Id: I0044abd888284c0b368cfb5f214098502

scripts/perf: Use sudo only when needed

Most of the monitoring tools can be executed as non-root so use
sudo only for those that explicitly require it.

Change-Id: I0044abd888284c0b368cfb5f21409850261ce4cb
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22699
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

scripts/perf: Drop MONITOR_RESOURCES_PIDS[@]

This list is in practice available only to a process which started
the monitors - every other process that calls to
signal_monitor_resources() relies sol

scripts/perf: Drop MONITOR_RESOURCES_PIDS[@]

This list is in practice available only to a process which started
the monitors - every other process that calls to
signal_monitor_resources() relies solely on the .pid file.

Change-Id: Ie239c1ae9e921d6babca45deba79febdba3dabad
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22698
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

regression/nvmf: fix nvmf_subsystem_add_ns parse arguments

With change f19c5a5 nvmf_subsystem_add_ns accepts kwargs arguments in
function

Change-Id: Ifd70c0dc0218e1dc055b83fb64ac7633f008507e
Signed

regression/nvmf: fix nvmf_subsystem_add_ns parse arguments

With change f19c5a5 nvmf_subsystem_add_ns accepts kwargs arguments in
function

Change-Id: Ifd70c0dc0218e1dc055b83fb64ac7633f008507e
Signed-off-by: Marek Chomnicki <marek.chomnicki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22517
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Ben Walker <ben@nvidia.com>
Reviewed-by: Michal Berger <michal.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>

show more ...

scripts/perf/pm: Add minimal FreeBSD support to vmstat collector

Change-Id: If30ba474193c24c610bfc4619525726c46516dcc
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://revi

scripts/perf/pm: Add minimal FreeBSD support to vmstat collector

Change-Id: If30ba474193c24c610bfc4619525726c46516dcc
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21548
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

scripts/perf/pm: Add $TEST_TAG to collect-bmc-pm's output

As we are at it, align some printouts and redirect them to stderr
as initially intended.

Change-Id: I3c26240bd72ad3ebecba657a8f6b8865f96c4a

scripts/perf/pm: Add $TEST_TAG to collect-bmc-pm's output

As we are at it, align some printouts and redirect them to stderr
as initially intended.

Change-Id: I3c26240bd72ad3ebecba657a8f6b8865f96c4a00
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21536
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

autotest: Add helpers for controling pm tools

And use them to hook collect-* perf scripts into autobuild,autopackage.

This is meant to serve as a debug exposing impact of the full SPDK build
on the

autotest: Add helpers for controling pm tools

And use them to hook collect-* perf scripts into autobuild,autopackage.

This is meant to serve as a debug exposing impact of the full SPDK build
on the underlying system.

Change-Id: I7e0fe48927dd5725ea415e99752fa33e608710ec
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21494
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

scripts/perf: Set nullglob, extglob in run_vhost_test

Do this to silence potential errors comming from a direct source
of test/vhost/common.sh.

Change-Id: Iad53981e3c01d8a7acf91e274ec289ca71b962d0

scripts/perf: Set nullglob, extglob in run_vhost_test

Do this to silence potential errors comming from a direct source
of test/vhost/common.sh.

Change-Id: Iad53981e3c01d8a7acf91e274ec289ca71b962d0
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21545
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

scripts/perf: Rename vhost_*master_core to vhost_*main_core

The initial implementation was based out of custom CI configuration
which was not changed after the following commit:

https://review.spd

scripts/perf: Rename vhost_*master_core to vhost_*main_core

The initial implementation was based out of custom CI configuration
which was not changed after the following commit:

https://review.spdk.io/gerrit/c/spdk/spdk/+/5353

So fix it now.

Change-Id: Ib5fcfc0d6535019c22066ccf36aabe6748c699e1
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21546
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

scripts/nvmf_perf: make fio numa alignment optional

Add fio section option to toggle NUMA aligning the
workload. Mostly to be used as an easy on/off switch
for experiments.

Signed-off-by: Karol Lat

scripts/nvmf_perf: make fio numa alignment optional

Add fio section option to toggle NUMA aligning the
workload. Mostly to be used as an easy on/off switch
for experiments.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I640954c0baa64b2e158bed55e52f92c48bc16fb2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21504
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

scripts/nvmf_perf: check dmesg for throttling or overheat issues

The method is designed to scan kernel logs for specific entries that
indicate CPU throttling or device health issues.

Change-Id: Ie3

scripts/nvmf_perf: check dmesg for throttling or overheat issues

The method is designed to scan kernel logs for specific entries that
indicate CPU throttling or device health issues.

Change-Id: Ie363670aff00527b31215b2ed46bf7964f6849ed
Signed-off-by: Jaroslaw Chachulski <jaroslawx.chachulski@intel.com>
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21161
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...