scripts/perf: Remove vhost/common.sh source from run_vhost_test.sh

It's slurped only to get default $VM_IMAGE which is already
parametrized and quite dependent on user's input.

Change-Id: If98698e1

scripts/perf: Remove vhost/common.sh source from run_vhost_test.sh

It's slurped only to get default $VM_IMAGE which is already
parametrized and quite dependent on user's input.

Change-Id: If98698e1d32244711c594316d4d6a2076d978524
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21060
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

scripts/check_format: Adjust bash code to adhere to shfmt v3.8.0

This bump ensures some of the obscure formatting bugs are finally
squashed. All the adjustments introduced here adhere to what we
alr

scripts/check_format: Adjust bash code to adhere to shfmt v3.8.0

This bump ensures some of the obscure formatting bugs are finally
squashed. All the adjustments introduced here adhere to what we
already try to enforce across the entire repo, but due to said bugs
could not be implemented.

Remove SHFMT_NO_EDITORCONFIG since it's not supported in latest
shfmt releases.

This patch doesn't introduce any functional change in the updated
files.

Change-Id: Ia2b011b86c8fe12376c4a8455ee638473a9f7179
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23088
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot

show more ...

scripts/perf: Set nullglob, extglob in run_vhost_test

Do this to silence potential errors comming from a direct source
of test/vhost/common.sh.

Change-Id: Iad53981e3c01d8a7acf91e274ec289ca71b962d0

scripts/perf: Set nullglob, extglob in run_vhost_test

Do this to silence potential errors comming from a direct source
of test/vhost/common.sh.

Change-Id: Iad53981e3c01d8a7acf91e274ec289ca71b962d0
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21545
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

scripts/perf: Rename vhost_*master_core to vhost_*main_core

The initial implementation was based out of custom CI configuration
which was not changed after the following commit:

https://review.spd

scripts/perf: Rename vhost_*master_core to vhost_*main_core

The initial implementation was based out of custom CI configuration
which was not changed after the following commit:

https://review.spdk.io/gerrit/c/spdk/spdk/+/5353

So fix it now.

Change-Id: Ib5fcfc0d6535019c22066ccf36aabe6748c699e1
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21546
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

test/vhost: Add cpu gen wrapper around conf-generator

The goal here is to not use a static cpu configuration in vhost
tests, but rather determine it dynamically based on the underlying
topology.

Al

test/vhost: Add cpu gen wrapper around conf-generator

The goal here is to not use a static cpu configuration in vhost
tests, but rather determine it dynamically based on the underlying
topology.

Also, change vm[N]_node override var to VM[N]_NODE as existing
implementation didn't really allow the user to override the node
this way (since it was explicitly set by the conf-generator while
constructing initial NUMA-aware setup).

Change-Id: I791afaa8ebbd77de85d1e29c2fa43d442c22cac8
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21420
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

perf/vhost: Add nullglob,extglob opts

Previously conf-generator did depend on these being set by the
scheduler/common.sh, but since the:

https://review.spdk.io/gerrit/c/spdk/spdk/+/19551

these wer

perf/vhost: Add nullglob,extglob opts

Previously conf-generator did depend on these being set by the
scheduler/common.sh, but since the:

https://review.spdk.io/gerrit/c/spdk/spdk/+/19551

these were moved to autotest_common.sh. conf-generator is not
meant to use it, so enable these opts directly inside of it.

Change-Id: I00a70250e2b240e27345a344848deb0fc406209e
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/20581
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

scripts/perf: Allow to override number of VMs to be pinned to nvme

Targeted for the vhost's TC3 use-case where single drive should be
split along multiple VMs. E.g.:

# vm_count=2 ./conf-generator

scripts/perf: Allow to override number of VMs to be pinned to nvme

Targeted for the vhost's TC3 use-case where single drive should be
split along multiple VMs. E.g.:

# vm_count=2 ./conf-generator -p disk
# Generated automatically by conf-generator
# NVMe Drives: 2 VM count: 2
0000:17:00.0,Nvme0,1,0
0000:18:00.0,Nvme1,1,1

vs

# vms_per_nvme=2 vm_count=2 ./conf-generator -p disk
# Generated automatically by conf-generator
# NVMe Drives: 1 VM count: 2
0000:17:00.0,Nvme0,2,0 1

Change-Id: I68ccaab896b763534cedbeac0331218b73cff454
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17982
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

perf/vhost: Auto generate VM cpu and disk map configs

The configuration is generated based on existing host's NUMA
topology (as seen via sysfs) instead of a total number of cpus as
it was done befor

perf/vhost: Auto generate VM cpu and disk map configs

The configuration is generated based on existing host's NUMA
topology (as seen via sysfs) instead of a total number of cpus as
it was done before. New logic attempts to load balance VMs and
their cpus based on nvme drives' NUMA location. If there is no
enough cpus left under the target node, all remaining nodes are
checked. For the sake of the performance, cpus are not mixed
between different numa nodes.

Disk map is created by mapping VMs to existing nvme drives
based on their NUMA location. Extra VMs are assigned in bus
order of the nvme drives.

SPDK cpus are split by matching the VM-to-nvme NUMA ratio.
Static list can be defined as well to override this behavior.

https://trello.com/c/HSoRtQkO/401-deprecate-vhost-performance-python-helper-script

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Ia63c6f9a472a685d252efd110eaba7b114a87d2c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12401
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>

show more ...

misc: Fix spelling mistakes

Found with misspell-fixer.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: If062df0189d92e4fb2da3f055fb981909780dc04
Reviewed-on: https://review.spdk.i

misc: Fix spelling mistakes

Found with misspell-fixer.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: If062df0189d92e4fb2da3f055fb981909780dc04
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15207
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

scripts/run_vhost_test: fix shellcheck SC2294 warning

The warning appeared during Fedora 37 VM image testing

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Signed-off-by: Michal Berger <mi

scripts/run_vhost_test: fix shellcheck SC2294 warning

The warning appeared during Fedora 37 VM image testing

Signed-off-by: Kamil Godzwon <kamilx.godzwon@intel.com>
Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I2ceff22d08614fec0c27b9931d71753f57cf99bd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15794
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

add (c) and SPDX header to bash files as needed

per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --forma

add (c) and SPDX header to bash files as needed

per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date default <file> | tail -1

and then pull just the year from the result.

Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc)

For intel copyrights added, --follow and -C95% were used.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I2ef86976095b88a9bf5b1003e59f3943cd6bbe4c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15209
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...