scripts/core-collector: Filter processes based on set of criteria

We do this to make sure that we catch not only what we did execute
from within the target repo, but also what we preloaded and/or
co

scripts/core-collector: Filter processes based on set of criteria

We do this to make sure that we catch not only what we did execute
from within the target repo, but also what we preloaded and/or
consider as a critical process for the tests that may be coming
from the outside world.

Change-Id: I3f61d4cf8f745ca52b0ba95e9d8d621f00a2597c
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19806
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

scripts/core-collector: Filter out processes based on CWD

Don't collect cores from processes that were executed outside of
the target SPDK repo.

Fixes issue #3110

Change-Id: I8f50ed0af30507386dbf9

scripts/core-collector: Filter out processes based on CWD

Don't collect cores from processes that were executed outside of
the target SPDK repo.

Fixes issue #3110

Change-Id: I8f50ed0af30507386dbf9387a2a0c1f532399bb0
Signed-off-by: Michal Berger <michal.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19766
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <jim.harris@gmail.com>

show more ...

scripts/core-collector: Add coredump_filter map to .json

This is done to make clear what data is included with the core dump.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I67f7

scripts/core-collector: Add coredump_filter map to .json

This is done to make clear what data is included with the core dump.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I67f7da7f342354df4688147eaf56283b33ba2863
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15996
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

scripts: Ignore RLIMIT_CORE while writing out core dumps

As stated in the core(5) the RLIMIT_CORE is not enforced when kernel
is piping core to the collector, hence it will be written out even
when

scripts: Ignore RLIMIT_CORE while writing out core dumps

As stated in the core(5) the RLIMIT_CORE is not enforced when kernel
is piping core to the collector, hence it will be written out even
when it's set to 0. Instead of playing nice and checking the validity
of RLIMIT_CORE, simply ignore it and apply our own limit (2GB).

This is done for convenience and to make sure that any changes to the
way how given distro applies limits (see fedora37) to the processes
doesn't affect our core collection.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: Iab8757f548f2a4f0f4b27b4ce66b0a4a84ed2579
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15994
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

add (c) and SPDX header to bash files as needed

per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --forma

add (c) and SPDX header to bash files as needed

per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date default <file> | tail -1

and then pull just the year from the result.

Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc)

For intel copyrights added, --follow and -C95% were used.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I2ef86976095b88a9bf5b1003e59f3943cd6bbe4c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15209
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

test/common: don't specify output dir for core_pattern in command line

Remove output directory argument for core-collector.sh when
setting system core_pattern. Instead save the full output
directory

test/common: don't specify output dir for core_pattern in command line

Remove output directory argument for core-collector.sh when
setting system core_pattern. Instead save the full output
directory path into a file to a known location and read it
directly in core-collector.sh.

The reason for this change is 128 bytes command line length
for core_pattern (see 'man core 5'). In case working with
long paths the core_pattern command line gets truncated and
core dumps are not generated.

Change-Id: Ia74c180e4923fb43d6ff66129ffe54c32827adca
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12755
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

scripts/core-collector: Don't limit core dump collection

We should gather the core regardless of the RLIMIT_CORE setting.
Also, don't limit number of parallel executions of the collector
in case mor

scripts/core-collector: Don't limit core dump collection

We should gather the core regardless of the RLIMIT_CORE setting.
Also, don't limit number of parallel executions of the collector
in case more than 2 processes crashed (unlikely to happen anyway,
but just in case make sure we cover it). Lastly, wait a bit longer
for the potential cores.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ic7de8f8000676122282469b729f45b235ab6f9ca
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10852
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

scripts: Get soft core ulimit from the template

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I6794886832ec641ec18b72621f21264970d805d5
Reviewed-on: https://review.spdk.io/gerri

scripts: Get soft core ulimit from the template

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I6794886832ec641ec18b72621f21264970d805d5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7118
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>

show more ...

scripts: Use core dump collector

This is done in order to track core dumps in a more efficient
manner. Till now, some cores could be missed if the binary was
executed outside of the cwd of the autot

scripts: Use core dump collector

This is done in order to track core dumps in a more efficient
manner. Till now, some cores could be missed if the binary was
executed outside of the cwd of the autotest (i.e. outside of
the spdk repo) but which was part of the critical path of the
actual test (e.g. fio in vhost-initiator tests). Also, since
core_pattern was set to plain "core", impact on the underlying
storage wasn't controlled either - if core was 20G in size,
this is what we would get. This could easly exhaust storage in
case error-prone patchsets were submitted on the CI side.

The collector will try to mitigate all the above by doing the
following:

- collecting all the cores, regardless of their cwd
- limiting size of the core to 2G
- compressing the cores (lz4)

Also, limit of 2 collectors executing at once is set - if more
processes crashes at approx. the same time, they will be logged
in the kernel log instead.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I5956a9030c463ae85a21bfe95f28af5568c5c285
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5369
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...