spdk_log: introduce spdk_log_ext API

Current spdk_log implementation allows to provide only log function.
In some cases it might not be enough because user have to take care
of managing resources as

spdk_log: introduce spdk_log_ext API

Current spdk_log implementation allows to provide only log function.
In some cases it might not be enough because user have to take care
of managing resources associated with logger.
Extended API supports providing open and close functions
with set context which will be called by SPDK in corresponded places.

Fixes #3480

Change-Id: Ia1d41fe3b414e2f603c2ae6612b7f9eb8d29bbb7
Signed-off-by: Vasilii Ivanov <iwanovvvasilij@gmail.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24847
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

so_ver: increase all major versions after SPDK 24.01

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to preve

so_ver: increase all major versions after SPDK 24.01

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.

Increasing major so version after LTS, allows the future releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.

This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components

Due to patch below being introduced after v24.01 code freeze,
bdev lib version should not be increased:
61623c5ca9 bdev: add spdk_bdev_io padding in place of io_submit_ch

Short reference to how the versions were changed:
MAX=$(git grep "SO_VER := " | cut -d" " -f 3 | sort -ubnr | head -1)
for((i=$MAX;i>0;i-=1)); do find . -name "Makefile" -exec \
sed -i -e "s/SO_VER := $i\$/SO_VER := $(($i+1))/g" {} +; done
find . -name "Makefile" -exec \
sed -i -e "s/SO_MINOR := .*/SO_MINOR := 0/g" {} +

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I0166d6be6c17bbbf985d48b5dfcb36f1d4af1b48
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21582
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

log: Add spdk_flog/vflog() to write messages to a specified file

Add two public APIs, spdk_flog() and spdk_vflog(), to write messages to
a specified file. These APIs will be used to write JSON RPC c

log: Add spdk_flog/vflog() to write messages to a specified file

Add two public APIs, spdk_flog() and spdk_vflog(), to write messages to
a specified file. These APIs will be used to write JSON RPC call history
to a specified file.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I21cc15cc7984d068a9ba567708b86c162e54705a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18826
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Sebastian Brzezinka <sebastian.brzezinka@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

build: enable Wpointer-arith in selected libraries

There's an organization, called Timerland SIG [1], that uses SPDK's NVMe
driver to implement NVMe Boot Specification under UEFI. The UEFI EDK2
bui

build: enable Wpointer-arith in selected libraries

There's an organization, called Timerland SIG [1], that uses SPDK's NVMe
driver to implement NVMe Boot Specification under UEFI. The UEFI EDK2
build environment uses both gcc and msvc. While SPDK can be built
without any issues with gcc, msvc complains about a few things, one of
which is void pointer arithmetic. So, to make Timberland's effort
easier, Wpointer-arith is enabled in the libraries they're using.

[1] https://github.com/timberland-sig

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ic90330baa4e69ee72f7e040de91893093b96f476
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18529
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

so_ver: increase all major versions

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where

so_ver: increase all major versions

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.

Increasing major so version after LTS, allows the future releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.

Disabled test for increasing SO version without ABI change, as
that is goal of this patch. This check shall be removed with SPDK 23.05
release.
Looks like this was left over from prior LTS, to avoid that
make sure it is only skipped when running against v23.01.x as latest
release.

This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components
- removes suppressions for ABI tests


Short reference to how the versions were changed:
MAX=$(git grep "SO_VER := " | cut -d" " -f 3 | sort -ubnr | head -1)
for((i=$MAX;i>0;i-=1)); do find . -name "Makefile" -exec \
sed -i -e "s/SO_VER := $i\$/SO_VER := $(($i+1))/g" {} +; done
find . -name "Makefile" -exec \
sed -i -e "s/SO_MINOR := .*/SO_MINOR := 0/g" {} +

Change-Id: I3e5681802c0a5ac6d7d652a18896997cd07cc8bf
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16419
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

log: add deprecated tracking API

When use of deprecated featues is encountered, SPDK now calls
SPDK_LOG_DEPRECATED(). This logs the use of deprecated functionality in
a consistent way, making it eas

log: add deprecated tracking API

When use of deprecated featues is encountered, SPDK now calls
SPDK_LOG_DEPRECATED(). This logs the use of deprecated functionality in
a consistent way, making it easy to add further instrumentation to catch
code paths that trigger deprecated behavior.

Change-Id: Idfd33ade171307e5e8235a7aa0d969dc5d93e33d
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15689
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Community-CI: Mellanox Build Bot

show more ...

update Intel copyright notices

per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date defa

update Intel copyright notices

per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date default <file> | tail -1

and then pull just the 4 digit year from the result.

Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc). Contribution date used
"--follow -C95%" to get the most accurate date.

Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line. Simply there for
consistency.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id5b7ce4f658fe87132f14139ead58d6e285c04d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15192
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot

show more ...

Replace most BSD 3-clause license text with SPDX identifier.

Many open source projects have moved to using SPDX identifiers
to specify license information, reducing the amount of
boilerplate code in

Replace most BSD 3-clause license text with SPDX identifier.

Many open source projects have moved to using SPDX identifiers
to specify license information, reducing the amount of
boilerplate code in every source file. This patch replaces
the bulk of SPDK .c, .cpp and Makefiles with the BSD-3-Clause
identifier.

Almost all of these files share the exact same license text,
and this patch only modifies the files that contain the
most common license text. There can be slight variations
because the third clause contains company names - most say
"Intel Corporation", but there are instances for Nvidia,
Samsung, Eideticom and even "the copyright holder".

Used a bash script to automate replacement of the license text
with SPDX identifier which is checked into scripts/spdx.sh.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa88ab5e92ea471691dc298cfe41ebfb5d169780
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12904
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: <qun.wan@intel.com>

show more ...

so_ver: increase all major versions

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where

so_ver: increase all major versions

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.

Increasing major so version after LTS, allows the future releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.

Disabled test for increasing SO version without ABI change, as
that is goal of this patch. This check shall be removed with SPDK 22.05
release.

This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components
- removes suppressions for ABI tests

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id1a5358882dc496faa5b0b5c9a63b326c378c551
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11361
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

log: add an API to return syslog level based on SPDK's log level

Change-Id: I970119385989e7d97bda065f5aa2bbd82519c655
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://revi

log: add an API to return syslog level based on SPDK's log level

Change-Id: I970119385989e7d97bda065f5aa2bbd82519c655
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9103
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: John Levon <levon@movementarian.org>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>

show more ...

so_ver: increase all major versions

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where

so_ver: increase all major versions

To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.

Increasing major so version after LTS, allows the quarterly releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.

Disabled test for increasing SO version without ABI change, as
that is goal of this patch. This check shall be removed with SPDK 21.04
release.

This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components
- removes suppressions for ABI tests

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I44d01154430a074103bd21c7084f44932e81fe72
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6167
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

lib/log: add RPC to toggle timestamps

Allow toggling log timestamps on and off by adding new RPC call.

Change-Id: I34c84bf89fae352ade266fbf7fd20594ff67bced
Signed-off-by: Vitaliy Mysak <vitaliy.mys

lib/log: add RPC to toggle timestamps

Allow toggling log timestamps on and off by adding new RPC call.

Change-Id: I34c84bf89fae352ade266fbf7fd20594ff67bced
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2024
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

log: remove backtrace printing

It was broken since long ago. --enable-log-bt doesn't change
anything. log.c expects SPDK_LOG_BACKTRACE_LVL to be defined
for backtrace to work, but it's not defined a

log: remove backtrace printing

It was broken since long ago. --enable-log-bt doesn't change
anything. log.c expects SPDK_LOG_BACKTRACE_LVL to be defined
for backtrace to work, but it's not defined anywhere.

Apparently nobody needs this, so remove it.

Change-Id: I2313fd24198b0bf718663f2eafee9b5c6efa0a7f
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2194
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

mk/lib: add a check that major and minor version is set for libs.

Also, while we are here, consolidate setting SO_SUFFIX to one spot.

Previously, it was possible for a library to slip through
witho

mk/lib: add a check that major and minor version is set for libs.

Also, while we are here, consolidate setting SO_SUFFIX to one spot.

Previously, it was possible for a library to slip through
without an SO version.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I4db5fa5839502d266c6259892e5719b05134518c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2361
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

lib/log: add spdk_vlog function

Some users of SPDK API, such as OCF,
may want to generate logs with arguments themselves.
In that case they would need to first create a buffer
and then pass that buf

lib/log: add spdk_vlog function

Some users of SPDK API, such as OCF,
may want to generate logs with arguments themselves.
In that case they would need to first create a buffer
and then pass that buffer as a single argument to spdk_log().
This change adds spdk_vlog() which accepts va_list as argument list,
so it is easier to use spdk_log().

Change-Id: Ie2a3ac481035a250fcd68d0f9b8292008ebb6fe0
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1946
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lib/log: add map file.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ib376a31d307a6bb4a3b4fde52b8ec14bc748a539
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2216
Community

lib/log: add map file.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ib376a31d307a6bb4a3b4fde52b8ec14bc748a539
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2216
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

make: rev SO versions individually for libraries.

This will allow us to keep track of compatibility issues on a
per-library basis.

Change-Id: Ib0c796adb1efe1570212a503ed660bef6f142b6e
Signed-off-by

make: rev SO versions individually for libraries.

This will allow us to keep track of compatibility issues on a
per-library basis.

Change-Id: Ib0c796adb1efe1570212a503ed660bef6f142b6e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1067
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

mk: move log_rpc from inside log

The log_rpc library has several dependencies which depend upon the log
library. This creates a circular dependency chain that makes single
threaded make unwieldy and

mk: move log_rpc from inside log

The log_rpc library has several dependencies which depend upon the log
library. This creates a circular dependency chain that makes single
threaded make unwieldy and makes multi-threaded make impossible.

Change-Id: I35e6532afcabce0f25974ed97444a56975654904
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465192
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

Make: Add deps for each shared obj individually

This prevents the SPDK build from requirng unnecessary dependencies when
not compiling certain features. Also, fixes github issue #434

Change-Id: I7d

Make: Add deps for each shared obj individually

This prevents the SPDK build from requirng unnecessary dependencies when
not compiling certain features. Also, fixes github issue #434

Change-Id: I7d0520474f3656ae32670313f2290e6b741c5ca8
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426131
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

log: Create a separate implementation file for common log flag operations.

Change-Id: I2524b7881885a0784492a8189bf22a0b8c92d470
Signed-off-by: Ed rodriguez <edwinr@netapp.com>
Reviewed-on: https://r

log: Create a separate implementation file for common log flag operations.

Change-Id: I2524b7881885a0784492a8189bf22a0b8c92d470
Signed-off-by: Ed rodriguez <edwinr@netapp.com>
Reviewed-on: https://review.gerrithub.io/386169
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

build: include spdk.common.mk in lib Makefiles

Explicitly include spdk.common.mk at the top of all lib Makefiles so
that CONFIG options and other predefined variables are set.

Change-Id: I1e560c294

build: include spdk.common.mk in lib Makefiles

Explicitly include spdk.common.mk at the top of all lib Makefiles so
that CONFIG options and other predefined variables are set.

Change-Id: I1e560c294fe8242602e45191a280f4295533ae44
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>

show more ...

log: Add a lib for integrating the log lib with event framework.

Linking this library into an application makes the log
configurable via RPC. This only works if the rpc subsystem
is also loaded.

Ch

log: Add a lib for integrating the log lib with event framework.

Linking this library into an application makes the log
configurable via RPC. This only works if the rpc subsystem
is also loaded.

Change-Id: If1340cf2a845ef159290232c26f341150c98fb9a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>

show more ...

log: add SPDK logging library

Change-Id: I3d4060b619a3976ffff75c05f86214e3fcd18fb5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>