env: add SPDK_ENV_NUMA_ID_ANY and replace socket_id with numa_id

We will try to avoid further proliferation of "SOCKET_ID" to refer
to a NUMA socket ID moving forward, and just use "NUMA_ID" to avoi

env: add SPDK_ENV_NUMA_ID_ANY and replace socket_id with numa_id

We will try to avoid further proliferation of "SOCKET_ID" to refer
to a NUMA socket ID moving forward, and just use "NUMA_ID" to avoid
confusion with TCP sockets.

Change all of the existing in-tree SPDK_ENV_SOCKET_ID_ANY uses to
SPDK_ENV_NUMA_ID_ANY, but keep the old #define around, at least for
now. Also change all 'socket_id' parameters to 'numa_id'.

We still have spdk_env_get_socket_id(), we will need to keep this
but next patch will add spdk_env_get_numa_id().

Signed-off-by: Jim Harris <jim.harris@samsung.com>
Change-Id: Idc31c29e32b708c24d88f9c6fecaf9a99e34ba1e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/24607
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <ben@nvidia.com>
Community-CI: Mellanox Build Bot

show more ...

update Intel copyright notices

per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date defa

update Intel copyright notices

per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date default <file> | tail -1

and then pull just the 4 digit year from the result.

Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc). Contribution date used
"--follow -C95%" to get the most accurate date.

Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line. Simply there for
consistency.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id5b7ce4f658fe87132f14139ead58d6e285c04d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15192
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot

show more ...

virtio/vfio_user: add virtio_scsi device support

Example usage using `bdevperf` as the client:

Start `spdk_tgt` with created virtio_scsi device:
1. scripts/rpc.py bdev_malloc_create -b malloc0 $((5

virtio/vfio_user: add virtio_scsi device support

Example usage using `bdevperf` as the client:

Start `spdk_tgt` with created virtio_scsi device:
1. scripts/rpc.py bdev_malloc_create -b malloc0 $((512)) 512
2. scripts/rpc.py vfu_virtio_create_scsi_endpoint vfu.0 --cpumask 0x1 --num-io-queues=4 \
--qsize=128 --packed-ring
3. scripts/rpc.py vfu_virtio_scsi_add_target vfu.0 --scsi-target-num=0 --bdev-name malloc0

Start `bdevperf`:
1. test/bdev/bdevperf/bdevperf -r /var/tmp/spdk.sock.1 -g -s 2048 -q 128 -o 4096 \
-w randread -t 30 -m 0x2
2. scripts/rpc.py -s /var/tmp/spdk.sock.1 bdev_virtio_attach_controller --dev-type scsi \
--trtype vfio-user --traddr vfu.0 VirtioScsi0
3. test/bdev/bdevperf/bdevperf.py -s /var/tmp/spdk.sock.1 perform_tests

Change-Id: Id95908a5243e9a224a9bd709a22b87740c50b835
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13897
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

bdev_virtio: fix use-after-free in scsi scan_ctx

Found while debugging issue #2596, unfortunately this
is not the root cause of that issue.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Chan

bdev_virtio: fix use-after-free in scsi scan_ctx

Found while debugging issue #2596, unfortunately this
is not the root cause of that issue.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I27501e283ce7c9bf7a431e8b48842c83f80792c8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14165
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>

show more ...

module/bdev_virtio_scsi: use the correct `num_queues` value

Parameter `num_queues` for virtio_scsi PCI device means
maximum number of queues, it SHOULD include the `eventq`
and `controlq`, while for

module/bdev_virtio_scsi: use the correct `num_queues` value

Parameter `num_queues` for virtio_scsi PCI device means
maximum number of queues, it SHOULD include the `eventq`
and `controlq`, while for `vhost_user` RPC call, it means
the number of IO queues, so here we use it as `max_queues`
in lib/virtio and add the fixed number queues for `vhost_user`
SCSI device.

Also fix `vhost_fuzz` to get `num_queues` earlier than
negotiate the feature bits.

Change-Id: I41b3da5e4b4dc37127befd414226ea6eafcd9ad0
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13791
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

bdev/virtio_blk|scsi: don't negotiate `VHOST_USER_F_BITS` for PCI devices

VHOST_USER_F_PROTOCOL_FEATURES is used for `vhost_user` transport,
so unmask it for PCI devices.

Change-Id: If84d6c0ee75588

bdev/virtio_blk|scsi: don't negotiate `VHOST_USER_F_BITS` for PCI devices

VHOST_USER_F_PROTOCOL_FEATURES is used for `vhost_user` transport,
so unmask it for PCI devices.

Change-Id: If84d6c0ee7558886f14647dad07e41530e306206
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13790
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

check_format: For C files only, fix return type breaks

In SPDK, declarations have the return type on the same line. Definitions
have the return type on a separate line. Astyle has an option for
enfo

check_format: For C files only, fix return type breaks

In SPDK, declarations have the return type on the same line. Definitions
have the return type on a separate line. Astyle has an option for
enforcing this. Unfortunately, it seems to have two bugs:

1) It doesn't work correctly at all on C++ files.
2) It often fails on functions that return enums, or long type names

Deal with 1) by adjusting the check_format.sh script to only tell astyle
to fix return type line breaks for C files and not C++. Deal with 2) by
adding a few typedefs to work around the problem.

Change-Id: Idf28281466cab8411ce252d5f02ab384166790c6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13437
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>

show more ...

Replace most BSD 3-clause license text with SPDX identifier.

Many open source projects have moved to using SPDX identifiers
to specify license information, reducing the amount of
boilerplate code in

Replace most BSD 3-clause license text with SPDX identifier.

Many open source projects have moved to using SPDX identifiers
to specify license information, reducing the amount of
boilerplate code in every source file. This patch replaces
the bulk of SPDK .c, .cpp and Makefiles with the BSD-3-Clause
identifier.

Almost all of these files share the exact same license text,
and this patch only modifies the files that contain the
most common license text. There can be slight variations
because the third clause contains company names - most say
"Intel Corporation", but there are instances for Nvidia,
Samsung, Eideticom and even "the copyright holder".

Used a bash script to automate replacement of the license text
with SPDX identifier which is checked into scripts/spdx.sh.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa88ab5e92ea471691dc298cfe41ebfb5d169780
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12904
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: <qun.wan@intel.com>

show more ...

lib/bdev: added spdk_bdev_module_fini_done()

Callback for bdev modules is called 'module_fini',
meanwhile after its execution bdev modules were to call
'spdk_bdev_module_finish_done()'.

This functi

lib/bdev: added spdk_bdev_module_fini_done()

Callback for bdev modules is called 'module_fini',
meanwhile after its execution bdev modules were to call
'spdk_bdev_module_finish_done()'.

This function carries incorrect name, so it was deprecated
and replaced with 'spdk_bdev_module_fini_done()'.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I9a12dff746ea8b4b1570a3794470f7b24e29003e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9148
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>

show more ...

virtio: add transitional virtio device support

SPDK virtio driver does not support the virtio legacy
device but it's ok for the modern and transitional
device. So update the probe function.

From th

virtio: add transitional virtio device support

SPDK virtio driver does not support the virtio legacy
device but it's ok for the modern and transitional
device. So update the probe function.

From the spec:
Transitional PCI Device ID Virtio Device
0x1000 network card
0x1001 block device
0x1002 memory ballooning (traditional)
0x1003 console
0x1004 SCSI host
0x1005 entropy source
0x1009 9P transport

Transitional Device: a device supporting both drivers conforming to
modern specification, and allowing legacy drivers.

Change-Id: I28cd277fb2b2e07a429082b7d7bd581f254eae9c
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5053
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

module/virtio_blk/sci: remove legacy config support

This patch removes legacy config support in virtio_blk and virtio_scsi bdev modules.
Most of the options through the legacy config are already ref

module/virtio_blk/sci: remove legacy config support

This patch removes legacy config support in virtio_blk and virtio_scsi bdev modules.
Most of the options through the legacy config are already reflected in JSON.

Exception to that is "[VirtioPci] Enable" option, which worked for virtio_blk
and virtio_scsi. It then on initialization scanned virtio_pci devices for
presence of blk/scsi. There is no RPC that scans it on demand, nor was it used
in any of the tests. So this patch removes that functionality,
if needed this can be reintroduced as new RPC (or two for each backend).

Module initialization for bdev_virtio_scsi changed to be synchronous.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib0978a4a084795e94993bb9122bd73ed49e14b96
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4667
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

bdev/virtio: Use spdk_bdev_open_ext() instead of spdk_bdev_open()

It is not really necessary to replace spdk_bdev_open() by
spdk_bdev_open_ext() for virtio scsi bdev, but spdk_bdev_open() has been
d

bdev/virtio: Use spdk_bdev_open_ext() instead of spdk_bdev_open()

It is not really necessary to replace spdk_bdev_open() by
spdk_bdev_open_ext() for virtio scsi bdev, but spdk_bdev_open() has been
deprecated. Hence do the replacement in this patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I995c01c3c5842b306497e32099f6cfe767333091
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4725
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

log: remove internal log.h header

There is nothing left here, so remove it.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib947d42bc577dbebb4650b1be885e05a80f8f8cf
Reviewed-

log: remove internal log.h header

There is nothing left here, so remove it.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib947d42bc577dbebb4650b1be885e05a80f8f8cf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4541
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI

show more ...

log: simplify SPDK_LOG_REGISTER_COMPONENT

This patch removes the string from register component.
Removed are all instances in libs or hardcoded in apps.

Starting with this patch literal passed to r

log: simplify SPDK_LOG_REGISTER_COMPONENT

This patch removes the string from register component.
Removed are all instances in libs or hardcoded in apps.

Starting with this patch literal passed to register,
serves as name for the flag.

All instances of SPDK_LOG_* were replaced with just *
in lowercase.
No actual name change for flags occur in this patch.

Affected are SPDK_LOG_REGISTER_COMPONENT() and
SPDK_*LOG() macros.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I002b232fde57ecf9c6777726b181fc0341f1bb17
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4495
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Mellanox Build Bot
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI

show more ...

virtio: fix scsi double free issue

During virtio_pci_dev_probe, if enum_cb fails, hw needs
to be released. But in bdev_virtio, if vdev fails after
initialization, it will enter the bdev destruction

virtio: fix scsi double free issue

During virtio_pci_dev_probe, if enum_cb fails, hw needs
to be released. But in bdev_virtio, if vdev fails after
initialization, it will enter the bdev destruction process
which call the modern_destruct_dev function and hw will
be released during the process. So we will encounter the
problem of hw being released twice.

Change-Id: I1e8116283cfd810dfb050f8928f4ecd4bb2d815b
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3566
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>

show more ...

pollers: Fix pollers to return correct busy status

Poller should return status > 0 when it did some work
(CPU was used for some time) marking its call as busy
CPU time.

Active pollers should return

pollers: Fix pollers to return correct busy status

Poller should return status > 0 when it did some work
(CPU was used for some time) marking its call as busy
CPU time.

Active pollers should return BUSY status only if they
did any meangful work besides checking some conditions
(e.g. processing requests, do some complicated operations).

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Id4636a0997489b129cecfe785592cc97b50992ba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2164
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

lib/thread: Use function name as poller name by using macro SPDK_POLLER_REGISTER

We will be create fine name for each poller but it will need large
effort. Replacing spdk_poller_register by the macr

lib/thread: Use function name as poller name by using macro SPDK_POLLER_REGISTER

We will be create fine name for each poller but it will need large
effort. Replacing spdk_poller_register by the macro SPDK_POLLER_REGISTER
will provide better name than function address with minimum effort.

Following patches may improve function name for clarification.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: If862a274c5879065c3f7cb04dcb5ca7844523e68
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1781
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Community-CI: Broadcom CI

show more ...

virtio: remove VHOST_USER_F_PROTOCOL_FEATURES from virtio.h

spdk_internal/vhost_user.h head file defines common vhost user
protocol, and it can be used both in the vhost target and virtio
initiator,

virtio: remove VHOST_USER_F_PROTOCOL_FEATURES from virtio.h

spdk_internal/vhost_user.h head file defines common vhost user
protocol, and it can be used both in the vhost target and virtio
initiator, so remove the definition from virtio.h.

Change-Id: I1fac1cb5a16f803cd0d49962c07d2179f881c76a
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478411
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

RPC: rename construct_virtio_dev to bdev_virtio_attach_controller

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ic15183de16023a6edb5db3812c6d6e30fdafb8a7
Reviewed-on: https://re

RPC: rename construct_virtio_dev to bdev_virtio_attach_controller

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ic15183de16023a6edb5db3812c6d6e30fdafb8a7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468124
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

mk: move the bdev modules under module directory.

This is more accurate to what they are, and will make defining library
dependencies much simpler. This change in directory does not affect the
final

mk: move the bdev modules under module directory.

This is more accurate to what they are, and will make defining library
dependencies much simpler. This change in directory does not affect the
final placement of naming of libraries at the end of time.

Change-Id: Ic48a9233dff564e39ce357a9ea0a111ea2b6414b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465454
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...