fsdev/aio: Add aio fsdev implementation

Change-Id: I5add43886241803eb3a65d64beae58ddd91d335d
Signed-off-by: Anton Nayshtut <anayshtut@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/sp

fsdev/aio: Add aio fsdev implementation

Change-Id: I5add43886241803eb3a65d64beae58ddd91d335d
Signed-off-by: Anton Nayshtut <anayshtut@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22534
Reviewed-by: Ben Walker <ben@nvidia.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>

show more ...

module/keyring: add keyring_file module

This module allows for using keys stored in files. Each such file
should only contain a single key and the module copies it without any
modifications. Addit

module/keyring: add keyring_file module

This module allows for using keys stored in files. Each such file
should only contain a single key and the module copies it without any
modifications. Additionally, it checks the permissions of the file to
ensure that only the owner of the file is allowed to access it.

This mechanism is very similar to the method employed by the NVMe driver
and the NVMe-oF target to supply NVMe/TLS pre-shared keys.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I6eb15549073e8388b3f806f2888df3b1e3676ad5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21737
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seung yeon Shin <syeon.shin@samsung.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>

show more ...

build: regenerate modules pkgconfig files when config changes

If modules are added or removed from the current configuration
when re-running "configure" (i.e. --with-idxd), we need to
also generate

build: regenerate modules pkgconfig files when config changes

If modules are added or removed from the current configuration
when re-running "configure" (i.e. --with-idxd), we need to
also generate the associated pkgconfig file that lists all of
the modules for that category (i.e. spdk_accel_modules.pc).

Do this by adding mk/config.mk as a dependency for each of the
modules pkgconfig files.

Fixes issue #3018.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ied37f378aeba2eddc44c0649710a9e8e43e4d35b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/18233
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

update Intel copyright notices

per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date defa

update Intel copyright notices

per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date default <file> | tail -1

and then pull just the 4 digit year from the result.

Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc). Contribution date used
"--follow -C95%" to get the most accurate date.

Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line. Simply there for
consistency.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id5b7ce4f658fe87132f14139ead58d6e285c04d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15192
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot

show more ...

module/vfu_device: add virtio-blk emulation

Here we use vfu-tgt library and emulate a virtio-blk device
as the first use case of vfu-tgt library.

Usage example with QEMU:

1. build/bin/spdk_tgt
2.

module/vfu_device: add virtio-blk emulation

Here we use vfu-tgt library and emulate a virtio-blk device
as the first use case of vfu-tgt library.

Usage example with QEMU:

1. build/bin/spdk_tgt
2. scripts/rpc.py bdev_malloc_create -b malloc0 $((512)) 512
3. scripts/rpc.py vfu_virtio_create_blk_endpoint vfu.0 --bdev-name malloc0 \
--cpumask=0x1 --num-queues=2 \
--qsize=256 --packed-ring
4. Start QEMU with '-device vfio-user-pci,socket=/spdk/vfu.0'

Change-Id: I45e45360c669584583b0b8a3f83250ab6c48efec
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12315
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot

show more ...

Replace most BSD 3-clause license text with SPDX identifier.

Many open source projects have moved to using SPDX identifiers
to specify license information, reducing the amount of
boilerplate code in

Replace most BSD 3-clause license text with SPDX identifier.

Many open source projects have moved to using SPDX identifiers
to specify license information, reducing the amount of
boilerplate code in every source file. This patch replaces
the bulk of SPDK .c, .cpp and Makefiles with the BSD-3-Clause
identifier.

Almost all of these files share the exact same license text,
and this patch only modifies the files that contain the
most common license text. There can be slight variations
because the third clause contains company names - most say
"Intel Corporation", but there are instances for Nvidia,
Samsung, Eideticom and even "the copyright holder".

Used a bash script to automate replacement of the license text
with SPDX identifier which is checked into scripts/spdx.sh.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa88ab5e92ea471691dc298cfe41ebfb5d169780
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12904
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: <qun.wan@intel.com>

show more ...

scheduler: create public API and subsystem for scheduler/governor

This patch moves schedueler and governor related API from
the internal event.h to public scheduler.h.

With this it is possible to c

scheduler: create public API and subsystem for scheduler/governor

This patch moves schedueler and governor related API from
the internal event.h to public scheduler.h.

With this it is possible to create subsystem responsible
for handling the schedulers.
Three schedulers and a governor were moved to scheduler modules
from event framework.

This will allow next patch to add JSON RPC configuration
to the whole subsystem.
Along with easier addition of other schedulers.

Removed debug logs from gscheduler, as they serve little purpose.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I98ca1ea4fb281beb71941656444267842a8875b7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6995
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

build: install generated pkg-config files

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I191ad5e3b153fb563256eba1aa695716f66db788
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/

build: install generated pkg-config files

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I191ad5e3b153fb563256eba1aa695716f66db788
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6377
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

build: generate pkg-config files for SPDK

Users can now generate the necessary linker args for their
own applications using something like:

PKG_CONFIG_PATH=build/lib/pkgconfig pkg-config --libs spd

build: generate pkg-config files for SPDK

Users can now generate the necessary linker args for their
own applications using something like:

PKG_CONFIG_PATH=build/lib/pkgconfig pkg-config --libs spdk_nvme

Dependencies between libraries are included in the generated
.pc files, so the user only needs to pass the top-level subsystems
or individual SPDK libraries they are using in their application.

Modules will automatically be added to the output if the associated
library is specified. For example, specifying "spdk_bdev" will include
the libraries not only for spdk_bdev, but also all of the bdev modules.

Users still need to supply the -Wl,--no-as-needed or -Wl,--whole-archive
flags. They cannot be added to the .pc files without increasing the length
of the argument string by a factor of 15x to 20x.

Modify the test/external_code/hello_world Makefile to use pkg-config to
ensure this gets tested at some level in our autotest environment.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie48a75f11969d5d775d514cf10bcb82d197eabfd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4371
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

global: rename copy to accel

The copy engine library, modules and public APIs have been renamed.
Use of the word `copy` has been replaced with the word `accel`
short for accelerator in preparation f

global: rename copy to accel

The copy engine library, modules and public APIs have been renamed.
Use of the word `copy` has been replaced with the word `accel`
short for accelerator in preparation for adding new capabilities
in the future. Additionally, APIs for what was previously called
the `memcpy` engine have been renamed to identify the engine as a
software accelerator.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ia607aa718416146fbba1e6792b8de0f66bd8a5de
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/576
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

env_dpdk/rpc: add rpc to get memory stats.

This will enable us to create a helper applciation to print memory
statistics for a given SPDK application.

Change-Id: I0ad5a8163166a5d67d9a5533303ec08582

env_dpdk/rpc: add rpc to get memory stats.

This will enable us to create a helper applciation to print memory
statistics for a given SPDK application.

Change-Id: I0ad5a8163166a5d67d9a5533303ec0858297ba6b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477511
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

mk : Fix compiling error.

Add DEPDIR in module Makefile for blobfs.
Avoid compiling blobfs_bdev before blob_bdev.

This is to fix issue: #982

Change-Id: I7f7bd35ec56cb5a7d0545d9966aaeb7249ffb4f4
Si

mk : Fix compiling error.

Add DEPDIR in module Makefile for blobfs.
Avoid compiling blobfs_bdev before blob_bdev.

This is to fix issue: #982

Change-Id: I7f7bd35ec56cb5a7d0545d9966aaeb7249ffb4f4
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471019
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>

show more ...

blobfs: add blobfs_detect RPC

Add module blobfs_bdev as a general module to simplify
the operations of blobfs on bdev. Then its functions
can be utilized by other libraries or apps.

blobfs_detect c

blobfs: add blobfs_detect RPC

Add module blobfs_bdev as a general module to simplify
the operations of blobfs on bdev. Then its functions
can be utilized by other libraries or apps.

blobfs_detect can be used to detect whether there is
one blobfs on given bdev.

Change-Id: Ib425574816061dc945fb652b539f791a44097a43
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466486
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>

show more ...

module: add shared lib deps for all modules.

This will allow us to link individually against dpdk module libraries
without having to define those libraries dependencies.

Change-Id: Ief2140ec0fadd97

module: add shared lib deps for all modules.

This will allow us to link individually against dpdk module libraries
without having to define those libraries dependencies.

Change-Id: Ief2140ec0fadd970aba990dab333d603dfb46317
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465704
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

lib/copy: move ioat subdir to module directory.

Change-Id: I29b31344a11efd2430e0f77062fb98fa0ab1c71e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spd

lib/copy: move ioat subdir to module directory.

Change-Id: I29b31344a11efd2430e0f77062fb98fa0ab1c71e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465458
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lib/blob: move bdev subdir under module directory.

Change-Id: Ifb9a1df919d32a98c328101029cc22e91915a977
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/

lib/blob: move bdev subdir under module directory.

Change-Id: Ifb9a1df919d32a98c328101029cc22e91915a977
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465457
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lib/sock: move subdirs to module/sock

Rounding out the module concept of SPDK libraries.

Change-Id: I2b316153809ae9f73361648fe505274a59d0bdb3
Signed-off-by: Seth Howell <seth.howell@intel.com>
Revi

lib/sock: move subdirs to module/sock

Rounding out the module concept of SPDK libraries.

Change-Id: I2b316153809ae9f73361648fe505274a59d0bdb3
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465456
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lib/event: move rpc and subsystems dirs to module

These directories fit in with the module concept we are forming inside
of SPDK. Essentially modules are derivative or specialized libraries that
rel

lib/event: move rpc and subsystems dirs to module

These directories fit in with the module concept we are forming inside
of SPDK. Essentially modules are derivative or specialized libraries that
rely on a general or core library.

Change-Id: Ib40f05422f144ff8fd579f47a3867ef4412b3372
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465455
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

mk: move the bdev modules under module directory.

This is more accurate to what they are, and will make defining library
dependencies much simpler. This change in directory does not affect the
final

mk: move the bdev modules under module directory.

This is more accurate to what they are, and will make defining library
dependencies much simpler. This change in directory does not affect the
final placement of naming of libraries at the end of time.

Change-Id: Ic48a9233dff564e39ce357a9ea0a111ea2b6414b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465454
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...