sock: add sock_get_default_impl RPC

Default socket implementation may be set without explicitly calling
`sock_set_default_impl` RPC (POSIX impl is registered as default),
but we currently have no wa

sock: add sock_get_default_impl RPC

Default socket implementation may be set without explicitly calling
`sock_set_default_impl` RPC (POSIX impl is registered as default),
but we currently have no way of letting user know what type of
implementation is the default one via RPC.
Considering above and that we already have `set` option, we should
add `sock_get_default_impl`, so users can check the socket impl.

Fixes #3280

Change-Id: I50145770a74874f3e19f1f5685c9717bcb82672e
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/22140
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

sock: remove global PSK related code

We will no longer support setting global PSK and PSK
identity via sock_impl_set_options RPC - PSK related
operations are now handled by TCP transport.

Change-Id

sock: remove global PSK related code

We will no longer support setting global PSK and PSK
identity via sock_impl_set_options RPC - PSK related
operations are now handled by TCP transport.

Change-Id: I41d28dafc25696f98fd68239675bb1ce85629045
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/17603
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>

show more ...

rpc: fix python style

A bunch of python files in rpc and scripts directories were missing two
blank lines after the opening comments. python3-pycodestyle-2.5.0-2.el8
was unhappy with this

Signed-of

rpc: fix python style

A bunch of python files in rpc and scripts directories were missing two
blank lines after the opening comments. python3-pycodestyle-2.5.0-2.el8
was unhappy with this

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Iee7f2bd4f0ddd96b2be89949a3aa324b2f9ea43a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16003
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>

show more ...

Add SPDX header to various files

They were missed by the initial set of patches which introduced this
header as a mandatory one across different types of files.

Signed-off-by: Michal Berger <michal

Add SPDX header to various files

They were missed by the initial set of patches which introduced this
header as a mandatory one across different types of files.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: I3f9b37d41298c843e1648e72fe8593768ccd37e0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15423
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

sock/ssl: Add psk_key and psk_identity options to spdk_sock_impl_opts

Note, this change only sets defaults for the ID/KEY,
more specific use cases like NVMe/TCP may set the ID and KEY on a per conne

sock/ssl: Add psk_key and psk_identity options to spdk_sock_impl_opts

Note, this change only sets defaults for the ID/KEY,
more specific use cases like NVMe/TCP may set the ID and KEY on a per connection basis.

Also simplify PSK identity string, that isn't NVMe focused.
NVMe libraries using this will need to construct more complicated
identity strings and pass them to the sock layer.

Example:
rpc.py sock_impl_set_options -i ssl --psk-key 4321DEADBEEF1234
rpc.py sock_impl_set_options -i ssl --psk-identity psk.spdk.io

./build/examples/perf --psk-key 4321DEADBEEF1234 --psk-identity psk.spdk.io

./build/examples/hello_sock --psk-key 4321DEADBEEF1234 --psk-identity psk.spdk.io

Change-Id: I1cb5b0b706bdeafbccbc71f8320bc8e2961cbb55
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13759
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot

show more ...

sock: Add ktls and tls_version to spdk_sock_impl_opts

Since `sock_impl_opts` was added to `sock_opts`
Can remove `ktls` and `tls_version` from spdk_sock_opts

Example:
rpc.py sock_impl_set_options

sock: Add ktls and tls_version to spdk_sock_impl_opts

Since `sock_impl_opts` was added to `sock_opts`
Can remove `ktls` and `tls_version` from spdk_sock_opts

Example:
rpc.py sock_impl_set_options -i ssl --enable-ktls
rpc.py sock_impl_set_options -i ssl --disable-ktls
rpc.py sock_impl_set_options -i ssl --tls-version=12

./build/examples/perf --enable-ktls
./build/examples/perf --disable-ktls
./build/examples/perf --tls-version=12

Check kTLS statistics here: /proc/net/tls_stat

Change-Id: Icf7ee822bad92fda149710be77feb77fc8d4f163
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13510
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

sock: fix typo

`enable_dynamic_zerocopy` is not even defined anywhere

Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Change-Id: I5c7a97bf661db233a5bf62f8eb00170685790cee
Signed-off-by: Bori

sock: fix typo

`enable_dynamic_zerocopy` is not even defined anywhere

Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Change-Id: I5c7a97bf661db233a5bf62f8eb00170685790cee
Signed-off-by: Boris Glimcher <Boris.Glimcher@emc.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12436
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

sock: introduce dynamic zerocopy according to data size

MSG_ZEROCOPY is not always effective as mentioned in
https://www.kernel.org/doc/html/v4.15/networking/msg_zerocopy.html.

Currently in spdk, o

sock: introduce dynamic zerocopy according to data size

MSG_ZEROCOPY is not always effective as mentioned in
https://www.kernel.org/doc/html/v4.15/networking/msg_zerocopy.html.

Currently in spdk, once we enable sendmsg zerocopy, then all data
transferred through _sock_flush are sent with zerocopy, and vice
versa. Here dynamic zerocopy is introduced to allow data sent with
MSG_ZEROCOPY or not according to its size, which can be enabled by
setting "enable_dynamic_zerocopy" as true.

Test with 16 P4610 NVMe SSD, 2 initiators, target's and initiators'
configurations are the same as spdk report:
https://ci.spdk.io/download/performance-reports/SPDK_tcp_perf_report_2104.pdf

For posix socket, rw_percent=0(randwrite), it has 1.9%~8.3% performance boost
tested with target 1~40 cpu cores and qdepth=128,256,512. And it has no obvious
influence when read percentage is greater than 50%.

For uring socket, rw_percent=0(randwrite), it has 1.8%~7.9% performance boost
tested with target 1~40 cpu cores and qdepth=128,256,512. And it still has
1%~7% improvement when read percentage is greater than 50%.

The following is part of the detailed data.

posix:
qdepth=128
rw_percent 0 | 30
cpu origin thisPatch opt | origin thisPatch opt
1 286.5 298.5 4.19% 307 304.15 -0.93%
4 1042.5 1107 6.19% 1135.5 1136 0.04%
8 1952.5 2058 5.40% 2170.5 2170.5 0.00%
12 2658.5 2879 8.29% 3042 3046 0.13%
16 3247.5 3460.5 6.56% 3793.5 3775 -0.49%
24 4232.5 4459.5 5.36% 4614.5 4756.5 3.08%
32 4810 5095 5.93% 4488 4845 7.95%
40 5306.5 5435 2.42% 4427.5 4902 10.72%

qdepth=512
rw_percent 0 | 30
cpu origin thisPatch opt | origin thisPatch opt
1 275 287 4.36% 294.4 295.45 0.36%
4 979 1041 6.33% 1073 1083.5 0.98%
8 1822.5 1914.5 5.05% 2030.5 2018.5 -0.59%
12 2441 2598.5 6.45% 2808.5 2779.5 -1.03%
16 2920.5 3109.5 6.47% 3455 3411.5 -1.26%
24 3709 3972.5 7.10% 4483.5 4502.5 0.42%
32 4225.5 4532.5 7.27% 4463.5 4733 6.04%
40 4790.5 4884.5 1.96% 4427 4904.5 10.79%

uring:
qdepth=128
rw_percent 0 | 30
cpu origin thisPatch opt | origin thisPatch opt
1 270.5 287.5 6.28% 295.75 304.75 3.04%
4 1018.5 1089.5 6.97% 1119.5 1156.5 3.31%
8 1907 2055 7.76% 2127 2211.5 3.97%
12 2614 2801 7.15% 2982.5 3061.5 2.65%
16 3169.5 3420 7.90% 3654.5 3781.5 3.48%
24 4109.5 4414 7.41% 4691.5 4750.5 1.26%
32 4752.5 4908 3.27% 4494 4825.5 7.38%
40 5233.5 5327 1.79% 4374.5 4891 11.81%

qdepth=512
rw_percent 0 | 30
cpu origin thisPatch opt | origin thisPatch opt
1 259.95 276 6.17% 286.65 294.8 2.84%
4 955 1021 6.91% 1070.5 1100 2.76%
8 1772 1903.5 7.42% 1992.5 2077.5 4.27%
12 2380.5 2543.5 6.85% 2752.5 2860 3.91%
16 2920.5 3099 6.11% 3391.5 3540 4.38%
24 3697 3912 5.82% 4401 4637 5.36%
32 4256.5 4454.5 4.65% 4516 4777 5.78%
40 4707 4968.5 5.56% 4400.5 4933 12.10%

Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: I730dcf89ed2bf3efe91586421a89045fc11c81f0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12210
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

scripts: move python modules to python directory

Up until now, importing an SPDK RPC python module was just a matter of
`import rpc`. It's fine until there's another module called `rpc`
installed o

scripts: move python modules to python directory

Up until now, importing an SPDK RPC python module was just a matter of
`import rpc`. It's fine until there's another module called `rpc`
installed on the system, in which case it's impossible to import both of
them. Therefore, to avoid this problem, all of the modules were moved
to a separate directory under the "spdk" namespace.

The decision to move to a location under a separate directory was
motivated by the fact that a directory called scripts/spdk would look
pretty confusing. Moreover, it should make it also easier to package
these scripts as a python package.

Other than moving the packages, all of the imports were updated to
reflect these changes. Files under python now use relative imports,
while those under scripts/ use the "spdk" namespace and have their
PYTHONPATH extended with python directory.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ib43dee73921d590a551dd83885e22870e72451cf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9692
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...