Changes in Makefile

2dc4a231ac65d10dd2e1a96684094bef1b7ebb95 - blob: Add support for variable metadata page size

Tue, 22 Oct 2024 21:20:10 +0000

blob: Add support for variable metadata page sizeCurrently SPDK blobstore depends on 4KB atomicity of metadata.So, Metadata page size will be made variable based on the physicalblock size reported by the underlying device. This will enableblobstore to store metadata in IU-sized chunks, not always 4KB.blobstore.c uses SPDK_BS_PAGE_SIZE (4KB) hardcoded at many places.To remove this make the metadata page size variable,phys_blocklen is added to spdk_bs_dev struct andmd_page_size is added to spdk_bs_super_block struct.Change-Id: I29d073eb4f4341a94a0675e70492b9186382f97fSigned-off-by: Atul Malakar Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25130Community-CI: Broadcom CI Community-CI: Mellanox Build BotReviewed-by: Ben Walker Tested-by: SPDK CI Jenkins Community-CI: Community CI Samsung Reviewed-by: Konrad Sztyber Reviewed-by: Jim Harris List of files: /spdk/module/blob/bdev/Makefile

355312bfcd3751f9af17fbefc90373fb8a269614 - so_ver: increase all major versions after SPDK 24.01

Fri, 26 Jan 2024 11:19:16 +0000

so_ver: increase all major versions after SPDK 24.01To allow SO_MINOR updates on LTS for the whole year it is supported,the major version for all components needs to be increased.This is to prevent scenario where two versions exists with matchingversions, but conflicting ABI.Ex. Next SPDK release adds an API call increasing the minor version,then LTS needs just a subset of those additions.Increasing major so version after LTS, allows the future releasesto update versions as needed. Yet allowing LTS to increase minorversion separately.This patch:- increases SO_VER by 1 for all components- resets SO_MINOR to 0 for all componentsDue to patch below being introduced after v24.01 code freeze,bdev lib version should not be increased:61623c5ca9 bdev: add spdk_bdev_io padding in place of io_submit_chShort reference to how the versions were changed:MAX=$(git grep "SO_VER := " | cut -d" " -f 3 | sort -ubnr | head -1)for((i=$MAX;i>0;i-=1)); do find . -name "Makefile" -exec \ sed -i -e "s/SO_VER := $i\$/SO_VER := $(($i+1))/g" {} +; donefind . -name "Makefile" -exec \ sed -i -e "s/SO_MINOR := .*/SO_MINOR := 0/g" {} +Signed-off-by: Tomasz Zawadzki Change-Id: I0166d6be6c17bbbf985d48b5dfcb36f1d4af1b48Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21582Reviewed-by: Konrad Sztyber Tested-by: SPDK CI Jenkins List of files: /spdk/module/blob/bdev/Makefile

91c0a9d2fa436f47fa15df5ddd738ee71649e53f - module/blob_bdev: add possibility to change size of bs_dev

Thu, 17 Aug 2023 13:43:52 +0000

module/blob_bdev: add possibility to change size of bs_devspdk bdevs can change their size in run time,so far there was no way for bs_dev to change its size toowithout re-creating it.Blobstore size in general is determined from size written downin super block. Size of bs_dev is used in following cases:- initializing blobstore (therefore will become super->size)- loading blobstore (to check if super->size is not larger)- growing blobstore (combines two of the above)- parsing/writing out super block when super->size is 0Last one is for compatibility with old versions of blobstore.This effectively means bs_dev size is not used in run-time,changing it will not affect blobstore even after reloads.Either way re-creating bs_dev already changed its size.This patch proposes adding interface that will allow to changethe bs_dev size in run-time, and later in the series adda way to grow blobstore without reloading it.There is no need to add a way to revert the bs_dev size.All error handling for changes in super->size is done inblobstore.Signed-off-by: Tomasz Zawadzki Change-Id: I3d7872d91f3b24d40976b4a8c64df860ddf642aeReviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19453Tested-by: SPDK CI Jenkins Reviewed-by: Jim Harris Reviewed-by: Shuhei Matsumoto Community-CI: Mellanox Build BotReviewed-by: Aleksey Marchuk List of files: /spdk/module/blob/bdev/Makefile

3359bf34d682202107c227ea42ace1c600a04657 - so_ver: increase all major versions

Mon, 23 Jan 2023 17:24:28 +0000

so_ver: increase all major versionsTo allow SO_MINOR updates on LTS for the whole year it is supported,the major version for all components needs to be increased.This is to prevent scenario where two versions exists with matchingversions, but conflicting ABI.Ex. Next SPDK release adds an API call increasing the minor version,then LTS needs just a subset of those additions.Increasing major so version after LTS, allows the future releasesto update versions as needed. Yet allowing LTS to increase minorversion separately.Disabled test for increasing SO version without ABI change, asthat is goal of this patch. This check shall be removed with SPDK 23.05release.Looks like this was left over from prior LTS, to avoid thatmake sure it is only skipped when running against v23.01.x as latestrelease.This patch:- increases SO_VER by 1 for all components- resets SO_MINOR to 0 for all components- removes suppressions for ABI testsShort reference to how the versions were changed:MAX=$(git grep "SO_VER := " | cut -d" " -f 3 | sort -ubnr | head -1)for((i=$MAX;i>0;i-=1)); do find . -name "Makefile" -exec \ sed -i -e "s/SO_VER := $i\$/SO_VER := $(($i+1))/g" {} +; donefind . -name "Makefile" -exec \ sed -i -e "s/SO_MINOR := .*/SO_MINOR := 0/g" {} +Change-Id: I3e5681802c0a5ac6d7d652a18896997cd07cc8bfSigned-off-by: Tomasz Zawadzki Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16419Reviewed-by: Ben Walker Reviewed-by: Konrad Sztyber Tested-by: SPDK CI Jenkins List of files: /spdk/module/blob/bdev/Makefile

9e843fdbd1f39af220e39a0a31ce33c91c89b902 - blob: Add translate_lba operation

Thu, 15 Sep 2022 21:56:12 +0000

blob: Add translate_lba operationNew `translate_lba` operation allows to translate blob lba to lba onthe underlying bdev. It recurses down the whole chain of bs_dev's. Theoperation may fail to do the translation when blob lba is not backedby the real bdev. For example, when we eventually hit zeroes device inthe chain.This operation is used in the next commit to get source LBA for copyoperation.Signed-off-by: Evgeniy Kochetov Change-Id: I89c2d03d1982d66b9137a3a3653a98c361984fabReviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14528Tested-by: SPDK CI Jenkins Reviewed-by: Jim Harris Reviewed-by: Shuhei Matsumoto Reviewed-by: Aleksey Marchuk List of files: /spdk/module/blob/bdev/Makefile

a6dbe3721eb3b5990707fc3e378c95e505dd8ab5 - update Intel copyright notices

Tue, 01 Nov 2022 20:26:26 +0000

update Intel copyright noticesper Intel policy to include file commit date using git cmdbelow. The policy does not apply to non-Intel (C) notices.git log --follow -C90% --format=%ad --date default | tail -1and then pull just the 4 digit year from the result.Intel copyrights were not added to files where Intel either hadno contribution ot the contribution lacked substance (ie licenseheader updates, formatting changes, etc). Contribution date used"--follow -C95%" to get the most accurate date.Note that several files in this patch didn't end the license/(c)block with a blank comment line so these were added as the vastmajority of files do have this last blank line. Simply there forconsistency.Signed-off-by: paul luse Change-Id: Id5b7ce4f658fe87132f14139ead58d6e285c04d4Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15192Tested-by: SPDK CI Jenkins Reviewed-by: Jim Harris Reviewed-by: Ben Walker Community-CI: Mellanox Build Bot List of files: /spdk/module/blob/bdev/Makefile

2e7a7fe530576930ae3abd3fe93bcaa39850066e - blob: Optimize copy-on-write flow for clusters backed by zeroes device

Wed, 22 Jun 2022 13:55:46 +0000

blob: Optimize copy-on-write flow for clusters backed by zeroes deviceWriting to unallocated cluster triggers copy-on-write sequence. Ifthis cluster is backed by zeroes device we can skip the copy part. Fora simple thin provisioned volume copy this shortcut is alreadyimplemented because `blob->parent_id == SPDK_BLOBID_INVALID`. But thiswill not work for thin provisioned volumes created from snapshot. Inthis case we need to traverse the whole stack of underlying`spdk_bs_dev` devices for specific cluster to check if it is zeroesbacked.This patch adds `is_zeroes` operation to `spdk_bs_dev`. For zeroesdevice it always returns 'true', for real bdev (`blob_bs_dev`) alwaysreturns false, for another layer of `blob_bs_dev` does lba conversionand forwards to backing device.In blobstore's cluster copy flow we check if cluster is backed byzeroes device and skip copy part if it is.Signed-off-by: Evgeniy Kochetov Change-Id: I640773ac78f8f466b96e96a34c3a6c3c91f87dabReviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13446Tested-by: SPDK CI Jenkins Community-CI: Mellanox Build BotCommunity-CI: Broadcom CI Reviewed-by: Shuhei Matsumoto Reviewed-by: Aleksey Marchuk Reviewed-by: Jim Harris Reviewed-by: Tomasz Zawadzki List of files: /spdk/module/blob/bdev/Makefile

488570ebd418ba07c9e69e65106dcc964f3bb41b - Replace most BSD 3-clause license text with SPDX identifier.

Fri, 03 Jun 2022 19:15:11 +0000

Replace most BSD 3-clause license text with SPDX identifier.Many open source projects have moved to using SPDX identifiersto specify license information, reducing the amount ofboilerplate code in every source file. This patch replacesthe bulk of SPDK .c, .cpp and Makefiles with the BSD-3-Clauseidentifier.Almost all of these files share the exact same license text,and this patch only modifies the files that contain themost common license text. There can be slight variationsbecause the third clause contains company names - most say"Intel Corporation", but there are instances for Nvidia,Samsung, Eideticom and even "the copyright holder".Used a bash script to automate replacement of the license textwith SPDX identifier which is checked into scripts/spdx.sh.Signed-off-by: Jim Harris Change-Id: Iaa88ab5e92ea471691dc298cfe41ebfb5d169780Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12904Community-CI: Broadcom CI Community-CI: Mellanox Build BotTested-by: SPDK CI Jenkins Reviewed-by: Aleksey Marchuk Reviewed-by: Changpeng Liu Reviewed-by: Dong Yi Reviewed-by: Konrad Sztyber Reviewed-by: Paul Luse Reviewed-by: List of files: /spdk/module/blob/bdev/Makefile

047c067c05ff22df06c053990998184b8a940514 - so_ver: increase all major versions

Mon, 31 Jan 2022 09:55:58 +0000

so_ver: increase all major versionsTo allow SO_MINOR updates on LTS for the whole year it is supported,the major version for all components needs to be increased.This is to prevent scenario where two versions exists with matchingversions, but conflicting ABI.Ex. Next SPDK release adds an API call increasing the minor version,then LTS needs just a subset of those additions.Increasing major so version after LTS, allows the future releasesto update versions as needed. Yet allowing LTS to increase minorversion separately.Disabled test for increasing SO version without ABI change, asthat is goal of this patch. This check shall be removed with SPDK 22.05release.This patch:- increases SO_VER by 1 for all components- resets SO_MINOR to 0 for all components- removes suppressions for ABI testsSigned-off-by: Tomasz Zawadzki Change-Id: Id1a5358882dc496faa5b0b5c9a63b326c378c551Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11361Tested-by: SPDK CI Jenkins Community-CI: Broadcom CI Reviewed-by: Jim Harris Reviewed-by: Ben Walker List of files: /spdk/module/blob/bdev/Makefile

413d25b0362384f3343c3df11776d55320b79bdd - bdev_module: add async fini_start()

Wed, 04 Aug 2021 09:31:27 +0000

bdev_module: add async fini_start()fini_start() is called for each bdev module beforeiterating over all unclaimed bdevs to unregister them.This allows bdev modules to behave differently duringeach such unregister. Ex. unloading lvol store whenall lvol bdevs on it are unregistered.Another use of this callback is to unclaim all bdevsthat can be at that point. Especially ones that willnot receive callback due to no bdev registered.Ex. offline raid bdev, when some underlying bdevs are missing.fini_start() being synchronous does not help in caseswhere to release claim on the bdev, an asynchronous operationis required. Ex. lvol store with no bdevs present, requiresasync lvs unload to be called.This patch adds async_fini_start flag for the bdev modules,to be used when async fini_start is required. When done,bdev module has to call spdk_bdev_module_finish_start_done().Signed-off-by: Tomasz Zawadzki Change-Id: I63438b325d4cc53fd236bf9ff143abf6bdd81c49Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9094Community-CI: Broadcom CI Community-CI: Mellanox Build BotTested-by: SPDK CI Jenkins Reviewed-by: Jim Harris Reviewed-by: Aleksey Marchuk List of files: /spdk/module/blob/bdev/Makefile

e4070ee0e012c345f4c059f125f4f9ce3be66690 - so_ver: increase all major versions

Fri, 29 Jan 2021 10:11:25 +0000

so_ver: increase all major versionsTo allow SO_MINOR updates on LTS for the whole year it is supported,the major version for all components needs to be increased.This is to prevent scenario where two versions exists with matchingversions, but conflicting ABI.Ex. Next SPDK release adds an API call increasing the minor version,then LTS needs just a subset of those additions.Increasing major so version after LTS, allows the quarterly releasesto update versions as needed. Yet allowing LTS to increase minorversion separately.Disabled test for increasing SO version without ABI change, asthat is goal of this patch. This check shall be removed with SPDK 21.04release.This patch:- increases SO_VER by 1 for all components- resets SO_MINOR to 0 for all components- removes suppressions for ABI testsSigned-off-by: Tomasz Zawadzki Change-Id: I44d01154430a074103bd21c7084f44932e81fe72Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6167Community-CI: Broadcom CICommunity-CI: Mellanox Build BotTested-by: SPDK CI Jenkins Reviewed-by: Ben Walker Reviewed-by: Jim Harris List of files: /spdk/module/blob/bdev/Makefile

296a6d98389b7cbf95f57fc05c3565c2d65229e5 - bdev: extend interface with get module ctx

Tue, 27 Oct 2020 16:13:54 +0000

bdev: extend interface with get module ctxCurrently only nvme bdev module implements this interface. Bdev modulecontext (in this case spdk_nvme_ctrlr opaque handle) allows for nvmeinterface usage for additional management.Signed-off-by: Jacek Kalwas Change-Id: I6302c9229d5f7f294a3c1472d9e8dc1519637ffbReviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4924Tested-by: SPDK CI Jenkins Reviewed-by: Tomasz Zawadzki Reviewed-by: Shuhei Matsumoto Reviewed-by: Aleksey Marchuk List of files: /spdk/module/blob/bdev/Makefile

6a72c19ee2952f388b12150e6bda716ee6465c54 - blob/bdev: Add an API spdk_bdev_create_bs_dev_ext()

Wed, 14 Oct 2020 14:55:06 +0000

blob/bdev: Add an API spdk_bdev_create_bs_dev_ext()spdk_bdev_open_ext() requires the caller to use bdev_event_cb_tand bdev_event_cb_t is more extensible than bdev_remove_cb_t.Hence use bdev_event_t as an argument.spdk_bdev_open_ext() calls spdk_bdev_get_by_name() inside andspdk_bdev_create_bs_dev_ext() calls spdk_bdev_open_ext() inside.The caller needs to know if the spdk_bdev_get_by_name() succeeded.Hence spdk_bdev_create_bs_dev_ext() returns return code by returnvalue and returns the created bs_dev by double pointer.Signed-off-by: Shuhei Matsumoto Change-Id: I1c225bfb66db036439c69c459f39c86684d8a540Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4692Tested-by: SPDK CI Jenkins Reviewed-by: Jim Harris Reviewed-by: Tomasz Zawadzki Reviewed-by: Aleksey Marchuk List of files: /spdk/module/blob/bdev/Makefile

7cd20dd3f526292dc01e7256865013474904391f - lib/bdev: Add spdk_bdev_abort API

Wed, 03 Jun 2020 01:11:28 +0000

lib/bdev: Add spdk_bdev_abort APIAdd spdk_bdev_abort function as a new public API.This goes all the way down to the bdev driver module and attempts toabort all I/Os which has bio_cb_arg as its callback argument.We can separate when only a single I/O has bio_cb_arg and when multipleI/Os have bio_cb_arg, but unify both by using parent - children I/Orelationship. To avoid confusion, return matched_ios by _bdev_abort() andstore it into split_outstanding by the caller.Exclude any I/O submitted after this abort command because the same cb_argmay be used by all I/Os and abort may never complete.bdev_io needs to have both bio_cb_arg and bio_to_abort because bio_cb_argis used to continue abort processing when it is stopped due to the capacityof bdev_io pool, and bio_to_abort is used to pass it to the underlyingbdev module at submission. Parent I/O is not submitted directly, and isonly used in the generic bdev layer, and parent I/O's bdev_io uses bio_cb_arg.Hence add bio_cb_arg to bdev structure and add bio_to_abort to abort structure.In the meantime of abort operation, target I/Os may be completed. Hencecheck if the target I/O still exists at completion, and set the completionstatus to false only if it still exists.Upon completion of this, i.e., this returned zero, the statusSPDK_BDEV_IO_STATUS_SUCCESS indicates all I/Os were successfully aborted,or the status SPDK_BDEV_IO_STATUS_FAILED indicates any I/O was failed toabort by any reason.spdk_bdev_abort() does not support aborting abort or reset requestdue to the complexity for now.Following patches will support I/O split case.Add unit tests together to cover the basic paths.Besides, ABI compatibility check required us to bump up SO version ofa few libraries or modules. Bump up SO version of blob bdev module simplybecause it does not have any out-of-tree consumer, and suppress bumpingup SO version of lvol library because the affected struct spdk_lvolis not part of public APIs.Signed-off-by: Shuhei Matsumoto Change-Id: I515da688503557615d491bf0bfb36322ce37df08Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2014Tested-by: SPDK CI Jenkins Community-CI: Mellanox Build BotCommunity-CI: Broadcom CIReviewed-by: Jim Harris Reviewed-by: Aleksey Marchuk Reviewed-by: Ben Walker List of files: /spdk/module/blob/bdev/Makefile

d18e63206a4b5d710756d7e7d9c42ef0db0c5d58 - mk/lib: add a check that major and minor version is set for libs.

Sun, 10 May 2020 20:40:09 +0000

mk/lib: add a check that major and minor version is set for libs.Also, while we are here, consolidate setting SO_SUFFIX to one spot.Previously, it was possible for a library to slip throughwithout an SO version.Signed-off-by: Seth Howell Change-Id: I4db5fa5839502d266c6259892e5719b05134518cReviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2361Tested-by: SPDK CI Jenkins Community-CI: Mellanox Build BotCommunity-CI: Broadcom CIReviewed-by: Paul Luse Reviewed-by: Ben Walker Reviewed-by: Aleksey Marchuk Reviewed-by: Tomasz Zawadzki List of files: /spdk/module/blob/bdev/Makefile

9211c005e53b34e71f0daddc2a3bb837ae33814d - module/blob_bdev: add a map file.

Sun, 10 May 2020 18:28:09 +0000

module/blob_bdev: add a map file.Signed-off-by: Seth Howell Change-Id: Ibc4ccc29fc254e26685d9d0fb2e004f73cd3c0cdReviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2342Community-CI: Mellanox Build BotCommunity-CI: Broadcom CITested-by: SPDK CI Jenkins Reviewed-by: Tomasz Zawadzki Reviewed-by: Aleksey Marchuk List of files: /spdk/module/blob/bdev/Makefile

193927830ddced41dfebfe7cdb14d543aaf84a2c - make: rev SO versions individually for libraries.

Fri, 28 Feb 2020 20:47:11 +0000

make: rev SO versions individually for libraries.This will allow us to keep track of compatibility issues on aper-library basis.Change-Id: Ib0c796adb1efe1570212a503ed660bef6f142b6eSigned-off-by: Seth Howell Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1067Tested-by: SPDK CI Jenkins Reviewed-by: Jim Harris Reviewed-by: Ben Walker List of files: /spdk/module/blob/bdev/Makefile

7392cdeff7d8d8d30ec1d8f24a3eebe6f762cde4 - lib/blob: move bdev subdir under module directory.

Fri, 16 Aug 2019 18:04:17 +0000

lib/blob: move bdev subdir under module directory.Change-Id: Ifb9a1df919d32a98c328101029cc22e91915a977Signed-off-by: Seth Howell Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465457Reviewed-by: Changpeng Liu Reviewed-by: Broadcom SPDK FC-NVMe CI Reviewed-by: Jim Harris Reviewed-by: Ben Walker Tested-by: SPDK CI Jenkins List of files: /spdk/module/blob/bdev/Makefile