Changes in Makefile

2dc4a231ac65d10dd2e1a96684094bef1b7ebb95 - blob: Add support for variable metadata page size

Tue, 22 Oct 2024 21:20:10 +0000

blob: Add support for variable metadata page sizeCurrently SPDK blobstore depends on 4KB atomicity of metadata.So, Metadata page size will be made variable based on the physicalblock size reported by the underlying device. This will enableblobstore to store metadata in IU-sized chunks, not always 4KB.blobstore.c uses SPDK_BS_PAGE_SIZE (4KB) hardcoded at many places.To remove this make the metadata page size variable,phys_blocklen is added to spdk_bs_dev struct andmd_page_size is added to spdk_bs_super_block struct.Change-Id: I29d073eb4f4341a94a0675e70492b9186382f97fSigned-off-by: Atul Malakar Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/25130Community-CI: Broadcom CI Community-CI: Mellanox Build BotReviewed-by: Ben Walker Tested-by: SPDK CI Jenkins Community-CI: Community CI Samsung Reviewed-by: Konrad Sztyber Reviewed-by: Jim Harris List of files: /spdk/lib/blobfs/Makefile

355312bfcd3751f9af17fbefc90373fb8a269614 - so_ver: increase all major versions after SPDK 24.01

Fri, 26 Jan 2024 11:19:16 +0000

so_ver: increase all major versions after SPDK 24.01To allow SO_MINOR updates on LTS for the whole year it is supported,the major version for all components needs to be increased.This is to prevent scenario where two versions exists with matchingversions, but conflicting ABI.Ex. Next SPDK release adds an API call increasing the minor version,then LTS needs just a subset of those additions.Increasing major so version after LTS, allows the future releasesto update versions as needed. Yet allowing LTS to increase minorversion separately.This patch:- increases SO_VER by 1 for all components- resets SO_MINOR to 0 for all componentsDue to patch below being introduced after v24.01 code freeze,bdev lib version should not be increased:61623c5ca9 bdev: add spdk_bdev_io padding in place of io_submit_chShort reference to how the versions were changed:MAX=$(git grep "SO_VER := " | cut -d" " -f 3 | sort -ubnr | head -1)for((i=$MAX;i>0;i-=1)); do find . -name "Makefile" -exec \ sed -i -e "s/SO_VER := $i\$/SO_VER := $(($i+1))/g" {} +; donefind . -name "Makefile" -exec \ sed -i -e "s/SO_MINOR := .*/SO_MINOR := 0/g" {} +Signed-off-by: Tomasz Zawadzki Change-Id: I0166d6be6c17bbbf985d48b5dfcb36f1d4af1b48Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/21582Reviewed-by: Konrad Sztyber Tested-by: SPDK CI Jenkins List of files: /spdk/lib/blobfs/Makefile

3359bf34d682202107c227ea42ace1c600a04657 - so_ver: increase all major versions

Mon, 23 Jan 2023 17:24:28 +0000

so_ver: increase all major versionsTo allow SO_MINOR updates on LTS for the whole year it is supported,the major version for all components needs to be increased.This is to prevent scenario where two versions exists with matchingversions, but conflicting ABI.Ex. Next SPDK release adds an API call increasing the minor version,then LTS needs just a subset of those additions.Increasing major so version after LTS, allows the future releasesto update versions as needed. Yet allowing LTS to increase minorversion separately.Disabled test for increasing SO version without ABI change, asthat is goal of this patch. This check shall be removed with SPDK 23.05release.Looks like this was left over from prior LTS, to avoid thatmake sure it is only skipped when running against v23.01.x as latestrelease.This patch:- increases SO_VER by 1 for all components- resets SO_MINOR to 0 for all components- removes suppressions for ABI testsShort reference to how the versions were changed:MAX=$(git grep "SO_VER := " | cut -d" " -f 3 | sort -ubnr | head -1)for((i=$MAX;i>0;i-=1)); do find . -name "Makefile" -exec \ sed -i -e "s/SO_VER := $i\$/SO_VER := $(($i+1))/g" {} +; donefind . -name "Makefile" -exec \ sed -i -e "s/SO_MINOR := .*/SO_MINOR := 0/g" {} +Change-Id: I3e5681802c0a5ac6d7d652a18896997cd07cc8bfSigned-off-by: Tomasz Zawadzki Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/16419Reviewed-by: Ben Walker Reviewed-by: Konrad Sztyber Tested-by: SPDK CI Jenkins List of files: /spdk/lib/blobfs/Makefile

9e843fdbd1f39af220e39a0a31ce33c91c89b902 - blob: Add translate_lba operation

Thu, 15 Sep 2022 21:56:12 +0000

blob: Add translate_lba operationNew `translate_lba` operation allows to translate blob lba to lba onthe underlying bdev. It recurses down the whole chain of bs_dev's. Theoperation may fail to do the translation when blob lba is not backedby the real bdev. For example, when we eventually hit zeroes device inthe chain.This operation is used in the next commit to get source LBA for copyoperation.Signed-off-by: Evgeniy Kochetov Change-Id: I89c2d03d1982d66b9137a3a3653a98c361984fabReviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14528Tested-by: SPDK CI Jenkins Reviewed-by: Jim Harris Reviewed-by: Shuhei Matsumoto Reviewed-by: Aleksey Marchuk List of files: /spdk/lib/blobfs/Makefile

a6dbe3721eb3b5990707fc3e378c95e505dd8ab5 - update Intel copyright notices

Tue, 01 Nov 2022 20:26:26 +0000

update Intel copyright noticesper Intel policy to include file commit date using git cmdbelow. The policy does not apply to non-Intel (C) notices.git log --follow -C90% --format=%ad --date default | tail -1and then pull just the 4 digit year from the result.Intel copyrights were not added to files where Intel either hadno contribution ot the contribution lacked substance (ie licenseheader updates, formatting changes, etc). Contribution date used"--follow -C95%" to get the most accurate date.Note that several files in this patch didn't end the license/(c)block with a blank comment line so these were added as the vastmajority of files do have this last blank line. Simply there forconsistency.Signed-off-by: paul luse Change-Id: Id5b7ce4f658fe87132f14139ead58d6e285c04d4Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15192Tested-by: SPDK CI Jenkins Reviewed-by: Jim Harris Reviewed-by: Ben Walker Community-CI: Mellanox Build Bot List of files: /spdk/lib/blobfs/Makefile

2e7a7fe530576930ae3abd3fe93bcaa39850066e - blob: Optimize copy-on-write flow for clusters backed by zeroes device

Wed, 22 Jun 2022 13:55:46 +0000

blob: Optimize copy-on-write flow for clusters backed by zeroes deviceWriting to unallocated cluster triggers copy-on-write sequence. Ifthis cluster is backed by zeroes device we can skip the copy part. Fora simple thin provisioned volume copy this shortcut is alreadyimplemented because `blob->parent_id == SPDK_BLOBID_INVALID`. But thiswill not work for thin provisioned volumes created from snapshot. Inthis case we need to traverse the whole stack of underlying`spdk_bs_dev` devices for specific cluster to check if it is zeroesbacked.This patch adds `is_zeroes` operation to `spdk_bs_dev`. For zeroesdevice it always returns 'true', for real bdev (`blob_bs_dev`) alwaysreturns false, for another layer of `blob_bs_dev` does lba conversionand forwards to backing device.In blobstore's cluster copy flow we check if cluster is backed byzeroes device and skip copy part if it is.Signed-off-by: Evgeniy Kochetov Change-Id: I640773ac78f8f466b96e96a34c3a6c3c91f87dabReviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13446Tested-by: SPDK CI Jenkins Community-CI: Mellanox Build BotCommunity-CI: Broadcom CI Reviewed-by: Shuhei Matsumoto Reviewed-by: Aleksey Marchuk Reviewed-by: Jim Harris Reviewed-by: Tomasz Zawadzki List of files: /spdk/lib/blobfs/Makefile

488570ebd418ba07c9e69e65106dcc964f3bb41b - Replace most BSD 3-clause license text with SPDX identifier.

Fri, 03 Jun 2022 19:15:11 +0000

Replace most BSD 3-clause license text with SPDX identifier.Many open source projects have moved to using SPDX identifiersto specify license information, reducing the amount ofboilerplate code in every source file. This patch replacesthe bulk of SPDK .c, .cpp and Makefiles with the BSD-3-Clauseidentifier.Almost all of these files share the exact same license text,and this patch only modifies the files that contain themost common license text. There can be slight variationsbecause the third clause contains company names - most say"Intel Corporation", but there are instances for Nvidia,Samsung, Eideticom and even "the copyright holder".Used a bash script to automate replacement of the license textwith SPDX identifier which is checked into scripts/spdx.sh.Signed-off-by: Jim Harris Change-Id: Iaa88ab5e92ea471691dc298cfe41ebfb5d169780Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/12904Community-CI: Broadcom CI Community-CI: Mellanox Build BotTested-by: SPDK CI Jenkins Reviewed-by: Aleksey Marchuk Reviewed-by: Changpeng Liu Reviewed-by: Dong Yi Reviewed-by: Konrad Sztyber Reviewed-by: Paul Luse Reviewed-by: List of files: /spdk/lib/blobfs/Makefile

047c067c05ff22df06c053990998184b8a940514 - so_ver: increase all major versions

Mon, 31 Jan 2022 09:55:58 +0000

so_ver: increase all major versionsTo allow SO_MINOR updates on LTS for the whole year it is supported,the major version for all components needs to be increased.This is to prevent scenario where two versions exists with matchingversions, but conflicting ABI.Ex. Next SPDK release adds an API call increasing the minor version,then LTS needs just a subset of those additions.Increasing major so version after LTS, allows the future releasesto update versions as needed. Yet allowing LTS to increase minorversion separately.Disabled test for increasing SO version without ABI change, asthat is goal of this patch. This check shall be removed with SPDK 22.05release.This patch:- increases SO_VER by 1 for all components- resets SO_MINOR to 0 for all components- removes suppressions for ABI testsSigned-off-by: Tomasz Zawadzki Change-Id: Id1a5358882dc496faa5b0b5c9a63b326c378c551Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11361Tested-by: SPDK CI Jenkins Community-CI: Broadcom CI Reviewed-by: Jim Harris Reviewed-by: Ben Walker List of files: /spdk/lib/blobfs/Makefile

f01146ae48b78dc5d62db2e0d8050eeabf369e34 - blob: use uint64_t for unmap and write_zeroes lba count

Tue, 05 Oct 2021 23:37:19 +0000

blob: use uint64_t for unmap and write_zeroes lba countPrevious patches (5363eb3c) tried to work around the32-bit unmap and write_zeroes LBA counts by breakingup larger operations into smaller chunks of max sizeUINT32_MAX lba chunks.But some SSDs may just ignore unmap operations thatare not aligned to full physical block boundaries -and a UINT32_MAX lba unmap on a 512B logical /4KiB physical SSD would not be aligned. If the SSDdecided to ignore the unmap/deallocate (which it isallowed to do according to NVMe spec), we could endup with not unmapping *any* blocks. Probably SSDsshould always be trying hard to unmap as manyblocks as possible, but let's not try to depend onthat in blobstore.So one option would be to break them into chunksclose to UINT32_MAX which are still aligned to4KiB boundaries. But the better fix is to justchange the unmap and write_zeroes APIs to take64-bit arguments, and then we can avoid thechunking altogether.Fixes issue #2190.Signed-off-by: Jim Harris Change-Id: I23998e493a764d466927c3520c7a8c7f943000a6Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9737Community-CI: Broadcom CI Community-CI: Mellanox Build BotReviewed-by: Xiaodong Liu Reviewed-by: Changpeng Liu Reviewed-by: Aleksey Marchuk Reviewed-by: Dong Yi Reviewed-by: Ben Walker Reviewed-by: Tomasz Zawadzki Tested-by: SPDK CI Jenkins List of files: /spdk/lib/blobfs/Makefile

e4070ee0e012c345f4c059f125f4f9ce3be66690 - so_ver: increase all major versions

Fri, 29 Jan 2021 10:11:25 +0000

so_ver: increase all major versionsTo allow SO_MINOR updates on LTS for the whole year it is supported,the major version for all components needs to be increased.This is to prevent scenario where two versions exists with matchingversions, but conflicting ABI.Ex. Next SPDK release adds an API call increasing the minor version,then LTS needs just a subset of those additions.Increasing major so version after LTS, allows the quarterly releasesto update versions as needed. Yet allowing LTS to increase minorversion separately.Disabled test for increasing SO version without ABI change, asthat is goal of this patch. This check shall be removed with SPDK 21.04release.This patch:- increases SO_VER by 1 for all components- resets SO_MINOR to 0 for all components- removes suppressions for ABI testsSigned-off-by: Tomasz Zawadzki Change-Id: I44d01154430a074103bd21c7084f44932e81fe72Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6167Community-CI: Broadcom CICommunity-CI: Mellanox Build BotTested-by: SPDK CI Jenkins Reviewed-by: Ben Walker Reviewed-by: Jim Harris List of files: /spdk/lib/blobfs/Makefile

d18e63206a4b5d710756d7e7d9c42ef0db0c5d58 - mk/lib: add a check that major and minor version is set for libs.

Sun, 10 May 2020 20:40:09 +0000

mk/lib: add a check that major and minor version is set for libs.Also, while we are here, consolidate setting SO_SUFFIX to one spot.Previously, it was possible for a library to slip throughwithout an SO version.Signed-off-by: Seth Howell Change-Id: I4db5fa5839502d266c6259892e5719b05134518cReviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2361Tested-by: SPDK CI Jenkins Community-CI: Mellanox Build BotCommunity-CI: Broadcom CIReviewed-by: Paul Luse Reviewed-by: Ben Walker Reviewed-by: Aleksey Marchuk Reviewed-by: Tomasz Zawadzki List of files: /spdk/lib/blobfs/Makefile

6569a0ea0630b45bb83582ac2893cda584375dc4 - lib/blobfs: add a map file.

Tue, 07 Apr 2020 00:45:22 +0000

lib/blobfs: add a map file.Also, increment the library version since some non-public symbols have been removed.Please note, SPDK_LOG_BLOBFS is included in the map file because it is used by theblobfs_bdev module.Signed-off-by: Seth Howell Change-Id: Ib05c0b6630951d97b1a5b2931746471eeb9d9630Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1702Community-CI: Mellanox Build BotCommunity-CI: Broadcom CITested-by: SPDK CI Jenkins Reviewed-by: Shuhei Matsumoto Reviewed-by: Aleksey Marchuk Reviewed-by: Jim Harris List of files: /spdk/lib/blobfs/Makefile

193927830ddced41dfebfe7cdb14d543aaf84a2c - make: rev SO versions individually for libraries.

Fri, 28 Feb 2020 20:47:11 +0000

make: rev SO versions individually for libraries.This will allow us to keep track of compatibility issues on aper-library basis.Change-Id: Ib0c796adb1efe1570212a503ed660bef6f142b6eSigned-off-by: Seth Howell Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1067Tested-by: SPDK CI Jenkins Reviewed-by: Jim Harris Reviewed-by: Ben Walker List of files: /spdk/lib/blobfs/Makefile

1edd9bf3e467eb7a9591aee2216eccdfb8cb4dfa - blobfs: Add a lightweight filesystem built on the blobstore

Wed, 22 Mar 2017 20:35:00 +0000

blobfs: Add a lightweight filesystem built on the blobstoreThis is the initial commit for "blobfs", a lightweightfilesystem built on top of the SPDK blobstore.Also included in this patch:1) a shim for using SPDK bdevs as the backing store for SPDK blobstore/blobfs2) documentation for using blobfs as the storage engine with RocksDB3) scripts for running a set of workloads and collecting profiling data with RocksDB and blobfsSee doc/blobfs/getting_started.md included in this commitfor more details on blobfs, including some of the currentlimitations.Signed-off-by: Jim Harris Change-Id: I2a6d3d4b87236730051228ed62c0c04e04c42c73 List of files: /spdk/lib/blobfs/Makefile