lib/ftl: Enable chunk recovery for non-vss cache

Non-vss NV cache uses P2L IO log. To recover chunk it reads
entires from the long and rebuilds P2L map.

Change-Id: I88bea8779dbd9f93e2897d5381343925

lib/ftl: Enable chunk recovery for non-vss cache

Non-vss NV cache uses P2L IO log. To recover chunk it reads
entires from the long and rebuilds P2L map.

Change-Id: I88bea8779dbd9f93e2897d53813439250da8627c
Signed-off-by: Mariusz Barczak <Mariusz.Barczak@solidigmtechnology.com>
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@solidigm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19626
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot

show more ...

lib/ftl: Rename unmap to trim

Change the naming to be consistent within the lib

Change-Id: I51e6d0b983effc6d0f4c39abc8e5662aae54b541
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@solidigm.com

lib/ftl: Rename unmap to trim

Change the naming to be consistent within the lib

Change-Id: I51e6d0b983effc6d0f4c39abc8e5662aae54b541
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@solidigm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/23153
Reviewed-by: Ben Walker <ben@nvidia.com>
Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lib/ftl: Non-VSS running P2L

Remove the usage of VSS from P2L. Now each block containing P2L will
have a header containing the same data as VSS had previously. This
means that a single 1MiB write of

lib/ftl: Non-VSS running P2L

Remove the usage of VSS from P2L. Now each block containing P2L will
have a header containing the same data as VSS had previously. This
means that a single 1MiB write of user data will generate 2 blocks
worth of P2L (instead of just one). The header however will allow
for different write unit size requirements for the whole FTL - P2L
itself can now be arbitrary, but other FTL components demand an
overall power of 2 write unit size.

Change-Id: Ia4396f30fc874e0eaad10bcbf3acdd4296f39f22
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@solidigm.com>
Signed-off-by: Mariusz Barczak <Mariusz.Barczak@solidigmtechnology.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19605
Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lib/ftl: Add ftl_md_persist_entries function

Allows multiple metadata entries to be persisted at once, expanding the
previous ftl_md_persist_entry functionality which could only do it one
entry at a

lib/ftl: Add ftl_md_persist_entries function

Allows multiple metadata entries to be persisted at once, expanding the
previous ftl_md_persist_entry functionality which could only do it one
entry at a time.

Change-Id: I336b8becc700606ae45ad3bb16fb7cf7a023dccf
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@solidigm.com>
Signed-off-by: Mariusz Barczak <Mariusz.Barczak@solidigmtechnology.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19604
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>

show more ...

lib/ftl: New memory allocation mode in MD

This patch adds ability to allocate FTL metadata's buffer
on SPDK memory.

Change-Id: Ic95d940dcb98538cd57c2ef9283bcd32eb2aa442
Signed-off-by: Mateusz Kozlo

lib/ftl: New memory allocation mode in MD

This patch adds ability to allocate FTL metadata's buffer
on SPDK memory.

Change-Id: Ic95d940dcb98538cd57c2ef9283bcd32eb2aa442
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@solidigm.com>
Signed-off-by: Mariusz Barczak <Mariusz.Barczak@solidigmtechnology.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19603
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

lib/ftl: Simplify ftl_md mirror operations

Use already instantiated ftl::layout::md struct for ftl_md mirror
operations, i.e. persist, restore, clear.
With this change, the ftl_md::mirror member isn

lib/ftl: Simplify ftl_md mirror operations

Use already instantiated ftl::layout::md struct for ftl_md mirror
operations, i.e. persist, restore, clear.
With this change, the ftl_md::mirror member isn't needed anymore.

Change-Id: Ia9282911daa88b3ce30e53fa852f54d031683dde
Signed-off-by: Lukasz Lasek <lukasz.lasek@solidigm.com>
Signed-off-by: Mariusz Barczak <Mariusz.Barczak@solidigmtechnology.com>
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@solidigm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/19592
Reviewed-by: Jim Harris <jim.harris@samsung.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-by: Ben Walker <ben@nvidia.com>

show more ...

misc: Fix spelling mistakes

Found with misspell-fixer.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: If062df0189d92e4fb2da3f055fb981909780dc04
Reviewed-on: https://review.spdk.i

misc: Fix spelling mistakes

Found with misspell-fixer.

Signed-off-by: Michal Berger <michal.berger@intel.com>
Change-Id: If062df0189d92e4fb2da3f055fb981909780dc04
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15207
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

update Intel copyright notices

per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date defa

update Intel copyright notices

per Intel policy to include file commit date using git cmd
below. The policy does not apply to non-Intel (C) notices.

git log --follow -C90% --format=%ad --date default <file> | tail -1

and then pull just the 4 digit year from the result.

Intel copyrights were not added to files where Intel either had
no contribution ot the contribution lacked substance (ie license
header updates, formatting changes, etc). Contribution date used
"--follow -C95%" to get the most accurate date.

Note that several files in this patch didn't end the license/(c)
block with a blank comment line so these were added as the vast
majority of files do have this last blank line. Simply there for
consistency.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id5b7ce4f658fe87132f14139ead58d6e285c04d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/15192
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot

show more ...

ftl: p2l checkpointing

Since base device doesn't require VSS, FTL introduces a mechanism that
will allow for recovering both the P2L and write pointer of open bands
after a dirty shutdown. After wri

ftl: p2l checkpointing

Since base device doesn't require VSS, FTL introduces a mechanism that
will allow for recovering both the P2L and write pointer of open bands
after a dirty shutdown. After writing 1MiB of data to a band, a 4KiB
block describing the P2L will be persisted to cache device, effectively
emulating VSS for the base device.

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: Ic6be52dc09b237297a5cda3e752d6c038e98b70e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13367
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

ftl: sequence id tracking

Track the relative sequence of opening and closing bands and chunks.
Necessary for detecting the most recent user data during dirty shutdown recovery.

Signed-off-by: Artur

ftl: sequence id tracking

Track the relative sequence of opening and closing bands and chunks.
Necessary for detecting the most recent user data during dirty shutdown recovery.

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: I682030e58284d7b090667e4e5a9f4bbc7615708a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13366
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

ftl: fast shutdown

Adds API for fast shutdown - the ability for FTL to skip most
of the metadata persists made during clean shutdown, and relying
on their representation in shared memory instead. Th

ftl: fast shutdown

Adds API for fast shutdown - the ability for FTL to skip most
of the metadata persists made during clean shutdown, and relying
on their representation in shared memory instead. This allows for
faster update of SPDK (or just FTL, assuming no metadata changes),
with downtime reduction from 2-5 seconds to 500-1000 ms (for
14TiB+800GiB base and cache drives).

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: I5999d31698a81512db8d5893eabee7b505c80d06
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13348
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>

show more ...

ftl: Add helper functions for creating md regions

Helper functions which determine which md regions will be stored
in shm.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id:

ftl: Add helper functions for creating md regions

Helper functions which determine which md regions will be stored
in shm.

Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: I94cbfca66dfb56457a350874dbd1de63a2e07661
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/14159
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>

show more ...

ftl: support for metadata on shared memory

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: Ibc259f61f0ef2aead

ftl: support for metadata on shared memory

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: Ibc259f61f0ef2aeadb0e5ac7230969e29d77f184
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13340
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...

ftl: metadata utils and initialization

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: Iaa9d7dd3f9e3147f0acfe

ftl: metadata utils and initialization

Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Signed-off-by: Kozlowski Mateusz <mateusz.kozlowski@intel.com>
Change-Id: Iaa9d7dd3f9e3147f0acfe18e23506a33fe3fd5a3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/13293
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>

show more ...