Merge 0c4cba1010 into 8482cb11bc

Add storage_full_failsafe_threshold to the RBM-backed transaction
manager tests: a fresh store reports not-full at the default ratio,
real extent allocations flip is_storage_full() once the used fraction
crosses osd_failsafe_full_ratio, and restoring the default ratio
clears the condition, verifying the threshold tracks the config knob.

Signed-off-by: Shai Fultheim <shai.fultheim@gmail.com>

PendingReleaseNotes

@@ -269,6 +269,10 @@
   rather rigid and also challenging to disseminate in some environments.  The
   key can be embedded in the migration spec or just referenced from there while
   stored in the MON config-key store.
+* RBD: `Group::list_images` python API will now correctly raise an ObjectNotFound
+  error when invoked on a non-existent group. The C APIs `rbd_group_image_list`,
+  `rbd_snap_list`, `rbd_group_snap_list` and C++ API `group_image_list` will now
+  return ENOENT instead of masking the error and returning 0.
 * BlueStore: The experimental PMEM device backend has been removed, together
   with the ``pmem`` value of the ``bdev_type`` option and the DML/DSA offload
   path. The hardware it targeted, Intel Optane DC persistent memory, was

debian/ceph-common.install

@@ -10,7 +10,6 @@ usr/bin/ceph
 usr/bin/ceph-authtool
 usr/bin/ceph-conf
 usr/bin/ceph-dencoder
-usr/bin/ceph-diff-sorted
 usr/bin/ceph-rbdnamer
 usr/bin/ceph-syn
 usr/bin/cephfs-data-scan
@@ -37,7 +36,6 @@ usr/lib/ceph/crypto/* [amd64]
 usr/share/man/man8/ceph-authtool.8
 usr/share/man/man8/ceph-conf.8
 usr/share/man/man8/ceph-dencoder.8
-usr/share/man/man8/ceph-diff-sorted.8
 usr/share/man/man8/ceph-rbdnamer.8
 usr/share/man/man8/ceph-syn.8
 usr/share/man/man8/ceph-post-file.8
@@ -49,8 +47,6 @@ usr/share/man/man8/radosgw-admin.8
 usr/share/man/man8/rgw-policy-check.8
 usr/share/man/man8/rgw-policy-test.8
 usr/share/man/man8/rgw-gap-list.8
-usr/share/man/man8/rgw-orphan-list.8
-usr/share/man/man8/rgw-restore-bucket-index.8
 usr/share/man/man8/rbd.8
 usr/share/man/man8/rbdmap.8
 usr/share/man/man8/rbd-replay*.8

debian/radosgw.install

@@ -4,11 +4,7 @@ usr/bin/radosgw
 usr/bin/radosgw-es
 usr/bin/radosgw-object-expirer
 usr/bin/radosgw-token
-usr/bin/rgw-gap-list
-usr/bin/rgw-gap-list-comparator
-usr/bin/rgw-orphan-list
 usr/share/man/man8/ceph-diff-sorted.8
 usr/share/man/man8/radosgw.8
-usr/share/man/man8/rgw-gap-list.8
 usr/share/man/man8/rgw-orphan-list.8
 usr/share/man/man8/rgw-restore-bucket-index.8

doc/cephfs/cephfs-mirroring.rst

@@ -376,7 +376,7 @@ values until the new owning daemon writes an updated entry.
    * - ``metrics_updated_at``
      - Yes
      - No
-     - Wall-clock time of the last omap write; exposed only by
+     - Wall-clock time of the last omap write (ISO-8601 in mgr CLI); exposed only by
        ``ceph fs snapshot mirror status`` (not ``fs mirror peer status``)
 
 **Omap cleanup**
@@ -660,7 +660,7 @@ command parameter is of format ``filesystem-name@filesystem-id peer-uuid``::
                         "crawl_duration": "2s",
                         "datasync_queue_wait_duration": "1s",
                         "sync_duration": "33s",
-                        "sync_time_stamp": "274900.558797s",
+                        "sync_time_stamp": "2026-07-15T12:00:00.558797+0530",
                         "sync_bytes": "149.94 MiB",
                         "sync_files": 5000
                     },
@@ -750,11 +750,14 @@ Plain unsigned integers with no unit suffix.
 Timestamp
 ---------
 
-Field: ``sync_time_stamp``.
+Field: ``sync_time_stamp`` (and ``metrics_updated_at`` in mgr status).
 
-Monotonic clock time in seconds (since daemon startup) when the snapshot sync finished,
-printed with sub-second precision and an ``s`` suffix (for example, ``274900.558797s``). This
-is not a wall-clock or epoch timestamp.
+Wall-clock time printed as an ISO-8601 local timestamp with sub-second precision
+and timezone offset (for example, ``2026-07-15T12:00:00.558797+0530``).
+``sync_time_stamp`` is when the snapshot sync finished;
+``metrics_updated_at`` is when omap was last written for that directory and peer.
+Omap persistence and ``counter dump`` (``last_sync_timestamp``) still store Unix
+epoch seconds.
 
 ``snaps_synced``, ``snaps_deleted``, and ``snaps_renamed`` are
 :ref:`per-session counters<cephfs_mirroring_sync_metric_fields>`: they count
@@ -923,7 +926,7 @@ E.g., adding a regular file for synchronization would result in failed status::
                         "crawl_duration": "2s",
                         "datasync_queue_wait_duration": "1s",
                         "sync_duration": "44s",
-                        "sync_time_stamp": "500900.600797s",
+                        "sync_time_stamp": "2026-07-15T12:05:33.600797+0530",
                         "sync_bytes": "149.94 MiB",
                         "sync_files": 5000
                     },
@@ -972,7 +975,7 @@ In the remote filesystem::
                         "crawl_duration": "2s",
                         "datasync_queue_wait_duration": "1s",
                         "sync_duration": "33s",
-                        "sync_time_stamp": "274900.558797s",
+                        "sync_time_stamp": "2026-07-15T12:00:00.558797+0530",
                         "sync_bytes": "149.94 MiB",
                         "sync_files": 5000
                     },
@@ -1227,7 +1230,7 @@ Counters are not updated on every file read or write. Behavior differs by field
      - Durations in seconds; bytes are raw counts
    * - ``last_sync_timestamp``
      - ``last_synced_snap.sync_time_stamp``
-     - ``utime_t`` (seconds since epoch), not the monotonic string shown in admin JSON
+     - Perf counter is Unix epoch seconds; admin/mgr JSON show ISO-8601 local time
    * - ``snaps_synced`` / ``snaps_deleted`` / ``snaps_renamed``
      - Same field names at directory level
      - Reset on daemon restart or directory reassignment (same as admin socket)

doc/dev/cephfs-mirroring.rst

@@ -444,7 +444,7 @@ status. Commands of this kind take the form ``filesystem-name@filesystem-id peer
                         "crawl_duration": "2s",
                         "datasync_queue_wait_duration": "1s",
                         "sync_duration": "33s",
-                        "sync_time_stamp": "274900.558797s",
+                        "sync_time_stamp": "2026-07-15T12:00:00.558797+0530",
                         "sync_bytes": "149.94 MiB",
                         "sync_files": 5000
                     },
@@ -548,7 +548,7 @@ status:
                         "crawl_duration": "2s",
                         "datasync_queue_wait_duration": "1s",
                         "sync_duration": "33s",
-                        "sync_time_stamp": "274900.558797s",
+                        "sync_time_stamp": "2026-07-15T12:00:00.558797+0530",
                         "sync_bytes": "149.94 MiB",
                         "sync_files": 5000
                     },

doc/dev/crimson/index.rst

@@ -101,7 +101,7 @@ run::
   $ MDS=0 MON=1 OSD=1 MGR=1 taskset -ac '0-95' /ceph/src/vstart.sh --new -x \
   --localhost --without-dashboard --redirect-output --seastore --osd-args \
   "--seastore_max_concurrent_transactions=128 --seastore_cachepin_type=LRU \
-  --seastore_main_device_type=RANDOM_BLOCK_SSD" --seastore-devs  /dev/nvme0n1 \
+  --seastore_hot_device_type=RANDOM_BLOCK_SSD" --seastore-devs  /dev/nvme0n1 \
   --crimson  --crimson-smp 1 --no-restart
 
 Another SeaStore example::

doc/dev/crimson/seastore.rst

@@ -204,7 +204,7 @@ Metadata Structures
 Device Types
 ------------
 
-Configured via the ``seastore_main_device_type`` option, the device types are
+Configured via the ``seastore_hot_device_type`` option, the device types are
 separated into **Segmented** and **RBM** backend types as follows:
 
 

doc/dev/crimson/seastore_logical_bucket_cache.rst

@@ -0,0 +1,85 @@
+=====================================
+ The Logical Bucket Cache in SeaStore
+=====================================
+
+This document introduces the LogicalBucketCache machinery which is
+mainly for using SSDs as non-volatile cache for the relatively slow
+HDD devices. Specifically, it serves to:
+
+#. Load frequently accessed data from the cold tier to the Logical
+   Bucket Cache.
+#. Evict relatively cold data from the Logical Bucket Cache to the
+   cold tier based on the heat of both data read and write.
+
+The Logical Bucket Cache is enabled when there is at least one cold
+tier.
+
+General Approach
+================
+
+#. Cache line: a range of continuous laddr address space, which we
+   call logical bucket. For now, that continuous laddr range is the
+   scope of a RADOS object.
+#. Extents are loaded to the cache device when: 1) they are newly
+   written by clients and don't trigger the write_through machinery.
+   For now, the write_through machinery is triggered when the newly
+   written data is larger than a specific threshold; 2) they are on
+   the slow cold device and read to or evicted from the in-memory Cache.
+#. Extents are evicted from the cache device in units of logical buckets,
+   that is, extents of the same logical bucket are evicted from the cache
+   device altogether.
+#. The process that write extents evicted from Cache down to the cache
+   device is called promotion.
+#. The process that evict extents from the cache device to the cold device
+   is called demotion.
+#. Extents that are loaded from the cold tier to the Logical Bucket Cache
+   are NOT removed from the cold tier, which means they'd have two paddrs,
+   one in the Logical Bucket Cache, the other in the cold tier, and the
+   promoted data is stored twice. This is designed to make the demote process
+   cheap. We call paddrs corresponding to the cold tier shadow paddrs.
+   So lba mappings have two paddrs now, the primary paddr and the shadow
+   paddr.
+
+Promotion
+=========
+
+Promotion is the process that load the extents in the cold tier to the
+Logical Bucket Cache.
+
+Extents that are read from the cold tier are first added to the in-memory
+cache; when evicted from the in-memory cache, they are added to the Promoter.
+
+The Promoter write its extents down to the Logical Bucket Cache when the size
+of its extents grows larger than ``seastore_cache_promotion_size``.
+
+Demotion
+========
+
+Demotion happens when:
+
+#. The usage of the Logical Bucket Cache reaches ``seastore_multiple_tiers_fast_evict_ratio``.
+#. The in-memory data index of the Logical Bucket Cache reaches ``seastore_logical_bucket_capacity``.
+
+Demoting extents that are live in both the Logical Bucket Cache and the
+cold tier only involves setting the shadow paddr to the primary paddr and
+removing the shadow paddr itself.
+
+Demoting extents that are only live in the Logical Bucket Cache would involve
+rewriting the extents to the cold tier.
+
+WriteThrough
+============
+
+When the newly written data is larger than ``seastore_write_through_size``,
+it would be written directly to the cold tier.
+
+Internal Test Workload
+=======================
+
+To test the correctness of the whole Logical Bucket Cache machinery, we also
+implemented an internal stress test workload. When turned on, SeaStore would
+aggressively load the extents from the cold tier to the Logical Bucket Cache,
+and demote extents back to the cold tier. The newly written data may also be
+written directly to the cold tier based on ``seastore_test_workload_write_through_probability``.
+
+The internal test workload is turned on by ``crimson_test_workload``.

doc/dev/release-checklists.rst

@@ -41,8 +41,7 @@ Make sure X (and, ideally, X+1) is defined:
 Github Actions
 ~~~~~~~~~~~~~~
 
-- [ ] .github/workflows/redmine-upkeep.yml add release branch to pull_request_target trigger
-- [ ] .github/workflows/releng-audit.yml add release branch to pull_request_target trigger
+- [ ] `.github/workflows/*.yml' add release branch to pull_request_target trigger
 
 Scripts
 ~~~~~~~
@@ -137,6 +136,8 @@ After dev freeze
 - [ ] open the Branch Protection settings of the Ceph repo.  Duplicate settings to the new release branch.
 - [ ] create vX.3.0 annotated tag on ``main`` so upgrades from new release to main are not wrongly considered downgrades.
 - [ ] remove ``doc/releases/*.rst``. This should leave behind ``doc/releases/releases.yml`` which is used for doc building purposes. See also commit 33d63c3 ("doc: remove release notes for release branch") for details.
+- [ ] remove ``.github/workflows``. ``main`` is authoritative for workflows; avoid possible confusion/conflicts.
+- [ ] remove ``.github/pull_request_template.md``. Backports do not use it.
 - [ ] cherry-pick 8cf9ad62949516666ad0f2c0bb7726ef68e4d666 ("doc: add releases links to toc"). There will be trivial conflicts.
 - [ ] add redirect for new major release at `RTD <https://readthedocs.org/dashboard/ceph/redirects/>`_.
 - [ ] add release name to redmine (using https://tracker.ceph.com/custom_fields/16/edit)

doc/rados/bluestore/bluefs-spillover-cleaner.rst

@@ -80,21 +80,66 @@ Example output:
    {
     "Files Migrated": [
         ...
-        "db.wal/000052.log size=0x24cd21 migrated=0x1200000 from dev=2->1 ts=2026-06-02T12:44:25.488250+0000",
-        "db.wal/000053.log size=0x3696ad migrated=0x1200000 from dev=2->1 ts=2026-06-02T12:44:25.833014+0000",
-        "db.wal/000054.log size=0x1571f61 migrated=0x2400000 from dev=2->1 ts=2026-06-02T12:44:26.134245+0000",
-        "db.wal/000055.log size=0x1b2b508 migrated=0x2400000 from dev=2->1 ts=2026-06-02T12:44:26.550633+0000"
+        "db/000084.sst size=0xf8cd6 migrated=0x100000 from dev=2->1 ts=2026-07-22T18:49:42.555723+0000",
+        "db/000086.sst size=0xfdd34 migrated=0x100000 from dev=2->1 ts=2026-07-22T18:49:42.595349+0000",
+        "db.wal/000078.log size=0x19de2e2 migrated=0x2400000 from dev=2->1 ts=2026-07-22T18:49:42.694700+0000",
+        "db.wal/000079.log size=0x19d32f7 migrated=0x2400000 from dev=2->1 ts=2026-07-22T18:49:43.424059+0000",
+        "db.wal/000080.log size=0xd1a637 migrated=0x1200000 from dev=2->1 ts=2026-07-22T18:49:44.204353+0000",
+        "db.wal/000081.log size=0xcbeaf5 migrated=0x1200000 from dev=2->1 ts=2026-07-22T18:49:44.618457+0000",
+        "db.wal/000082.log size=0xd6737 migrated=0x1200000 from dev=2->1 ts=2026-07-22T18:49:44.892949+0000",
+        "db.wal/000083.log size=0x696fe8 migrated=0x1200000 from dev=2->1 ts=2026-07-22T18:49:45.279432+0000"
     ],
     "pending_files": [
-        "db.wal/000056.log",
-        "db.wal/000057.log",
-        "db.wal/000058.log",
-        "db.wal/000059.log",
-        "db.wal/000060.log",
+        "db.wal/000091.log",
+        "db.wal/000092.log",
+        "db.wal/000093.log",
+        "db.wal/000094.log",
+        "db.wal/000095.log",
         ...
-    ]
+    ],
+    "active_files": [
+        "db.wal/000085.log num_writers=1",
+        "db.wal/000087.log num_writers=1",
+        "db.wal/000088.log num_writers=1",
+        "db.wal/000089.log num_writers=1",
+        "db.wal/000090.log num_writers=1"
+    ],
+    "last_scan_time": "2026-07-22T18:51:15.895644+0000"
    }
 
+Interpreting Cleaner Stats
+==========================
+
+The spillover cleaner operates in repeated scan cycles of alternating Active and Idle phases.
+The output of ``ceph tell osd.N bluefs spillover cleaner stats`` shows the
+current cycle state and migration history of spillover cleaner.
+
+``Files Migrated``
+  Contains a list of entries for files that have been recently migrated from the slow device
+  to the DB device.
+
+  Example entry::
+
+     db/000084.sst size=0xf8cd6 migrated=0x100000 from dev=2->1 ts=2026-07-22T18:49:42.555723+0000
+
+``pending_files``
+  At the beginning of each cycle, the spillover cleaner scans for files on the slow device and adds them to ``pending_files``.
+  These files are later processed one by one.
+
+  - Successfully migrated files are removed from ``pending_files`` and added
+    to ``Files Migrated``.
+  - Files open for write are removed from ``pending_files`` and reported
+    in ``active_files`` for that cycle.
+
+  A new scan is performed at the beginning of the next cycle.
+
+``active_files``
+  Contains files that have active writers when the cleaner attempted to
+  migrate them. Migration of these files are skipped during the current cycle.
+
+``last_scan_time``
+  Shows the time of the most recent cleaner scan.
+
 Testing
 =======
 

qa/config/crimson_logical_bucket_cache.yaml

@@ -0,0 +1,5 @@
+overrides:
+  ceph:
+    conf:
+      osd:
+        seastore logical bucket cache test stress: true

qa/config/crimson_qa_overrides.yaml

@@ -14,6 +14,8 @@ overrides:
         debug ms: 20
         seastore cachepin size pershard: 64M
         seastore max concurrent transactions: 8
+      client:
+        debug_objecter: 20
     crimson_compat: true
   workunit:
     env:

qa/objectstore_crimson/seastore/rbm$/crimson_seastore_rbm.yaml

@@ -4,7 +4,10 @@ overrides:
       osd:
         # crimson's osd objectstore option
         osd objectstore: seastore
-        seastore main device type: RANDOM_BLOCK_SSD
+        seastore hot device type: RANDOM_BLOCK_SSD
+        seastore hot backend type: RANDOM_BLOCK
+        seastore cold device type: RANDOM_BLOCK_HDD
+        seastore cold backend type: RANDOM_BLOCK
         debug seastore: 20
         debug seastore onode: 20
         debug seastore odata: 20

qa/objectstore_crimson/seastore/rbm$/crimson_seastore_rbm_2q.yaml

@@ -5,7 +5,10 @@ overrides:
         # crimson's osd objectstore option
         osd objectstore: seastore
         seastore cachepin type: 2Q
-        seastore main device type: RANDOM_BLOCK_SSD
+        seastore hot device type: RANDOM_BLOCK_SSD
+        seastore hot backend type: RANDOM_BLOCK
+        seastore cold device type: RANDOM_BLOCK_HDD
+        seastore cold backend type: RANDOM_BLOCK
         debug seastore: 20
         debug seastore onode: 20
         debug seastore odata: 20

qa/objectstore_crimson/seastore/segmented$/crimson_seastore_segmented.yaml

@@ -4,6 +4,8 @@ overrides:
       osd:
         # crimson's osd objectstore option
         osd objectstore: seastore
+        seastore cold device type: RANDOM_BLOCK_HDD
+        seastore cold backend type: RANDOM_BLOCK
         debug seastore: 20
         debug seastore onode: 20
         debug seastore odata: 20

qa/objectstore_crimson/seastore/segmented$/crimson_seastore_segmented_2q.yaml

@@ -5,6 +5,8 @@ overrides:
         # crimson's osd objectstore option
         osd objectstore: seastore
         seastore cachepin type: 2Q
+        seastore cold device type: RANDOM_BLOCK_HDD
+        seastore cold backend type: RANDOM_BLOCK
         debug seastore: 20
         debug seastore onode: 20
         debug seastore odata: 20

qa/suites/crimson-rados/basic/deploy/ceph.yaml

@@ -11,3 +11,8 @@ tasks:
         mon osdmap full prune min: 15
         mon osdmap full prune interval: 2
         mon osdmap full prune txsize: 2
+overrides:
+  ceph:
+    conf:
+      osd:
+        seastore logical bucket cache test stress: true

qa/suites/crimson-rados/cephfs/crimson_logical_bucket_cache.yaml

@@ -0,0 +1 @@
+.qa/config/crimson_logical_bucket_cache.yaml

qa/suites/crimson-rados/objectstore_tool/crimson_logical_bucket_cache.yaml

@@ -0,0 +1 @@
+.qa/config/crimson_logical_bucket_cache.yaml

qa/suites/crimson-rados/osd_shards/crimson_logical_bucket_cache.yaml

@@ -0,0 +1 @@
+.qa/config/crimson_logical_bucket_cache.yaml

qa/suites/crimson-rados/rbd/crimson_logical_bucket_cache.yaml

@@ -0,0 +1 @@
+.qa/config/crimson_logical_bucket_cache.yaml

qa/suites/crimson-rados/rgw/crimson_logical_bucket_cache.yaml

@@ -0,0 +1 @@
+.qa/config/crimson_logical_bucket_cache.yaml

qa/suites/crimson-rados/thrash/crimson_logical_bucket_cache.yaml

@@ -0,0 +1 @@
+.qa/config/crimson_logical_bucket_cache.yaml

qa/tasks/ceph_objectstore_tool.py

@@ -209,8 +209,8 @@ def task(ctx, config):
     manager.raw_cluster_cmd('osd', 'set', 'nodown')
 
     if CRIMSON:
-        CRIMSON_DEVICE_TYPE = manager.get_config('osd', 0, 'seastore_main_device_type')
-        log.info('seastore_main_device_type is {}...'.format(CRIMSON_DEVICE_TYPE))
+        CRIMSON_DEVICE_TYPE = manager.get_config('osd', 0, 'seastore_hot_device_type')
+        log.info('seastore_hot_device_type is {}...'.format(CRIMSON_DEVICE_TYPE))
 
     PGNUM = config.get('pgnum', 12)
     log.info("pgnum: {num}".format(num=PGNUM))

qa/tasks/cephfs/test_mirroring.py

@@ -11,6 +11,7 @@ import functools
 
 from io import StringIO
 from collections import deque
+from datetime import datetime
 
 from tasks.cephfs.cephfs_test_case import CephFSTestCase
 from teuthology.exceptions import CommandFailedError
@@ -19,6 +20,16 @@ from teuthology.orchestra import run
 
 log = logging.getLogger(__name__)
 
+# ISO-8601 local time with offset, as dumped by peer_status / mgr status
+# (e.g. 2026-07-15T12:00:00.558797+0530)
+SYNC_TIME_STAMP_RE = re.compile(
+    r'^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{6}[+-]\d{4}$')
+
+
+def parse_sync_time_stamp(ts):
+    """Parse sync_time_stamp / metrics_updated_at ISO-8601 display strings."""
+    return datetime.strptime(ts, '%Y-%m-%dT%H:%M:%S.%f%z')
+
 
 # Exceptions to retry in test assertions
 RETRY_EXCEPTIONS = (AssertionError, KeyError, IndexError, CommandFailedError)
@@ -410,6 +421,7 @@ class TestMirroring(CephFSTestCase):
         self.assertRegex(
             last_synced_snap['sync_bytes'],
             r'^\d+(\.\d+)?\s+(B|KiB|MiB|GiB|TiB|PiB)$')
+        self.assertRegex(last_synced_snap['sync_time_stamp'], SYNC_TIME_STAMP_RE)
         self.assertIsInstance(last_synced_snap['sync_files'], int)
         self.assertGreaterEqual(last_synced_snap['sync_files'], 0)
 
@@ -2765,12 +2777,12 @@ class TestMirroring(CephFSTestCase):
                                          'fs', 'mirror', 'peer', 'status',
                                          f'{self.primary_fs_name}@{self.primary_fs_id}',
                                          peer_uuid)
-        d0_sync_time_stamp = float(self.peer_dir_status(res, '/d0', peer_uuid)
-                                  ['last_synced_snap']['sync_time_stamp'].rstrip('s'))
-        d1_sync_time_stamp = float(self.peer_dir_status(res, '/d1', peer_uuid)
-                                  ['last_synced_snap']['sync_time_stamp'].rstrip('s'))
-        d2_sync_time_stamp = float(self.peer_dir_status(res, '/d2', peer_uuid)
-                                  ['last_synced_snap']['sync_time_stamp'].rstrip('s'))
+        d0_sync_time_stamp = parse_sync_time_stamp(
+            self.peer_dir_status(res, '/d0', peer_uuid)['last_synced_snap']['sync_time_stamp'])
+        d1_sync_time_stamp = parse_sync_time_stamp(
+            self.peer_dir_status(res, '/d1', peer_uuid)['last_synced_snap']['sync_time_stamp'])
+        d2_sync_time_stamp = parse_sync_time_stamp(
+            self.peer_dir_status(res, '/d2', peer_uuid)['last_synced_snap']['sync_time_stamp'])
 
         self.assertGreaterEqual(d1_sync_time_stamp, d0_sync_time_stamp)
         self.assertGreaterEqual(d2_sync_time_stamp, d0_sync_time_stamp)
@@ -2837,12 +2849,12 @@ class TestMirroring(CephFSTestCase):
                                          'fs', 'mirror', 'peer', 'status',
                                          f'{self.primary_fs_name}@{self.primary_fs_id}',
                                          peer_uuid)
-        d0_sync_time_stamp = float(self.peer_dir_status(res, '/d0', peer_uuid)
-                                  ['last_synced_snap']['sync_time_stamp'].rstrip('s'))
-        d1_sync_time_stamp = float(self.peer_dir_status(res, '/d1', peer_uuid)
-                                  ['last_synced_snap']['sync_time_stamp'].rstrip('s'))
-        d2_sync_time_stamp = float(self.peer_dir_status(res, '/d2', peer_uuid)
-                                  ['last_synced_snap']['sync_time_stamp'].rstrip('s'))
+        d0_sync_time_stamp = parse_sync_time_stamp(
+            self.peer_dir_status(res, '/d0', peer_uuid)['last_synced_snap']['sync_time_stamp'])
+        d1_sync_time_stamp = parse_sync_time_stamp(
+            self.peer_dir_status(res, '/d1', peer_uuid)['last_synced_snap']['sync_time_stamp'])
+        d2_sync_time_stamp = parse_sync_time_stamp(
+            self.peer_dir_status(res, '/d2', peer_uuid)['last_synced_snap']['sync_time_stamp'])
 
         self.assertLess(d1_sync_time_stamp, d0_sync_time_stamp)
         self.assertLess(d2_sync_time_stamp, d0_sync_time_stamp)

src/cephadm/cephadmlib/daemons/smb.py

@@ -327,6 +327,12 @@ class SambaContainerCommon(ContainerCommon):
         # variable. This can be dropped once sambacc no longer needs it,
         # possibly after the next sambacc release.
         environ['SAMBACC_CTDB'] = 'ctdb-is-experimental'
+        environ[
+            'SAMBACC_PASSWD_LOCATION'
+        ] = '/etc/passwd:/var/lib/samba/passwd:/lib/passwd'
+        environ[
+            'SAMBACC_GROUP_LOCATION'
+        ] = '/etc/group:/var/lib/samba/group:/lib/group'
         if self.cfg.ceph_config_entity:
             environ['SAMBACC_CEPH_ID'] = f'name={self.cfg.ceph_config_entity}'
         if self.cfg.rank >= 0:
@@ -368,8 +374,11 @@ class SMBDContainer(SambaContainerCommon):
         args = super().args()
         args.append('run')
         if self.cfg.clustered:
-            auth_kind = 'nsswitch' if self.cfg.domain_member else 'users'
-            args.append(f'--setup={auth_kind}')
+            if self.cfg.domain_member:
+                args.append('--setup=nsswitch')
+            else:
+                args.append('--setup=nsswitch_auto')
+                args.append('--setup=users')
             args.append('--setup=smb_ctdb')
             args.append('--wait-for=ctdb')
         args.append('smbd')
@@ -445,11 +454,17 @@ class ConfigWatchContainer(SambaContainerCommon):
         return 'configwatch'
 
     def args(self) -> List[str]:
-        return super().args() + [
-            'update-config',
-            '--watch',
-            f'--signal-pids-dir={_WANT_SIGNAL_DIR}',
-        ]
+        args = super().args()
+        args.append('run')
+        if self.cfg.clustered:
+            if self.cfg.domain_member:
+                args.append('--setup=nsswitch')
+            else:
+                args.append('--setup=nsswitch_auto')
+            args.append('--wait-for=ctdb')
+        args.append(f'--config-watch-signal-pids-dir={_WANT_SIGNAL_DIR}')
+        args.append('configwatch')
+        return args
 
 
 class SMBMetricsContainer(ContainerCommon):

src/common/hobject.h

@@ -60,6 +60,12 @@ public:
   static bool is_meta_pool(int64_t pool) {
     return pool == POOL_META;
   }
+  static int64_t get_logical_pool(int64_t pool) {
+    if (is_temp_pool(pool))
+      return get_temp_pool(pool);  // it's reversible
+    else
+      return pool;
+  }
 
 public:
   object_t oid;
@@ -114,10 +120,7 @@ public:
     return is_meta_pool(pool);
   }
   int64_t get_logical_pool() const {
-    if (is_temp_pool(pool))
-      return get_temp_pool(pool);  // it's reversible
-    else
-      return pool;
+    return get_logical_pool(pool);
   }
 
   hobject_t() : snap(0), hash(0), max(false), pool(INT64_MIN) {

src/common/options/crimson.yaml.in

@@ -154,6 +154,13 @@ options:
   desc: CPU cores on which POSIX threads alienized to seastar will run in cpuset(7) format
   flags:
   - startup
+- name: seastore_logical_bucket_cache_test_stress
+  type: bool
+  level: dev
+  desc: Enable SeaStore Logical Bucket Cache stress mode. When enabled, SeaStore exercises cold-tier
+        cache paths by forcing promotion/demotion activity and probabilistic write-through behavior.
+        Intended for testing only.
+  default: false
 
 # Seastore options
 
@@ -245,15 +252,46 @@ options:
   level: advanced
   desc: maximum concurrent transactions that seastore allows (per reactor)
   default: 128
-- name: seastore_main_device_type
+- name: seastore_hot_device_type
   type: str
   level: dev
-  desc: The main device type seastore uses (SSD or RANDOM_BLOCK_SSD)
+  desc: The type of the devices in the hot (or only) tier, valid values are HDD, SSD, RANDOM_BLOCK_SSD and RANDOM_BLOCK_HDD
   default: SSD
+- name: seastore_hot_backend_type
+  type: str
+  level: dev
+  desc: The back end used by the devices in the hot (or only) tier, valid values are SEGMENTED and RANDOM_BLOCK
+  default: SEGMENTED
+- name: seastore_hot_backend_bw_throttle
+  type: size
+  level: advanced
+  desc: Size in bytes per second written to the hot devices, 0 indicates no limit
+  default: 0
+- name: seastore_cold_devices_count
+  type: uint
+  level: dev
+  desc: create cold devices during mkfs
+  default: 0
+- name: seastore_cold_device_type
+  type: str
+  level: dev
+  desc: The cold device type SeaStore uses, valid values are RANDOM_BLOCK_SSD, RANDOM_BLOCK_HDD, SSD and HDD.
+        This is only effective when there are two tiers, a hot one and a cold one.
+  default: RANDOM_BLOCK_HDD
+- name: seastore_cold_backend_type
+  type: str
+  level: dev
+  desc: The backend used by cold devices (SEGMENTED or RANDOM_BLOCK)
+  default: RANDOM_BLOCK
+- name: seastore_cold_backend_bw_throttle
+  type: size
+  level: advanced
+  desc: Size in bytes per second written to cold devices, 0 indicates no limit
+  default: 100_M
 - name: seastore_cbjournal_size
   type: size
   level: dev
-  desc: Total size to use for CircularBoundedJournal if created, it is valid only if seastore_main_device_type is RANDOM_BLOCK
+  desc: Total size to use for CircularBoundedJournal if created, it is valid only if seastore_hot_device_type is RANDOM_BLOCK
   default: 5_G
 - name: seastore_multiple_tiers_stop_evict_ratio
   type: float
@@ -342,3 +380,38 @@ options:
   desc: disable osd shards changes upon restart.
   flags:
   - startup
+- name: seastore_cache_promotion_size
+  type: size
+  level: advanced
+  desc: Size in bytes of extents to be promoted from cold devices, set 0 to disable promotion
+  default: 2_M
+- name: seastore_logical_bucket_capacity
+  type: size
+  level: advanced
+  desc: Size in bytes of memory used by logical bucket
+  default: 200_M
+- name: seastore_logical_bucket_proceed_size_per_cycle
+  type: size
+  level: advanced
+  desc: Size in bytes of extents to be demoted from logical bucket
+  default: 2_M
+- name: seastore_write_through_size
+  type: size
+  level: dev
+  desc: Select write through policy when data length is greater than this value.
+  default: 512_K
+- name: seastore_test_workload_write_through_probability
+  type: float
+  level: dev
+  desc: The percentage of writes that are applied directly to slow devices
+  default: 0.5
+- name: seastore_test_workload_2Q_promote_probability
+  type: float
+  level: dev
+  desc: The probability of promoting the extents evicted from the warm_in queue to faster devices
+  default: 0.5
+- name: seastore_test_workload_force_process_background_tasks_period
+  type: uint
+  level: dev
+  desc: Seconds of the period to force process background tasks
+  default: 5

src/crimson/os/futurized_store.h

@@ -52,6 +52,11 @@ public:
       return true;
     }
 
+    // Returns true when the local store is full (failsafe limit); checked at
+    // OSDOp boundary, where data-allocating ops are dropped with -EAGAIN so
+    // the client resends. Default false.
+    virtual bool is_storage_full() const { return false; }
+
     using CollectionRef = boost::intrusive_ptr<FuturizedCollection>;
     using base_errorator = crimson::errorator<crimson::ct_error::input_output_error>;
     using read_errorator = crimson::errorator<crimson::ct_error::enoent,

src/crimson/os/seastore/CMakeLists.txt

@@ -9,6 +9,7 @@ set(crimson_seastore_srcs
   transaction_manager.cc
   cache.cc
   extent_pinboard.cc
+  logical_bucket.cc
   root_block.cc
   lba_manager.cc
   async_cleaner.cc
@@ -45,6 +46,7 @@ set(crimson_seastore_srcs
   random_block_manager/block_rb_manager.cc
   random_block_manager/rbm_device.cc
   random_block_manager/nvme_block_device.cc
+  random_block_manager/hdd_device.cc
   random_block_manager/avlallocator.cc
   journal/segmented_journal.cc
   journal/segment_allocator.cc

src/crimson/os/seastore/async_cleaner.cc

@@ -603,10 +603,10 @@ std::size_t JournalTrimmerImpl::get_alloc_journal_size() const
   return static_cast<std::size_t>(ret);
 }
 
-seastar::future<> JournalTrimmerImpl::trim() {
+seastar::future<> JournalTrimmerImpl::trim(bool force) {
   return seastar::when_all(
-    [this] {
-      if (should_trim_alloc()) {
+    [this, force] {
+      if (force || should_trim_alloc()) {
         return trim_alloc(
         ).handle_error(
           crimson::ct_error::assert_all(
@@ -617,8 +617,8 @@ seastar::future<> JournalTrimmerImpl::trim() {
         return seastar::now();
       }
     },
-    [this] {
-      if (should_start_trim_dirty()) {
+    [this, force] {
+      if (force || should_start_trim_dirty()) {
         return trim_dirty(
         ).handle_error(
           crimson::ct_error::assert_all(
@@ -717,8 +717,11 @@ JournalTrimmerImpl::trim_dirty()
 	    return trans_intr::do_for_each(
 	      dirty_list,
 	      [this, &t](auto &e) {
-	      return extent_callback->rewrite_extent(
+              return extent_callback->maybe_remove_shadow(t, *e
+              ).si_then([this, &t, e] {
+	        return extent_callback->rewrite_extent(
 		  t, e, INIT_GENERATION, NULL_TIME);
+              });
 	    });
 	  });
 	}).si_then([this, &t] {
@@ -1388,15 +1391,11 @@ do_reclaim_space_ret do_reclaim_space(
 		    &reclaimed, &t, modify_time, target_generation] {
           DEBUGT("reclaim {} extents", t, extents.size());
           // rewrite live extents
-          return trans_intr::do_for_each(
-            extents,
-            [&extent_callback, modify_time, &t,
-	    &reclaimed, target_generation](auto ext)
-          {
-            reclaimed += ext->get_length();
-            return extent_callback.rewrite_extent(
-                t, ext, target_generation, modify_time);
-          });
+	  for (auto &ext : extents) {
+	    reclaimed += ext->get_length();
+	  }
+	  return extent_callback.rewrite_extents(
+	    t, extents, target_generation, PLACEMENT_HINT_NULL, modify_time);
         });
       }).si_then([&extent_callback, &t] {
         return extent_callback.submit_transaction_direct(t);
@@ -1962,9 +1961,11 @@ RBMCleaner::RBMCleaner(
   RBMDeviceGroupRef&& rb_group,
   BackrefManager &backref_manager,
   LBAManager &lba_manager,
-  bool detailed)
+  bool detailed,
+  bool is_cold)
   : store_index(store_index),
     detailed(detailed),
+    is_cold(is_cold),
     rb_group(std::move(rb_group)),
     backref_manager(backref_manager),
     lba_manager(lba_manager)
@@ -2194,7 +2195,13 @@ void RBMCleaner::register_metrics()
 {
   namespace sm = seastar::metrics;
 
-  metrics.add_group("rbm_cleaner", {
+  std::string prefix;
+  if (is_cold) {
+    prefix.append("cold_");
+  }
+  prefix.append("rbm_cleaner");
+
+  metrics.add_group(prefix, {
     sm::make_counter("total_bytes",
 		     [this] { return get_total_bytes(); },
 		     sm::description("the size of the space"),

src/crimson/os/seastore/async_cleaner.h

@@ -10,6 +10,7 @@
 
 #include "osd/osd_types.h"
 
+#include "crimson/common/config_proxy.h"
 #include "crimson/os/seastore/cached_extent.h"
 #include "crimson/os/seastore/seastore_types.h"
 #include "crimson/os/seastore/segment_manager.h"
@@ -351,6 +352,66 @@ public:
     rewrite_gen_t target_generation,
     sea_time_point modify_time) = 0;
 
+  using rewrite_extents_iertr = base_iertr;
+  using rewrite_extents_ret = rewrite_extents_iertr::future<>;
+  virtual rewrite_extents_ret rewrite_extents(
+    Transaction &t,
+    std::vector<CachedExtentRef> &extents,
+    rewrite_gen_t target_generation,
+    placement_hint_t hint,
+    sea_time_point modify_time) = 0;
+
+  /**
+   * promote_extent
+   *
+   * Promote an extent located from slow devices to the faster devices.
+   * When the type of extent is ObjectDataBlock, the original extent won't
+   * be retired, so that this extent is located in two differenct devices
+   * at the same time, which is helpful to reduce the cost of cleaner process.
+   * The others follow the normal rewrite process but its rewrite generation
+   * will be INIT_GENERATION(XXX: or the maximum generation of the hot tier?).
+   */
+  using promote_extent_iertr = base_iertr;
+  using promote_extent_ret = promote_extent_iertr::future<>;
+  virtual promote_extent_ret promote_extent(
+    Transaction &t,
+    CachedExtentRef extent) = 0;
+
+  virtual promote_extent_ret promote_extents_from_disk(
+    Transaction &t,
+    paddr_t paddr) = 0;
+
+  /**
+   * demote_region
+   *
+   * Demote the logical extents promoted from the slower device and evict
+   * the extents to the cold tier under the given laddr prefix.
+   */
+  struct demote_region_res_t {
+    std::size_t demoted_size = 0;
+    std::size_t evicted_size = 0;
+    bool complete = false;
+  };
+  using demote_region_iertr = base_iertr;
+  using demote_region_ret = demote_region_iertr::future<
+    demote_region_res_t>;
+  virtual demote_region_ret demote_region(
+    Transaction &t,
+    laddr_t prefix,
+    std::size_t max_proceed_size) = 0;
+
+  /**
+   * maybe_remove_shadow
+   *
+   * Remove the shadow of the extent if it exists
+   */
+  using maybe_remove_shadow_iertr = base_iertr;
+  using maybe_remove_shadow_ret =
+    maybe_remove_shadow_iertr::future<>;
+  virtual maybe_remove_shadow_ret maybe_remove_shadow(
+    Transaction &t,
+    CachedExtent &e) = 0;
+
   /**
    * get_extents_if_live
    *
@@ -418,6 +479,7 @@ struct BackgroundListener {
   virtual ~BackgroundListener() = default;
   virtual void maybe_wake_background() = 0;
   virtual void maybe_wake_blocked_io() = 0;
+  virtual void maybe_wake_promote() = 0;
   virtual state_t get_state() const = 0;
 
   bool is_ready() const {
@@ -622,7 +684,7 @@ public:
     reserved_usage -= usage;
   }
 
-  seastar::future<> trim();
+  seastar::future<> trim(bool force);
 
   static JournalTrimmerImplRef create(
       store_index_t store_index,
@@ -654,7 +716,7 @@ private:
   }
 
   bool can_drop_backref() const {
-    return get_backend_type() == backend_type_t::RANDOM_BLOCK;
+    return !tail_include_alloc;
   }
 
   bool should_trim_alloc() const {
@@ -1231,18 +1293,26 @@ public:
 
   virtual void release_projected_usage(std::size_t) = 0;
 
+  // Returns true when the allocator is full (failsafe limit) and writes should
+  // be refused. Only RBMCleaner overrides this; SegmentCleaner returns false.
+  virtual bool is_storage_full() const { return false; }
+
   virtual bool should_block_io_on_clean() const = 0;
 
   virtual bool can_clean_space() const = 0;
 
   virtual bool should_clean_space() const = 0;
 
+  virtual double get_alive_ratio() const = 0;
+
   using clean_space_ertr = base_ertr;
   using clean_space_ret = clean_space_ertr::future<>;
   virtual clean_space_ret clean_space() = 0;
 
   virtual const std::set<device_id_t>& get_device_ids() const = 0;
 
+  virtual backend_type_t get_backend_type() const = 0;
+
   virtual std::size_t get_reclaim_size_per_cycle() const = 0;
 
   // Periodic hook for adaptive threshold control. Default: no-op.
@@ -1467,6 +1537,10 @@ public:
     return sm_group->get_device_ids();
   }
 
+  backend_type_t get_backend_type() const final {
+    return backend_type_t::SEGMENTED;
+  }
+
   std::size_t get_reclaim_size_per_cycle() const final {
     return config.reclaim_bytes_per_cycle;
   }
@@ -1609,7 +1683,7 @@ private:
     if (segments.get_unavailable_bytes() == 0) return 0;
     return (double)get_unavailable_unused_bytes() / (double)segments.get_unavailable_bytes();
   }
-  double get_alive_ratio() const {
+  double get_alive_ratio() const final {
     return stats.used_bytes / (double)segments.get_total_bytes();
   }
 
@@ -1741,17 +1815,19 @@ public:
     RBMDeviceGroupRef&& rb_group,
     BackrefManager &backref_manager,
     LBAManager &lba_manager,
-    bool detailed);
+    bool detailed,
+    bool is_cold);
 
   static RBMCleanerRef create(
       store_index_t store_index,
       RBMDeviceGroupRef&& rb_group,
       BackrefManager &backref_manager,
       LBAManager &lba_manager,
-      bool detailed) {
+      bool detailed,
+      bool is_cold) {
     return std::make_unique<RBMCleaner>(
       store_index,
-      std::move(rb_group), backref_manager, lba_manager, detailed);
+      std::move(rb_group), backref_manager, lba_manager, detailed, is_cold);
   }
 
   RBMDeviceGroup* get_rb_group() {
@@ -1783,6 +1859,10 @@ public:
     return st;
   }
 
+  double get_alive_ratio() const final {
+    return stats.used_bytes / (double)get_total_bytes();
+  }
+
   void print(std::ostream &, bool is_detailed) const final;
 
   mount_ret mount() final;
@@ -1801,6 +1881,19 @@ public:
     return false;
   }
 
+  bool is_storage_full() const final {
+    auto st = get_stat();
+    if (st.total == 0) {
+      return false;
+    }
+    // Failsafe limit: full once the used fraction reaches the same ratio
+    // classic uses for its failsafe check (osd_failsafe_full_ratio, default
+    // 0.97), i.e. free fraction drops below (1 - ratio).
+    const double failsafe_full_ratio =
+      crimson::common::get_conf<double>("osd_failsafe_full_ratio");
+    return st.available < st.total * (1.0 - failsafe_full_ratio);
+  }
+
   bool can_clean_space() const final {
     return false;
   }
@@ -1815,6 +1908,10 @@ public:
     return rb_group->get_device_ids();
   }
 
+  backend_type_t get_backend_type() const final {
+    return backend_type_t::RANDOM_BLOCK;
+  }
+
   std::size_t get_reclaim_size_per_cycle() const final {
     return 0;
   }
@@ -1850,10 +1947,11 @@ public:
     return paddr;
   }
 
-  std::list<alloc_paddr_result> alloc_paddrs(extent_len_t length) {
+  std::list<alloc_paddr_result> alloc_paddrs(
+    extent_len_t length, paddr_t hint) {
     // TODO: implement allocation strategy (dirty metadata and multiple devices)
     auto rbs = rb_group->get_rb_managers();
-    auto ret = rbs[0]->alloc_extents(length);
+    auto ret = rbs[0]->alloc_extents(length, hint);
     if (!ret.empty()) {
       stats.used_bytes += length;
     }
@@ -1893,6 +1991,7 @@ private:
 
   store_index_t store_index;
   const bool detailed;
+  const bool is_cold;
   RBMDeviceGroupRef rb_group;
   BackrefManager &backref_manager;
   LBAManager &lba_manager;

src/crimson/os/seastore/backref/btree_backref_manager.cc

@@ -1,6 +1,7 @@
 // -*- mode:C++; tab-width:8; c-basic-offset:2; indent-tabs-mode:nil -*-
 // vim: ts=8 sw=2 sts=2 expandtab
 
+#include "crimson/common/coroutine.h"
 #include "crimson/os/seastore/backref/btree_backref_manager.h"
 
 SET_SUBSYS(seastore_backref);
@@ -71,16 +72,16 @@ BtreeBackrefManager::mkfs(
 {
   LOG_PREFIX(BtreeBackrefManager::mkfs);
   INFOT("start", t);
-  return cache.get_root(t).si_then([this, &t](auto croot) {
-    assert(croot->is_mutation_pending());
-    croot->get_root().backref_root = BackrefBtree::mkfs(croot, get_context(t));
-    return mkfs_iertr::now();
-  }).handle_error_interruptible(
-    mkfs_iertr::pass_further{},
-    crimson::ct_error::assert_all(
-      "Invalid error in BtreeBackrefManager::mkfs"
-    )
-  );
+
+  auto croot = co_await cache.get_root(t);
+  croot->get_root().backref_root =
+    co_await BackrefBtree::mkfs(croot, get_context(t)
+    ).handle_error_interruptible(
+      mkfs_iertr::pass_further{},
+      crimson::ct_error::assert_all(
+        "Invalid error in BtreeBackrefManager::mkfs"
+      )
+    );
 }
 
 BtreeBackrefManager::get_mapping_ret
@@ -456,6 +457,50 @@ BtreeBackrefManager::scan_mapped_space(
   });
 }
 
+BtreeBackrefManager::scan_device_ret
+BtreeBackrefManager::scan_device(
+  Transaction &t,
+  paddr_t paddr,
+  scan_device_func_t &f)
+{
+  LOG_PREFIX(BtreeBackrefManager::scan_device);
+  auto c = get_context(t);
+  auto croot = co_await cache.get_root(t);
+  auto btree = BackrefBtree(croot);
+  auto iter = co_await btree.lower_bound(c, paddr);
+  while (!iter.is_end()) {
+    auto key = iter.get_key();
+    auto bentry = cache.get_cached_backref_entry(key);
+    if (bentry) {
+      assert(bentry->paddr == key);
+      DEBUGT("found in cache: {} {}", t, bentry->paddr, bentry->laddr);
+    }
+    if (bentry && bentry->laddr == L_ADDR_NULL) {
+      DEBUGT("{} is removed", t, bentry->paddr);
+      iter = co_await iter.next(c);
+      continue;
+    }
+    if (key.get_device_id() == paddr.get_device_id()) {
+      auto val = iter.get_val();
+      if (bentry && bentry->laddr != val.laddr) {
+        DEBUGT("{} changed from {} to {}",
+          t, bentry->paddr, val.laddr, bentry->laddr);
+        iter = co_await iter.next(c);
+        continue;
+      }
+      DEBUGT("scanned {}, {}", t, key, val.laddr);
+      auto ret = co_await f(key, val.len, val.type, val.laddr);
+      if (ret == seastar::stop_iteration::yes) {
+	break;
+      }
+    } else if (key.get_device_id() > paddr.get_device_id()) {
+      break;
+    }
+    iter = co_await iter.next(c);
+  }
+  co_return;
+}
+
 base_iertr::future<> _init_cached_extent(
   op_context_t c,
   const CachedExtentRef &e,

src/crimson/os/seastore/backref/btree_backref_manager.h

@@ -55,6 +55,11 @@ public:
     Transaction &t,
     scan_mapped_space_func_t &&f) final;
 
+  scan_device_ret scan_device(
+    Transaction &t,
+    paddr_t paddr,
+    scan_device_func_t &f) final;
+
   init_cached_extent_ret init_cached_extent(
     Transaction &t,
     CachedExtentRef e) final;

src/crimson/os/seastore/backref_manager.h

@@ -139,6 +139,15 @@ public:
     Transaction &t,
     scan_mapped_space_func_t &&f) = 0;
 
+  using scan_device_ret = base_iertr::future<>;
+  using scan_device_func_t = std::function<
+    base_iertr::future<seastar::stop_iteration>(
+      paddr_t, extent_len_t, extent_types_t, laddr_t)>;
+  virtual scan_device_ret scan_device(
+    Transaction &t,
+    paddr_t start,
+    scan_device_func_t &f) = 0;
+
   virtual ~BackrefManager() {}
 };
 

src/crimson/os/seastore/btree/btree_types.cc

@@ -12,9 +12,13 @@ namespace lba {
 
 std::ostream& operator<<(std::ostream& out, const lba_map_val_t& v)
 {
-  return out << "lba_map_val_t("
-             << v.pladdr
-             << "~0x" << std::hex << v.len
+  out << "lba_map_val_t(";
+  if (v.shadow_paddr != P_ADDR_NULL) {
+    out << '[' << v.pladdr << ',' << v.shadow_paddr << ']';
+  } else {
+    out << v.pladdr;
+  }
+  return out << "~0x" << std::hex << v.len
              << ", type=" << (extent_types_t)v.type
              << ", checksum=0x" << v.checksum
              << ", refcount=" << std::dec << v.refcount

src/crimson/os/seastore/btree/btree_types.h

@@ -110,6 +110,7 @@ struct lba_map_val_t {
   extent_len_t len = 0;  ///< length of mapping
   pladdr_t pladdr;         ///< direct addr of mapping or
 			   //	laddr of a direct lba mapping(see btree_lba_manager.h)
+  paddr_t shadow_paddr;
   extent_ref_count_t refcount = 0; ///< refcount
   checksum_t checksum = 0; ///< checksum of original block written at paddr (TODO)
   extent_types_t type = extent_types_t::NONE;
@@ -118,11 +119,12 @@ struct lba_map_val_t {
   lba_map_val_t(
     extent_len_t len,
     pladdr_t pladdr,
+    paddr_t shadow_paddr,
     extent_ref_count_t refcount,
     checksum_t checksum,
     extent_types_t type)
-    : len(len), pladdr(pladdr), refcount(refcount),
-      checksum(checksum), type(type) {}
+    : len(len), pladdr(pladdr), shadow_paddr(shadow_paddr),
+      refcount(refcount), checksum(checksum), type(type) {}
   bool operator==(const lba_map_val_t&) const = default;
 };
 
@@ -136,6 +138,7 @@ std::ostream& operator<<(std::ostream& out, const lba_map_val_t&);
 struct __attribute__((packed)) lba_map_val_le_t {
   extent_len_le_t len = init_extent_len_le(0);
   pladdr_le_t pladdr;
+  paddr_le_t shadow_paddr;
   extent_ref_count_le_t refcount{0};
   checksum_le_t checksum{0};
   extent_types_le_t type{EXTENT_TYPES_MAX};
@@ -145,6 +148,7 @@ struct __attribute__((packed)) lba_map_val_le_t {
   explicit lba_map_val_le_t(const lba_map_val_t &val)
     : len(init_extent_len_le(val.len)),
       pladdr(pladdr_le_t(val.pladdr)),
+      shadow_paddr(paddr_le_t(val.shadow_paddr)),
       refcount(val.refcount),
       checksum(val.checksum),
       type(static_cast<extent_types_le_t>(val.type)) {}
@@ -153,6 +157,7 @@ struct __attribute__((packed)) lba_map_val_le_t {
     return lba_map_val_t{
       len,
       pladdr,
+      shadow_paddr,
       refcount,
       checksum,
       static_cast<extent_types_t>(type)};
@@ -293,3 +298,7 @@ std::ostream &operator<<(
 }
 
 } // namespace crimson::os::seastore
+
+#if FMT_VERSION >= 90000
+template <> struct fmt::formatter<crimson::os::seastore::lba::lba_map_val_t> : fmt::ostream_formatter {};
+#endif

src/crimson/os/seastore/btree/fixed_kv_btree.h

@@ -216,6 +216,21 @@ public:
 
     }
 
+    // return true if reached the next mapping and false otherwise
+    bool next_sync(op_context_t c) {
+#ifndef NDEBUG
+      assert_valid();
+#endif
+      assert(is_full());
+      assert(!is_end());
+      leaf.pos++;
+      if (at_boundary()) {
+        return handle_boundary_sync(c);
+      } else {
+        return true;
+      }
+    }
+
     /**
      * Move to the previous entry.  If already at position 0 in the current
      * leaf, walks up the internal stack (ensure_internal_bottom_up) to find
@@ -520,6 +535,50 @@ public:
       });
     }
 
+    // return true if reached the next mapping and false otherwise
+    bool handle_boundary_sync(op_context_t c) {
+      assert(at_boundary());
+      assert(is_full());
+      depth_t depth = 2;
+      // find the first level (from the leaf's parent up) with a right sibling
+      for (; depth <= get_depth(); depth++) {
+        auto &internal = get_internal(depth);
+        auto it = internal.node->iter_idx(internal.pos + 1);
+        if (it != internal.node->end()) {
+          break;
+        }
+      }
+      if (depth > get_depth()) {
+        return false; // no next leaf (end)
+      }
+      // step to the right sibling at that level, then descend leftmost to the leaf
+      get_internal(depth).pos++;
+      while (depth > 2) {
+        auto &internal = get_internal(depth);
+        auto it = get_internal(depth).node->iter_idx(internal.pos);
+        auto child = get_internal(depth).node->template get_child_sync<
+          internal_node_t>(c.trans, c.cache, internal.pos, it.get_key());
+        if (!is_valid_child_ptr(child.get())) {
+          return false;
+        }
+        depth--;
+        get_internal(depth).node = child;
+        get_internal(depth).pos = 0;
+      }
+      assert(depth == 2);
+      // position `leaf` at the next leaf
+      auto &parent = get_internal(depth);
+      auto it = parent.node->begin();
+      auto child = parent.node->template get_child_sync<
+        leaf_node_t>(c.trans, c.cache, parent.pos, it.get_key());
+      if (!is_valid_child_ptr(child.get())) {
+        return false;
+      }
+      leaf.node = child;
+      leaf.pos = 0;
+      return true;
+    }
+
     /**
      * Called when leaf.pos has advanced past the end of the current leaf
      * (at_boundary() == true).  Walks up the internal stack to find the
@@ -649,14 +708,25 @@ public:
    * leaf node as the root, links it to the RootBlock, and returns the
    * tree's root descriptor.
    */
-  using mkfs_ret = phy_tree_root_t;
+  using mkfs_iertr = base_iertr;
+  using mkfs_ret = mkfs_iertr::future<phy_tree_root_t>;
   static mkfs_ret mkfs(RootBlockRef &root_block, op_context_t c) {
     assert(root_block->is_mutation_pending());
-    auto root_leaf = c.cache.template alloc_new_non_data_extent<leaf_node_t>(
-      c.trans,
-      node_size,
-      placement_hint_t::HOT,
-      INIT_GENERATION);
+    LeafNodeRef root_leaf;
+    while (!root_leaf) {
+      try {
+        root_leaf = c.cache.template alloc_new_non_data_extent<leaf_node_t>(
+          c.trans, node_size, {placement_hint_t::HOT, INIT_GENERATION});
+      } catch (crimson::ct_error::eagain&) {
+      } catch (crimson::ct_error::enospc&) {
+        ceph_abort("shouldn't have no space");
+      }
+      if (!root_leaf) {
+        c.cache.get_epm().maybe_wake_background();
+        co_await trans_intr::make_interruptible(
+          c.cache.get_epm().wait_background());
+      }
+    }
     root_leaf->set_size(0);
     fixed_kv_node_meta_t<node_key_t> meta{min_max_t<node_key_t>::min, min_max_t<node_key_t>::max, 1};
     root_leaf->set_meta(meta);
@@ -664,7 +734,7 @@ public:
     get_tree_stats<self_type>(c.trans).depth = 1u;
     get_tree_stats<self_type>(c.trans).extents_num_delta++;
     TreeRootLinker<RootBlock, leaf_node_t>::link_root(root_block, root_leaf.get());
-    return phy_tree_root_t{root_leaf->get_paddr(), 1u};
+    co_return phy_tree_root_t{root_leaf->get_paddr(), 1u};
   }
 
   /**
@@ -752,10 +822,12 @@ public:
       if (depth > 1) {
         auto child = entry.node->template get_child_sync<internal_node_t>(
           c.trans, c.cache, entry.pos, riter.get_key());
+        ceph_assert(is_valid_child_ptr(child.get()));
         iter.get_internal(depth).node = child;
       } else {
         auto child = entry.node->template get_child_sync<leaf_node_t>(
           c.trans, c.cache, entry.pos, riter.get_key());
+        ceph_assert(is_valid_child_ptr(child.get()));
         iter.leaf.node = child;
       }
     }
@@ -1730,15 +1802,29 @@ public:
     LOG_PREFIX(FixedKVBtree::rewrite_extent);
     assert(is_lba_backref_node(e->get_type()));
 
-    auto do_rewrite = [&](auto &fixed_kv_extent) {
-      auto n_fixed_kv_extent = c.cache.template alloc_new_non_data_extent<
-        std::remove_reference_t<decltype(fixed_kv_extent)>
-        >(
-        c.trans,
-        fixed_kv_extent.get_length(),
+    auto do_rewrite = [&](auto &fixed_kv_extent) -> rewrite_extent_ret {
+      auto opt = Cache::alloc_option_t {
         fixed_kv_extent.get_user_hint(),
         // get target rewrite generation
-        fixed_kv_extent.get_rewrite_generation());
+        fixed_kv_extent.get_rewrite_generation()
+      };
+      TCachedExtentRef<std::remove_reference_t<
+        decltype(fixed_kv_extent)>> n_fixed_kv_extent;
+      while (!n_fixed_kv_extent) {
+        try {
+          n_fixed_kv_extent = c.cache.template alloc_new_non_data_extent<
+            std::remove_reference_t<decltype(fixed_kv_extent)>
+            >(
+            c.trans,
+            fixed_kv_extent.get_length(),
+            opt);
+        } catch (crimson::ct_error::eagain&) {}
+        if (!n_fixed_kv_extent) {
+          c.cache.get_epm().maybe_wake_background();
+          co_await trans_intr::make_interruptible(
+            c.cache.get_epm().wait_background());
+        }
+      }
       n_fixed_kv_extent->rewrite(c.trans, fixed_kv_extent, 0);
 
       SUBTRACET(
@@ -1747,26 +1833,24 @@ public:
         c.trans,
         fixed_kv_extent,
         *n_fixed_kv_extent);
-
-      return update_internal_mapping(
+      
+      co_await update_internal_mapping(
         c,
         n_fixed_kv_extent->get_node_meta().depth,
         n_fixed_kv_extent->get_node_meta().begin,
         e->get_paddr(),
         n_fixed_kv_extent->get_paddr(),
-        n_fixed_kv_extent
-      ).si_then([c, e] {
-        c.cache.retire_extent(c.trans, e);
-      });
+        n_fixed_kv_extent);
+      c.cache.retire_extent(c.trans, e);
     };
 
     if (e->get_type() == internal_node_t::TYPE) {
       auto lint = e->cast<internal_node_t>();
-      return do_rewrite(*lint);
+      co_await do_rewrite(*lint);
     } else {
       assert(e->get_type() == leaf_node_t::TYPE);
       auto lleaf = e->cast<leaf_node_t>();
-      return do_rewrite(*lleaf);
+      co_await do_rewrite(*lleaf);
     }
   }
 
@@ -2601,126 +2685,150 @@ private:
   {
     LOG_PREFIX(FixedKVBtree::handle_split);
 
-    return iter.check_split(c
-    ).si_then([FNAME, this, c, &iter](auto split_from) {
-      SUBTRACET(seastore_fixedkv_tree,
-        "split_from {}, depth {}", c.trans, split_from, iter.get_depth());
+    auto split_from = co_await iter.check_split(c);
+    SUBTRACET(seastore_fixedkv_tree,
+      "split_from {}, depth {}", c.trans, split_from, iter.get_depth());
 
-      if (split_from == iter.get_depth()) {
-        assert(iter.is_full());
-        auto nroot = c.cache.template alloc_new_non_data_extent<internal_node_t>(
-          c.trans, node_size, placement_hint_t::HOT, INIT_GENERATION);
-        fixed_kv_node_meta_t<node_key_t> meta{
-          min_max_t<node_key_t>::min, min_max_t<node_key_t>::max, iter.get_depth() + 1};
-        nroot->set_meta(meta);
-        nroot->range = meta;
-        nroot->journal_insert(
-          nroot->begin(),
-          min_max_t<node_key_t>::min,
-          get_root().get_location(),
-          nullptr);
-        iter.internal.push_back({nroot, 0});
+    if (split_from == iter.get_depth()) {
+      assert(iter.is_full());
+      InternalNodeRef nroot;
+      while (!nroot) {
+        try {
+          nroot = c.cache.template alloc_new_non_data_extent<internal_node_t>(
+            c.trans, node_size, {placement_hint_t::HOT, INIT_GENERATION});
+        } catch (crimson::ct_error::eagain&) {}
+        if (!nroot) {
+          c.cache.get_epm().maybe_wake_background();
+          co_await trans_intr::make_interruptible(
+            c.cache.get_epm().wait_background());
+        }
+      }
+      fixed_kv_node_meta_t<node_key_t> meta{
+        min_max_t<node_key_t>::min, min_max_t<node_key_t>::max, iter.get_depth() + 1};
+      nroot->set_meta(meta);
+      nroot->range = meta;
+      nroot->journal_insert(
+        nroot->begin(),
+        min_max_t<node_key_t>::min,
+        get_root().get_location(),
+        nullptr);
+      iter.internal.push_back({nroot, 0});
 
-        get_tree_stats<self_type>(c.trans).depth = iter.get_depth();
-        get_tree_stats<self_type>(c.trans).extents_num_delta++;
+      get_tree_stats<self_type>(c.trans).depth = iter.get_depth();
+      get_tree_stats<self_type>(c.trans).extents_num_delta++;
 
-        root_block = c.cache.duplicate_for_write(
-          c.trans, root_block)->template cast<RootBlock>();
-        get_root().set_location(nroot->get_paddr());
-        get_root().set_depth(iter.get_depth());
-        ceph_assert(get_root().get_depth() <= MAX_DEPTH);
-        set_root_node(nroot);
+      root_block = c.cache.duplicate_for_write(
+        c.trans, root_block)->template cast<RootBlock>();
+      get_root().set_location(nroot->get_paddr());
+      get_root().set_depth(iter.get_depth());
+      ceph_assert(get_root().get_depth() <= MAX_DEPTH);
+      set_root_node(nroot);
+    }
+
+    /* pos may be either node_position_t<leaf_node_t> or
+     * node_position_t<internal_node_t> */
+    auto split_level = [&, c, FNAME](auto &parent_pos, auto &pos)
+                        -> handle_split_iertr::future<std::pair<
+                          decltype(pos.node), decltype(pos.node)>> {
+      decltype(pos.node) left, right;
+      node_key_t pivot = min_max_t<node_key_t>::null;
+      while (!left) {
+        assert(!right);
+        assert(pivot == min_max_t<node_key_t>::null);
+        try {
+          auto t = pos.node->make_split_children(c);
+          left = std::get<0>(t);
+          right = std::get<1>(t);
+          pivot = std::get<2>(t);
+        } catch (crimson::ct_error::eagain&) {}
+        if (!left) {
+          c.cache.get_epm().maybe_wake_background();
+          co_await trans_intr::make_interruptible(
+            c.cache.get_epm().wait_background());
+        }
       }
 
-      /* pos may be either node_position_t<leaf_node_t> or
-       * node_position_t<internal_node_t> */
-      auto split_level = [&, c, FNAME](auto &parent_pos, auto &pos) {
-        auto [left, right, pivot] = pos.node->make_split_children(c);
+      auto parent_node = parent_pos.node;
+      auto parent_iter = parent_pos.get_iter();
 
-        auto parent_node = parent_pos.node;
-        auto parent_iter = parent_pos.get_iter();
+      parent_node->update(
+        parent_iter,
+        left->get_paddr(),
+        left.get());
+      parent_node->insert(
+        parent_iter + 1,
+        pivot,
+        right->get_paddr(),
+        right.get());
 
-        parent_node->update(
-          parent_iter,
-          left->get_paddr(),
-          left.get());
-        parent_node->insert(
-          parent_iter + 1,
-          pivot,
-          right->get_paddr(),
-          right.get());
+      SUBTRACET(
+        seastore_fixedkv_tree,
+        "splitted {} into left: {}, right: {}",
+        c.trans,
+        *pos.node,
+        *left,
+        *right);
+      c.cache.retire_extent(c.trans, pos.node);
 
+      get_tree_stats<self_type>(c.trans).extents_num_delta++;
+      co_return std::make_pair(left, right);
+    };
+
+    for (; split_from > 0; --split_from) {
+      auto &parent_pos = iter.get_internal(split_from + 1);
+      if (!parent_pos.node->is_mutable()) {
+        parent_pos.node = c.cache.duplicate_for_write(
+          c.trans, parent_pos.node
+        )->template cast<internal_node_t>();
+      }
+
+      if (split_from > 1) {
+        auto &pos = iter.get_internal(split_from);
         SUBTRACET(
           seastore_fixedkv_tree,
-          "splitted {} into left: {}, right: {}",
+          "splitting internal {} at depth {}, parent: {} at pos: {}",
           c.trans,
           *pos.node,
-          *left,
-          *right);
-        c.cache.retire_extent(c.trans, pos.node);
+          split_from,
+          *parent_pos.node,
+          parent_pos.pos);
+        auto [left, right] = co_await split_level(parent_pos, pos);
 
-        get_tree_stats<self_type>(c.trans).extents_num_delta++;
-        return std::make_pair(left, right);
-      };
-
-      for (; split_from > 0; --split_from) {
-        auto &parent_pos = iter.get_internal(split_from + 1);
-        if (!parent_pos.node->is_mutable()) {
-          parent_pos.node = c.cache.duplicate_for_write(
-            c.trans, parent_pos.node
-          )->template cast<internal_node_t>();
-        }
-
-        if (split_from > 1) {
-          auto &pos = iter.get_internal(split_from);
-          SUBTRACET(
-            seastore_fixedkv_tree,
-            "splitting internal {} at depth {}, parent: {} at pos: {}",
-            c.trans,
-            *pos.node,
-            split_from,
-            *parent_pos.node,
-            parent_pos.pos);
-          auto [left, right] = split_level(parent_pos, pos);
-
-          if (pos.pos < left->get_size()) {
-            pos.node = left;
-          } else {
-            pos.node = right;
-            pos.pos -= left->get_size();
-
-            parent_pos.pos += 1;
-          }
+        if (pos.pos < left->get_size()) {
+          pos.node = left;
         } else {
-          auto &pos = iter.leaf;
-          SUBTRACET(
-            seastore_fixedkv_tree,
-            "splitting leaf {}, parent: {} at pos: {}",
-            c.trans,
-            *pos.node,
-            *parent_pos.node,
-            parent_pos.pos);
-          auto [left, right] = split_level(parent_pos, pos);
+          pos.node = right;
+          pos.pos -= left->get_size();
 
-          /* right->get_node_meta().begin == pivot == right->begin()->get_key()
-           * Thus, if pos.pos == left->get_size(), we want iter to point to
-           * left with pos.pos at the end rather than right with pos.pos = 0
-           * since the insertion would be to the left of the first element
-           * of right and thus necessarily less than right->get_node_meta().begin.
-           */
-          if (pos.pos <= left->get_size()) {
-            pos.node = left;
-          } else {
-            pos.node = right;
-            pos.pos -= left->get_size();
+          parent_pos.pos += 1;
+        }
+      } else {
+        auto &pos = iter.leaf;
+        SUBTRACET(
+          seastore_fixedkv_tree,
+          "splitting leaf {}, parent: {} at pos: {}",
+          c.trans,
+          *pos.node,
+          *parent_pos.node,
+          parent_pos.pos);
+        auto [left, right] = co_await split_level(parent_pos, pos);
 
-            parent_pos.pos += 1;
-          }
+        /* right->get_node_meta().begin == pivot == right->begin()->get_key()
+         * Thus, if pos.pos == left->get_size(), we want iter to point to
+         * left with pos.pos at the end rather than right with pos.pos = 0
+         * since the insertion would be to the left of the first element
+         * of right and thus necessarily less than right->get_node_meta().begin.
+         */
+        if (pos.pos <= left->get_size()) {
+          pos.node = left;
+        } else {
+          pos.node = right;
+          pos.pos -= left->get_size();
+
+          parent_pos.pos += 1;
         }
       }
-
-      return seastar::now();
-    });
+    }
   }
 
 
@@ -2896,7 +3004,7 @@ private:
     
     SUBTRACET(seastore_fixedkv_tree, "parent: {}, node: {}", c.trans, *parent_pos.node, *pos.node);
     auto do_merge = [c, iter, donor_iter, donor_is_left, &parent_pos, &pos](
-                typename NodeType::Ref donor) {
+                typename NodeType::Ref donor) -> handle_merge_ret {
       LOG_PREFIX(FixedKVBtree::merge_level);
       auto [l, r] = donor_is_left ?
         std::make_pair(donor, pos.node) : std::make_pair(pos.node, donor);
@@ -2905,7 +3013,17 @@ private:
         std::make_pair(donor_iter, iter) : std::make_pair(iter, donor_iter);
 
       if (donor->at_min_capacity()) {
-        auto replacement = l->make_full_merge(c, r);
+        typename NodeType::Ref replacement;
+        while (!replacement) {
+          try {
+            replacement = l->make_full_merge(c, r);
+          } catch (crimson::ct_error::eagain&) {}
+          if (!replacement) {
+            c.cache.get_epm().maybe_wake_background();
+            co_await trans_intr::make_interruptible(
+              c.cache.get_epm().wait_background());
+          }
+        }
 
         parent_pos.node->update(
           liter,
@@ -2926,11 +3044,23 @@ private:
       } else {
         auto pivot_idx = l->get_balance_pivot_idx(*l, *r);
         LOG_PREFIX(FixedKVBtree::merge_level);
-        auto [replacement_l, replacement_r, pivot] =
-          l->make_balanced(
-            c,
-            r,
-            pivot_idx);
+        typename NodeType::Ref replacement_l, replacement_r;
+        node_key_t pivot = min_max_t<node_key_t>::null;
+        while (!replacement_l) {
+          assert(!replacement_r);
+          assert(pivot == min_max_t<node_key_t>::null);
+          try {
+            auto t = l->make_balanced(c, r, pivot_idx);
+            replacement_l = std::get<0>(t);
+            replacement_r = std::get<1>(t);
+            pivot = std::get<2>(t);
+          } catch (crimson::ct_error::eagain&) {}
+          if (!replacement_l) {
+            c.cache.get_epm().maybe_wake_background();
+            co_await trans_intr::make_interruptible(
+              c.cache.get_epm().wait_background());
+          }
+        }
 
         parent_pos.node->update(
           liter,
@@ -2966,8 +3096,6 @@ private:
         c.cache.retire_extent(c.trans, l);
         c.cache.retire_extent(c.trans, r);
       }
-
-      return seastar::now();
     };
 
     auto v = parent_pos.node->template get_child<NodeType>(
@@ -2975,30 +3103,26 @@ private:
     // checking the lba child must be atomic with creating
     // and linking the absent child
     if (v.has_child()) {
-      return std::move(v.get_child_fut()
-      ).si_then([do_merge=std::move(do_merge), &pos,
-                donor_iter, donor_is_left, c, parent_pos](auto child) {
-        LOG_PREFIX(FixedKVBtree::merge_level);
-        SUBTRACET(seastore_fixedkv_tree,
-          "got child on {}, pos: {}, res: {}",
-          c.trans,
-          *parent_pos.node,
-          donor_iter.get_offset(),
-          *child);
-        std::ignore = donor_is_left;
-        std::ignore = pos;
-        [[maybe_unused]] auto &node = (typename internal_node_t::base_t&)*child;
-        assert(donor_is_left ?
-          node.get_node_meta().end == pos.node->get_node_meta().begin :
-          node.get_node_meta().begin == pos.node->get_node_meta().end);
-        assert(node.get_node_meta().begin == donor_iter.get_key());
-        assert(node.get_node_meta().end > donor_iter.get_key());
-        return do_merge(child->template cast<NodeType>());
-      });
+      auto child = co_await std::move(v.get_child_fut());
+      SUBTRACET(seastore_fixedkv_tree,
+        "got child on {}, pos: {}, res: {}",
+        c.trans,
+        *parent_pos.node,
+        donor_iter.get_offset(),
+        *child);
+      std::ignore = donor_is_left;
+      std::ignore = pos;
+      [[maybe_unused]] auto &node = (typename internal_node_t::base_t&)*child;
+      assert(donor_is_left ?
+        node.get_node_meta().end == pos.node->get_node_meta().begin :
+        node.get_node_meta().begin == pos.node->get_node_meta().end);
+      assert(node.get_node_meta().begin == donor_iter.get_key());
+      assert(node.get_node_meta().end > donor_iter.get_key());
+      co_return co_await do_merge(child->template cast<NodeType>());
     }
 
     auto child_pos = v.get_child_pos();
-    return get_node<NodeType>(
+    auto donor = co_await get_node<NodeType>(
       c,
       depth,
       donor_iter.get_val().maybe_relative_to(parent_pos.node->get_paddr()),
@@ -3006,10 +3130,8 @@ private:
       end,
       std::make_optional<node_position_t<internal_node_t>>(
         child_pos.get_parent(),
-        child_pos.get_pos())
-    ).si_then([do_merge=std::move(do_merge)](typename NodeType::Ref donor) {
-      return do_merge(donor);
-    });
+        child_pos.get_pos()));
+    co_await do_merge(donor);
   }
 };
 

src/crimson/os/seastore/btree/fixed_kv_node.h

@@ -85,7 +85,7 @@ struct FixedKVNode : CachedExtent {
   void on_delta_write(paddr_t record_block_offset) final {
     // All in-memory relative addrs are necessarily record-relative
     assert(get_prior_instance());
-    assert(pending_for_transaction);
+    assert(t != nullptr);
     resolve_relative_addrs(record_block_offset);
   }
 
@@ -264,8 +264,9 @@ struct FixedKVInternalNode
     paddr_t addr,
     base_child_t* nextent) {
     LOG_PREFIX(FixedKVInternalNode::update);
+    assert(this->t != nullptr);
     SUBTRACE(seastore_fixedkv_tree, "trans.{}, pos {}, {}",
-      this->pending_for_transaction,
+      this->t->get_trans_id(),
       iter.get_offset(),
       (void*)nextent);
     this->update_child_ptr(iter.get_offset(), nextent);
@@ -281,8 +282,9 @@ struct FixedKVInternalNode
     paddr_t addr,
     base_child_t* nextent) {
     LOG_PREFIX(FixedKVInternalNode::insert);
+    assert(this->t != nullptr);
     SUBTRACE(seastore_fixedkv_tree, "trans.{}, pos {}, key {}, {}",
-      this->pending_for_transaction,
+      this->t->get_trans_id(),
       iter.get_offset(),
       pivot,
       (void*)nextent);
@@ -296,8 +298,9 @@ struct FixedKVInternalNode
 
   void remove(internal_const_iterator_t iter) {
     LOG_PREFIX(FixedKVInternalNode::remove);
+    assert(this->t != nullptr);
     SUBTRACE(seastore_fixedkv_tree, "trans.{}, pos {}, key {}",
-      this->pending_for_transaction,
+      this->t->get_trans_id(),
       iter.get_offset(),
       iter.get_key());
     this->remove_child_ptr(iter.get_offset());
@@ -312,8 +315,9 @@ struct FixedKVInternalNode
     paddr_t addr,
     base_child_t* nextent) {
     LOG_PREFIX(FixedKVInternalNode::replace);
+    assert(this->t != nullptr);
     SUBTRACE(seastore_fixedkv_tree, "trans.{}, pos {}, old key {}, key {}, {}",
-      this->pending_for_transaction,
+      this->t->get_trans_id(),
       iter.get_offset(),
       iter.get_key(),
       pivot,
@@ -329,9 +333,9 @@ struct FixedKVInternalNode
   std::tuple<Ref, Ref, NODE_KEY>
   make_split_children(op_context_t c) {
     auto left = c.cache.template alloc_new_non_data_extent<node_type_t>(
-      c.trans, node_size, placement_hint_t::HOT, INIT_GENERATION);
+      c.trans, node_size, {placement_hint_t::HOT, INIT_GENERATION});
     auto right = c.cache.template alloc_new_non_data_extent<node_type_t>(
-      c.trans, node_size, placement_hint_t::HOT, INIT_GENERATION);
+      c.trans, node_size, {placement_hint_t::HOT, INIT_GENERATION});
     this->split_child_ptrs(c.trans, *left, *right);
     auto pivot = this->split_into(*left, *right);
     left->range = left->get_meta();
@@ -347,7 +351,7 @@ struct FixedKVInternalNode
     op_context_t c,
     Ref &right) {
     auto replacement = c.cache.template alloc_new_non_data_extent<node_type_t>(
-      c.trans, node_size, placement_hint_t::HOT, INIT_GENERATION);
+      c.trans, node_size, {placement_hint_t::HOT, INIT_GENERATION});
     replacement->merge_child_ptrs(
       c.trans, static_cast<node_type_t&>(*this), *right);
     replacement->merge_from(*this, *right->template cast<node_type_t>());
@@ -365,9 +369,9 @@ struct FixedKVInternalNode
     ceph_assert(_right->get_type() == this->get_type());
     auto &right = *_right->template cast<node_type_t>();
     auto replacement_left = c.cache.template alloc_new_non_data_extent<node_type_t>(
-      c.trans, node_size, placement_hint_t::HOT, INIT_GENERATION);
+      c.trans, node_size, {placement_hint_t::HOT, INIT_GENERATION});
     auto replacement_right = c.cache.template alloc_new_non_data_extent<node_type_t>(
-      c.trans, node_size, placement_hint_t::HOT, INIT_GENERATION);
+      c.trans, node_size, {placement_hint_t::HOT, INIT_GENERATION});
 
     // We should do full merge if pivot_idx == right.get_size().
     ceph_assert(pivot_idx != right.get_size());
@@ -750,9 +754,9 @@ struct FixedKVLeafNode
   std::tuple<Ref, Ref, NODE_KEY>
   make_split_children(op_context_t c) {
     auto left = c.cache.template alloc_new_non_data_extent<node_type_t>(
-      c.trans, node_size, placement_hint_t::HOT, INIT_GENERATION);
+      c.trans, node_size, {placement_hint_t::HOT, INIT_GENERATION});
     auto right = c.cache.template alloc_new_non_data_extent<node_type_t>(
-      c.trans, node_size, placement_hint_t::HOT, INIT_GENERATION);
+      c.trans, node_size, {placement_hint_t::HOT, INIT_GENERATION});
     this->on_split(c.trans, *left, *right);
     auto pivot = this->split_into(*left, *right);
     left->range = left->get_meta();
@@ -777,7 +781,7 @@ struct FixedKVLeafNode
     op_context_t c,
     Ref &right) {
     auto replacement = c.cache.template alloc_new_non_data_extent<node_type_t>(
-      c.trans, node_size, placement_hint_t::HOT, INIT_GENERATION);
+      c.trans, node_size, {placement_hint_t::HOT, INIT_GENERATION});
     replacement->on_merge(c.trans, static_cast<node_type_t&>(*this), *right);
     replacement->merge_from(*this, *right->template cast<node_type_t>());
     replacement->range = replacement->get_meta();
@@ -809,9 +813,9 @@ struct FixedKVLeafNode
     ceph_assert(_right->get_type() == this->get_type());
     auto &right = *_right->template cast<node_type_t>();
     auto replacement_left = c.cache.template alloc_new_non_data_extent<node_type_t>(
-      c.trans, node_size, placement_hint_t::HOT, INIT_GENERATION);
+      c.trans, node_size, {placement_hint_t::HOT, INIT_GENERATION});
     auto replacement_right = c.cache.template alloc_new_non_data_extent<node_type_t>(
-      c.trans, node_size, placement_hint_t::HOT, INIT_GENERATION);
+      c.trans, node_size, {placement_hint_t::HOT, INIT_GENERATION});
 
     this->on_balance(
       c.trans,

src/crimson/os/seastore/cache.cc

@@ -36,9 +36,12 @@ Cache::Cache(
     delta_based_overwrite_enabled(
       crimson::common::get_conf<Option::size_t>(
         "seastore_data_delta_based_overwrite") > 0),
+    force_backref(crimson::common::get_conf<bool>(
+        "seastore_logical_bucket_cache_test_stress")),
     pinboard(create_extent_pinboard(
       crimson::common::get_conf<Option::size_t>(
-       "seastore_cachepin_size_pershard")))
+       "seastore_cachepin_size_pershard"),
+      &epm))
 {
   register_metrics(store_index);
   segment_providers_by_device_id.resize(DEVICE_ID_MAX, nullptr);
@@ -119,6 +122,8 @@ void Cache::register_metrics(store_index_t store_index)
   last_dirty_io_by_src_ext = {};
   last_trim_rewrites = {};
   last_reclaim_rewrites = {};
+  last_promote_rewrites = {};;
+  last_demote_rewrites = {};
   last_access = {};
   last_cache_absent_by_src = {};
   last_access_by_src_ext = {};
@@ -133,6 +138,8 @@ void Cache::register_metrics(store_index_t store_index)
     {src_t::TRIM_ALLOC, {sm::label_instance("src", "TRIM_ALLOC")}},
     {src_t::CLEANER_MAIN, {sm::label_instance("src", "CLEANER_MAIN")}},
     {src_t::CLEANER_COLD, {sm::label_instance("src", "CLEANER_COLD")}},
+    {src_t::PROMOTE, {sm::label_instance("src", "PROMOTE")}},
+    {src_t::DEMOTE, {sm::label_instance("src", "DEMOTE")}},
   };
   assert(labels_by_src.size() == (std::size_t)src_t::MAX);
 
@@ -228,6 +235,29 @@ void Cache::register_metrics(store_index_t store_index)
     }
   );
 
+  metrics.add_group("cache", {
+    sm::make_counter(
+      "write_hit_hot",
+      stats.write_hit_hot,
+      sm::description("the number of the lbc hits of data writes"),
+      {sm::label_instance("shard_store_index", std::to_string(store_index))}),
+    sm::make_counter(
+      "write_hit_cold",
+      stats.write_hit_cold,
+      sm::description("the number of the lbc misses of data writes"),
+      {sm::label_instance("shard_store_index", std::to_string(store_index))}),
+    sm::make_counter(
+      "read_hit_hot",
+      stats.read_hit_hot,
+      sm::description("the number of the lbc hits of data reads"),
+      {sm::label_instance("shard_store_index", std::to_string(store_index))}),
+    sm::make_counter(
+      "read_hit_cold",
+      stats.read_hit_cold,
+      sm::description("the number of the lbc misses of data reads"),
+      {sm::label_instance("shard_store_index", std::to_string(store_index))}),
+  });
+
   {
     /*
      * efforts discarded/committed
@@ -672,6 +702,10 @@ void Cache::register_metrics(store_index_t store_index)
            src2 == Transaction::src_t::CLEANER_MAIN) ||
           (src1 == Transaction::src_t::CLEANER_COLD &&
            src2 == Transaction::src_t::CLEANER_COLD) ||
+          (src1 == Transaction::src_t::PROMOTE &&
+           src2 == Transaction::src_t::PROMOTE) ||
+          (src1 == Transaction::src_t::DEMOTE &&
+           src2 == Transaction::src_t::DEMOTE) ||
           (src1 == Transaction::src_t::TRIM_ALLOC &&
            src2 == Transaction::src_t::TRIM_ALLOC)) {
         continue;
@@ -747,6 +781,34 @@ void Cache::register_metrics(store_index_t store_index)
         sm::description("sum of the version from rewrite-reclaim extents"),
         {sm::label_instance("shard_store_index", std::to_string(store_index))}
       ),
+      sm::make_counter(
+        "version_count_promote",
+        [this] {
+          return stats.promote_rewrites.get_num_rewrites();
+        },
+        sm::description("total number of rewrite-promote extents"),
+        {sm::label_instance("shard_store_index", std::to_string(store_index))}
+      ),
+      sm::make_counter(
+        "version_sum_promote",
+        stats.promote_rewrites.dirty_version,
+        sm::description("sum of the version from rewrite-promote extents"),
+        {sm::label_instance("shard_store_index", std::to_string(store_index))}
+      ),
+      sm::make_counter(
+        "version_count_demote",
+        [this] {
+          return stats.demote_rewrites.get_num_rewrites();
+        },
+        sm::description("total number of rewrite-demote extents"),
+        {sm::label_instance("shard_store_index", std::to_string(store_index))}
+      ),
+      sm::make_counter(
+        "version_sum_demote",
+        stats.demote_rewrites.dirty_version,
+        sm::description("sum of the version from rewrite-demote extents"),
+        {sm::label_instance("shard_store_index", std::to_string(store_index))}
+      ),
     }
   );
 }
@@ -1194,6 +1256,9 @@ CachedExtentRef Cache::alloc_remapped_extent_by_type(
   case extent_types_t::TEST_BLOCK:
     return alloc_remapped_extent<TestBlock>(
       t, remap_laddr, remap_paddr, remap_offset, remap_length, original_bptr);
+  case extent_types_t::LOG_NODE:
+    return alloc_remapped_extent<log_manager::LogNode>(
+      t, remap_laddr, remap_paddr, remap_offset, remap_length, original_bptr);
   default:
     ceph_abort("invalid extent type");
     return CachedExtentRef();
@@ -1205,42 +1270,45 @@ CachedExtentRef Cache::alloc_new_non_data_extent_by_type(
   extent_types_t type,   ///< [in] type tag
   extent_len_t length,   ///< [in] length
   placement_hint_t hint, ///< [in] user hint
-  rewrite_gen_t gen      ///< [in] rewrite generation
+  rewrite_gen_t gen,     ///< [in] rewrite generation
+  paddr_t paddr_hint,
+  bool is_tracked
 )
 {
   LOG_PREFIX(Cache::alloc_new_non_data_extent_by_type);
   SUBDEBUGT(seastore_cache, "allocate {} 0x{:x}B, hint={}, gen={}",
             t, type, length, hint, rewrite_gen_printer_t{gen});
   ceph_assert(get_extent_category(type) == data_category_t::METADATA);
+  auto opt = alloc_option_t{hint, gen, is_tracked, paddr_hint};
   switch (type) {
   case extent_types_t::ROOT:
     ceph_assert(0 == "ROOT is never directly alloc'd");
     return CachedExtentRef();
   case extent_types_t::LADDR_INTERNAL:
-    return alloc_new_non_data_extent<lba::LBAInternalNode>(t, length, hint, gen);
+    return alloc_new_non_data_extent<lba::LBAInternalNode>(t, length, opt);
   case extent_types_t::LADDR_LEAF:
     return alloc_new_non_data_extent<lba::LBALeafNode>(
-      t, length, hint, gen);
+      t, length, opt);
   case extent_types_t::ROOT_META:
     return alloc_new_non_data_extent<RootMetaBlock>(
-      t, length, hint, gen);
+      t, length, opt);
   case extent_types_t::ONODE_BLOCK_STAGED:
     return alloc_new_non_data_extent<onode::SeastoreNodeExtent>(
-      t, length, hint, gen);
+      t, length, opt);
   case extent_types_t::OMAP_INNER:
     return alloc_new_non_data_extent<omap_manager::OMapInnerNode>(
-      t, length, hint, gen);
+      t, length, opt);
   case extent_types_t::OMAP_LEAF:
     return alloc_new_non_data_extent<omap_manager::OMapLeafNode>(
-      t, length, hint, gen);
+      t, length, opt);
   case extent_types_t::COLL_BLOCK:
     return alloc_new_non_data_extent<collection_manager::CollectionNode>(
-      t, length, hint, gen);
+      t, length, opt);
   case extent_types_t::TEST_BLOCK_PHYSICAL:
-    return alloc_new_non_data_extent<TestBlockPhysical>(t, length, hint, gen);
+    return alloc_new_non_data_extent<TestBlockPhysical>(t, length, opt);
   case extent_types_t::LOG_NODE:
      return alloc_new_non_data_extent<log_manager::LogNode>(
-       t, length, hint, gen);
+       t, length, opt);
   case extent_types_t::NONE: {
     ceph_assert(0 == "NONE is an invalid extent type");
     return CachedExtentRef();
@@ -1255,27 +1323,26 @@ std::vector<CachedExtentRef> Cache::alloc_new_data_extents_by_type(
   Transaction &t,        ///< [in, out] current transaction
   extent_types_t type,   ///< [in] type tag
   extent_len_t length,   ///< [in] length
-  placement_hint_t hint, ///< [in] user hint
-  rewrite_gen_t gen      ///< [in] rewrite generation
+  alloc_option_t opt     ///< [in] allocation options
 )
 {
   LOG_PREFIX(Cache::alloc_new_data_extents_by_type);
   SUBDEBUGT(seastore_cache, "allocate {} 0x{:x}B, hint={}, gen={}",
-            t, type, length, hint, rewrite_gen_printer_t{gen});
+            t, type, length, opt.hint, rewrite_gen_printer_t{opt.gen});
   ceph_assert(get_extent_category(type) == data_category_t::DATA);
   std::vector<CachedExtentRef> res;
   switch (type) {
   case extent_types_t::OBJECT_DATA_BLOCK:
     {
       auto extents = alloc_new_data_extents<
-	ObjectDataBlock>(t, length, hint, gen);
+	ObjectDataBlock>(t, length, std::move(opt));
       res.insert(res.begin(), extents.begin(), extents.end());
     }
     return res;
   case extent_types_t::TEST_BLOCK:
     {
       auto extents = alloc_new_data_extents<
-	TestBlock>(t, length, hint, gen);
+	TestBlock>(t, length, std::move(opt));
       res.insert(res.begin(), extents.begin(), extents.end());
     }
     return res;
@@ -1322,7 +1389,7 @@ CachedExtentRef Cache::duplicate_for_write(
   }
 
   auto ret = i->duplicate_for_write(t);
-  ret->pending_for_transaction = t.get_trans_id();
+  ret->t = &t;
   ret->set_prior_instance(i);
   if (!is_root_type(ret->get_type())) {
     assert(ret->get_paddr().is_absolute());
@@ -1723,7 +1790,7 @@ record_t Cache::prepare_record(
     }
     assert(i->state == CachedExtent::extent_state_t::DIRTY);
     assert(i->version > 0);
-    assert(i->pending_for_transaction == TRANS_ID_NULL);
+    assert(i->t == nullptr);
     assert(!i->prior_instance);
     remove_from_dirty(i, &trans_src);
     // set the version to zero because the extent state is now clean
@@ -1753,10 +1820,10 @@ record_t Cache::prepare_record(
 
     if (i->is_exist_clean()) {
       assert(i->version == 0);
-      assert(!i->prior_instance);
+      assert(!i->prior_instance || t.get_src() == transaction_type_t::DEMOTE);
       // no set_io_wait(), skip complete_commit()
       assert(!i->is_pending_io());
-      i->pending_for_transaction = TRANS_ID_NULL;
+      i->t = nullptr;
       i->state = CachedExtent::extent_state_t::CLEAN;
     } else {
       assert(i->is_exist_mutation_pending());
@@ -1764,13 +1831,22 @@ record_t Cache::prepare_record(
                      should_use_no_conflict_publish(t, i->get_type()));
     }
 
-    // exist mutation pending extents must be in t.mutated_block_list
-    add_extent(i);
-    const auto t_src = t.get_src();
-    if (i->is_stable_dirty()) {
-      add_to_dirty(i, &t_src);
+    assert(i->is_logical());
+    if (t.get_src() == transaction_type_t::DEMOTE) {
+      assert(!i->committer);
+      assert(!i->get_prior_instance()->committer);
+      i->new_committer(t);
+      assert(i->committer);
+      i->get_prior_instance()->committer = i->committer;
     } else {
-      touch_extent_fully(*i, &t_src, t.get_cache_hint());
+      // exist mutation pending extents must be in t.mutated_block_list
+      add_extent(i);
+      const auto t_src = t.get_src();
+      if (i->is_stable_dirty()) {
+        add_to_dirty(i, &t_src);
+      } else {
+        touch_extent_fully(*i, &t_src, t.get_cache_hint());
+      }
     }
 
     alloc_delta.alloc_blk_ranges.emplace_back(
@@ -1945,6 +2021,10 @@ record_t Cache::prepare_record(
   } else if (trans_src == Transaction::src_t::CLEANER_MAIN ||
              trans_src == Transaction::src_t::CLEANER_COLD) {
     stats.reclaim_rewrites.add(rewrite_stats);
+  } else if (trans_src == Transaction::src_t::PROMOTE) {
+    stats.promote_rewrites.add(rewrite_stats);
+  } else if (trans_src == Transaction::src_t::DEMOTE) {
+    stats.demote_rewrites.add(rewrite_stats);
   } else {
     assert(rewrite_stats.is_clear());
   }
@@ -2022,7 +2102,7 @@ void Cache::complete_commit(
       assert(i->get_last_committed_crc() == CRC_NULL);
     }
 #endif
-    i->pending_for_transaction = TRANS_ID_NULL;
+    i->t = nullptr;
     i->on_initial_write();
     const auto t_src = t.get_src();
     if (should_use_no_conflict_publish(t, i->get_type())) {
@@ -2033,7 +2113,7 @@ void Cache::complete_commit(
       TRACET("committing rewritten extent into "
              "existing, inline={} -- {}, prior={}",
              t, is_inline, *i, prior);
-      prior.pending_for_transaction = TRANS_ID_NULL;
+      prior.t = nullptr;
       committer.commit_state();
       committer.sync_checksum();
       committer.commit_and_share_paddr();
@@ -2042,6 +2122,9 @@ void Cache::complete_commit(
       }
       if (i->is_logical()) {
         committer.maybe_sync_copied_lba_key();
+        if (t.get_src() == transaction_type_t::PROMOTE) {
+          committer.commit_shadow_promote(t);
+        }
       }
       touch_extent_fully(prior, &t_src, t.get_cache_hint());
       committer.sync_version();
@@ -2126,7 +2209,7 @@ void Cache::complete_commit(
       committer.commit_state();
       committer.sync_checksum();
       auto &prior = *i->prior_instance;
-      prior.pending_for_transaction = TRANS_ID_NULL;
+      prior.t = nullptr;
       ceph_assert(prior.is_valid());
       if (is_lba_backref_node(i->get_type())) {
         committer.commit_data();
@@ -2138,7 +2221,7 @@ void Cache::complete_commit(
       prior.committer.reset();
     }
 
-    i->pending_for_transaction = TRANS_ID_NULL;
+    i->t = nullptr;
     i->reset_prior_instance();
     assert(i->version > 0);
     i->complete_io();
@@ -2150,6 +2233,30 @@ void Cache::complete_commit(
       continue;
     }
     epm.mark_space_used(i->get_paddr(), i->get_length());
+    assert(i->is_logical());
+    auto t_src = t.get_src();
+    if (t.get_src() == transaction_type_t::DEMOTE) {
+      assert(i->committer);
+      auto &committer = *i->committer;
+      auto &prior = static_cast<LogicalChildNode&>(
+        *i->get_prior_instance());
+      ceph_assert(prior.is_valid());
+      TRACET("committing rewritten extent into "
+             "existing -- {}, prior={}",
+             t, *i, prior);
+      prior.t = nullptr;
+      if (auto shadow = prior.get_shadow(); shadow) {
+        committer.commit_shadow_demote(t);
+        prior.reset_shadow();
+      }
+      committer.commit_state();
+      committer.sync_checksum();
+      committer.commit_and_share_paddr();
+      touch_extent_fully(prior, &t_src, t.get_cache_hint());
+      committer.sync_version();
+      i->committer.reset();
+      prior.committer.reset();
+    }
   }
   for (auto &i: t.pre_alloc_list) {
     if (!i->is_valid()) {
@@ -2173,6 +2280,9 @@ void Cache::complete_commit(
         i->set_invalid(t);
     }
   }
+
+  stats.write_hit_hot += t.write_hit_hot;
+  stats.write_hit_cold += t.write_hit_cold;
 }
 
 void Cache::init()
@@ -2190,7 +2300,8 @@ void Cache::init()
              P_ADDR_ROOT,
              PLACEMENT_HINT_NULL,
              NULL_GENERATION,
-             TRANS_ID_NULL);
+             nullptr,
+	     write_policy_t::WRITE_BACK);
   root->set_modify_time(seastar::lowres_system_clock::now());
   INFO("init root -- {}", *root);
   add_extent(root);
@@ -2614,7 +2725,8 @@ Cache::_get_absent_extent_by_type(
 	    offset,
 	    PLACEMENT_HINT_NULL,
 	    NULL_GENERATION,
-	    TRANS_ID_NULL);
+	    nullptr,
+	    write_policy_t::WRITE_BACK);
   DEBUGT("{} length=0x{:x} is absent, add extent ... -- {}",
     t, type, length, *ret);
   add_extent(ret);
@@ -2837,6 +2949,10 @@ cache_stats_t Cache::get_stats(
     _trim_rewrites.minus(last_trim_rewrites);
     rewrite_stats_t _reclaim_rewrites = stats.reclaim_rewrites;
     _reclaim_rewrites.minus(last_reclaim_rewrites);
+    rewrite_stats_t _promote_rewrites = stats.promote_rewrites;
+    _promote_rewrites.minus(last_promote_rewrites);
+    rewrite_stats_t _demote_rewrites = stats.demote_rewrites;
+    _demote_rewrites.minus(last_demote_rewrites);
     oss << "\nrewrite trim ndirty="
         << fmt::format(dfmt, _trim_rewrites.num_n_dirty/seconds)
         << "ps, dirty="
@@ -2848,7 +2964,19 @@ cache_stats_t Cache::get_stats(
         << "ps, dirty="
         << fmt::format(dfmt, _reclaim_rewrites.num_dirty/seconds)
         << "ps, dversion="
-        << fmt::format(dfmt, _reclaim_rewrites.get_avg_version());
+        << fmt::format(dfmt, _reclaim_rewrites.get_avg_version())
+        << "; promote ndirty="
+        << fmt::format(dfmt, _promote_rewrites.num_n_dirty/seconds)
+        << "ps, dirty="
+        << fmt::format(dfmt, _promote_rewrites.num_dirty/seconds)
+        << "ps, dversion="
+        << fmt::format(dfmt, _promote_rewrites.get_avg_version())
+        << "; demote ndirty="
+        << fmt::format(dfmt, _demote_rewrites.num_n_dirty/seconds)
+        << "ps, dirty="
+        << fmt::format(dfmt, _demote_rewrites.num_dirty/seconds)
+        << "ps, dversion="
+        << fmt::format(dfmt, _demote_rewrites.get_avg_version());
 
     oss << "\ncache total"
         << cache_size_stats_t{extents_index.get_bytes(), extents_index.size()};
@@ -2907,6 +3035,8 @@ cache_stats_t Cache::get_stats(
     last_dirty_io_by_src_ext = stats.dirty_io_by_src_ext;
     last_trim_rewrites = stats.trim_rewrites;
     last_reclaim_rewrites = stats.reclaim_rewrites;
+    last_promote_rewrites = stats.promote_rewrites;
+    last_demote_rewrites = stats.demote_rewrites;
     last_cache_absent_by_src = stats.cache_absent_by_src;
     last_access_by_src_ext = stats.access_by_src_ext;
   }

src/crimson/os/seastore/cache.h

@@ -742,10 +742,18 @@ public:
     }
   }
 
+  ExtentPlacementManager& get_epm() {
+    return epm;
+  }
+
   extent_len_t get_block_size() const {
     return epm.get_block_size();
   }
 
+  ExtentPinboard *get_extent_pinboard() {
+    return pinboard.get();
+  }
+
 // Interfaces only for tests.
 public:
   CachedExtentRef test_query_cache(paddr_t offset) {
@@ -754,7 +762,12 @@ public:
   }
 
   bool can_drop_backref() const {
-    return epm.is_pure_rbm();
+    return epm.is_pure_rbm() && !force_backref;
+  }
+
+  void update_read_ratio(Transaction &t) {
+    stats.read_hit_hot += t.read_hit_hot;
+    stats.read_hit_cold += t.read_hit_cold;
   }
 
 private:
@@ -853,7 +866,8 @@ private:
 	      offset,
 	      PLACEMENT_HINT_NULL,
 	      NULL_GENERATION,
-	      TRANS_ID_NULL);
+	      nullptr,
+	      write_policy_t::WRITE_BACK);
     SUBDEBUGT(seastore_cache,
 	"{} {}~0x{:x} is absent, add extent and reading range 0x{:x}~0x{:x} ... -- {}",
 	t, T::TYPE, offset, length, partial_off, partial_len, *ret);
@@ -892,7 +906,8 @@ private:
                 offset,
                 PLACEMENT_HINT_NULL,
                 NULL_GENERATION,
-		TRANS_ID_NULL);
+		nullptr,
+		write_policy_t::WRITE_BACK);
       SUBDEBUG(seastore_cache,
           "{} {}~0x{:x} is absent, add extent and reading range 0x{:x}~0x{:x} ... -- {}",
           T::TYPE, offset, length, partial_off, partial_len, *ret);
@@ -1065,6 +1080,24 @@ private:
     return res;
   }
 
+  std::optional<backref_entry_t> get_cached_backref_entry(paddr_t addr) {
+    auto it = backref_entry_mset.lower_bound(
+      addr,
+      backref_entry_t::cmp_t());
+    if (it == backref_entry_mset.end()) {
+      return std::nullopt;
+    }
+    while (it->paddr == addr) {
+      auto &backref_entry = *it;
+      ++it;
+      if (it == backref_entry_mset.end() ||
+          it->paddr != addr) {
+        return backref_entry;
+      }
+    }
+    return std::nullopt;
+  }
+
   const backref_entry_mset_t& get_backref_entry_mset() {
     return backref_entry_mset;
   }
@@ -1142,6 +1175,7 @@ public:
     }
   }
 
+  using alloc_option_t = ExtentPlacementManager::alloc_option_t;
   /**
    * alloc_new_non_data_extent
    *
@@ -1152,25 +1186,20 @@ public:
   TCachedExtentRef<T> alloc_new_non_data_extent(
     Transaction &t,         ///< [in, out] current transaction
     extent_len_t length,    ///< [in] length
-    placement_hint_t hint,  ///< [in] user hint
-#ifdef UNIT_TESTS_BUILT
-    rewrite_gen_t gen,      ///< [in] rewrite generation
-    std::optional<paddr_t> epaddr = std::nullopt ///< [in] paddr fed by callers
-#else
-    rewrite_gen_t gen
-#endif
+    alloc_option_t opt      ///< [in] allocation options
   ) {
     LOG_PREFIX(Cache::alloc_new_non_data_extent);
-    SUBTRACET(seastore_cache, "allocate {} 0x{:x}B, hint={}, gen={}",
-              t, T::TYPE, length, hint, rewrite_gen_printer_t{gen});
-#ifdef UNIT_TESTS_BUILT
-    auto result = epm.alloc_new_non_data_extent(t, T::TYPE, length, hint, gen, epaddr);
-#else
-    auto result = epm.alloc_new_non_data_extent(t, T::TYPE, length, hint, gen);
-#endif
+    SUBTRACET(seastore_cache, "allocate {} 0x{:x}B, opt.hint={}, gen={}",
+              t, T::TYPE, length, opt.hint, rewrite_gen_printer_t{opt.gen});
+    auto result = epm.alloc_new_non_data_extent(t, T::TYPE, length, opt);
     if (!result) {
-      SUBERRORT(seastore_cache, "insufficient space", t);
-      std::rethrow_exception(crimson::ct_error::enospc::exception_ptr());
+      if (epm.is_full()) {
+        SUBERRORT(seastore_cache, "insufficient space", t);
+        std::rethrow_exception(crimson::ct_error::enospc::exception_ptr());
+      } else {
+        SUBERRORT(seastore_cache, "insufficient space, wait for demoting", t);
+        std::rethrow_exception(crimson::ct_error::eagain::exception_ptr());
+      }
     }
     auto ret = CachedExtent::make_cached_extent_ref<T>(std::move(result->bp));
     assert(is_rewrite_generation(
@@ -1178,14 +1207,15 @@ public:
       epm.dynamic_max_rewrite_generation));
     ret->init(CachedExtent::extent_state_t::INITIAL_WRITE_PENDING,
               result->paddr,
-              hint,
+              opt.hint,
               result->gen,
-	      t.get_trans_id());
+	      &t,
+	      write_policy_t::WRITE_BACK);
     t.add_fresh_extent(ret);
     SUBDEBUGT(seastore_cache,
               "allocated {} 0x{:x}B extent at {}, hint={}, gen={} -- {}",
               t, T::TYPE, length, result->paddr,
-              hint, rewrite_gen_printer_t{result->gen}, *ret);
+              opt.hint, rewrite_gen_printer_t{result->gen}, *ret);
     return ret;
   }
   /**
@@ -1198,25 +1228,20 @@ public:
   std::vector<TCachedExtentRef<T>> alloc_new_data_extents(
     Transaction &t,         ///< [in, out] current transaction
     extent_len_t length,    ///< [in] length
-    placement_hint_t hint,  ///< [in] user hint
-#ifdef UNIT_TESTS_BUILT
-    rewrite_gen_t gen,      ///< [in] rewrite generation
-    std::optional<paddr_t> epaddr = std::nullopt ///< [in] paddr fed by callers
-#else
-    rewrite_gen_t gen
-#endif
+    alloc_option_t opt      ///< [in] allocation options
   ) {
     LOG_PREFIX(Cache::alloc_new_data_extents);
     SUBTRACET(seastore_cache, "allocate {} 0x{:x}B, hint={}, gen={}",
-              t, T::TYPE, length, hint, rewrite_gen_printer_t{gen});
-#ifdef UNIT_TESTS_BUILT
-    auto results = epm.alloc_new_data_extents(t, T::TYPE, length, hint, gen, epaddr);
-#else
-    auto results = epm.alloc_new_data_extents(t, T::TYPE, length, hint, gen);
-#endif
+              t, T::TYPE, length, opt.hint, rewrite_gen_printer_t{opt.gen});
+    auto results = epm.alloc_new_data_extents(t, T::TYPE, length, opt);
     if (results.empty()) {
-      SUBERRORT(seastore_cache, "insufficient space", t);
-      std::rethrow_exception(crimson::ct_error::enospc::exception_ptr());
+      if (epm.is_full()) {
+        SUBERRORT(seastore_cache, "insufficient space", t);
+        std::rethrow_exception(crimson::ct_error::enospc::exception_ptr());
+      } else {
+        SUBERRORT(seastore_cache, "insufficient space, wait for demoting", t);
+        std::rethrow_exception(crimson::ct_error::eagain::exception_ptr());
+      }
     }
     std::vector<TCachedExtentRef<T>> extents;
     for (auto &result : results) {
@@ -1226,14 +1251,15 @@ public:
 	epm.dynamic_max_rewrite_generation));
       ret->init(CachedExtent::extent_state_t::INITIAL_WRITE_PENDING,
                 result.paddr,
-                hint,
+                opt.hint,
                 result.gen,
-                t.get_trans_id());
+                &t,
+		opt.write_policy);
       t.add_fresh_extent(ret);
       SUBDEBUGT(seastore_cache,
                 "allocated {} 0x{:x}B extent at {}, hint={}, gen={} -- {}",
                 t, T::TYPE, length, result.paddr,
-                hint, rewrite_gen_printer_t{result.gen}, *ret);
+                opt.hint, rewrite_gen_printer_t{result.gen}, *ret);
       extents.emplace_back(std::move(ret));
     }
     return extents;
@@ -1270,7 +1296,8 @@ public:
 	      remap_paddr,
 	      PLACEMENT_HINT_NULL,
 	      NULL_GENERATION,
-              t.get_trans_id());
+              &t,
+	      write_policy_t::WRITE_BACK);
 
     auto extent = ext->template cast<T>();
     extent->set_laddr(remap_laddr);
@@ -1299,7 +1326,9 @@ public:
     extent_types_t type,   ///< [in] type tag
     extent_len_t length,   ///< [in] length
     placement_hint_t hint, ///< [in] user hint
-    rewrite_gen_t gen      ///< [in] rewrite generation
+    rewrite_gen_t gen,     ///< [in] rewrite generation
+    paddr_t paddr_hint,
+    bool is_tracked
     );
 
   /**
@@ -1311,8 +1340,7 @@ public:
     Transaction &t,        ///< [in, out] current transaction
     extent_types_t type,   ///< [in] type tag
     extent_len_t length,   ///< [in] length
-    placement_hint_t hint, ///< [in] user hint
-    rewrite_gen_t gen      ///< [in] rewrite generation
+    alloc_option_t opt     ///< [in] allocation options
     );
 
   /**
@@ -1701,6 +1729,9 @@ public:
       read_extent_futs, [](auto &fut) { return std::move(fut); });
   }
 
+  bool is_on_cold_tier(paddr_t paddr) const {
+    return epm.is_cold_device(paddr.get_device_id());
+  }
 private:
   void touch_extent_fully(
       CachedExtent &ext,
@@ -1722,6 +1753,9 @@ private:
       extent_len_t load_length)
   {
     assert(ext.get_paddr().is_absolute());
+    if (ext.is_shadow_extent()) {
+      return;
+    }
     if (hint == CACHE_HINT_NOCACHE && is_logical_type(ext.get_type())) {
       return;
     }
@@ -1742,6 +1776,8 @@ private:
 
   transaction_id_t next_id = 0;
 
+  const bool force_backref = false;
+
   /**
    * dirty
    *
@@ -1875,8 +1911,15 @@ private:
     std::array<uint64_t, NUM_SRC_COMB> trans_conflicts_by_srcs;
     counter_by_src_t<uint64_t> trans_conflicts_by_unknown;
 
+    uint64_t write_hit_hot = 0;
+    uint64_t write_hit_cold = 0;
+    uint64_t read_hit_hot = 0;
+    uint64_t read_hit_cold = 0;
+
     rewrite_stats_t trim_rewrites;
     rewrite_stats_t reclaim_rewrites;
+    rewrite_stats_t promote_rewrites;
+    rewrite_stats_t demote_rewrites;
   } stats;
 
   mutable dirty_io_stats_t last_dirty_io;
@@ -1884,6 +1927,8 @@ private:
     last_dirty_io_by_src_ext;
   mutable rewrite_stats_t last_trim_rewrites;
   mutable rewrite_stats_t last_reclaim_rewrites;
+  mutable rewrite_stats_t last_promote_rewrites;
+  mutable rewrite_stats_t last_demote_rewrites;
   mutable cache_access_stats_t last_access;
   mutable counter_by_src_t<uint64_t> last_cache_absent_by_src;
   mutable counter_by_src_t<counter_by_extent_t<cache_access_stats_t> >
@@ -1905,6 +1950,10 @@ private:
 	     src2 == Transaction::src_t::CLEANER_MAIN));
     assert(!(src1 == Transaction::src_t::CLEANER_COLD &&
 	     src2 == Transaction::src_t::CLEANER_COLD));
+    assert(!(src1 == Transaction::src_t::PROMOTE &&
+	     src2 == Transaction::src_t::PROMOTE));
+    assert(!(src1 == Transaction::src_t::DEMOTE &&
+	     src2 == Transaction::src_t::DEMOTE));
     assert(!(src1 == Transaction::src_t::TRIM_ALLOC &&
              src2 == Transaction::src_t::TRIM_ALLOC));
 

src/crimson/os/seastore/cached_extent.cc

@@ -66,11 +66,59 @@ std::ostream &operator<<(std::ostream &out, CachedExtent::extent_state_t state)
   }
 }
 
+std::ostream &operator<<(std::ostream &out, const extent_pin_state_t &s) {
+  switch (s) {
+  case extent_pin_state_t::Fresh:
+    return out << "Fresh";
+  case extent_pin_state_t::PendingPromote:
+    return out << "PendingPromote";
+  case extent_pin_state_t::Promoting:
+    return out << "Promoting";
+  case extent_pin_state_t::WarmIn:
+    return out << "WarmIn";
+  case extent_pin_state_t::Hot:
+    return out << "Hot";
+  case extent_pin_state_t::Max:
+    return out << "Max";
+  default:
+    __builtin_unreachable();
+    return out;
+  }
+}
+
 std::ostream &operator<<(std::ostream &out, const CachedExtent &ext)
 {
   return ext.print(out);
 }
 
+trans_spec_view_t::trans_spec_view_t(
+  Transaction &t) : t(&t) {}
+
+bool trans_spec_view_t::cmp_t::operator()(
+  const trans_spec_view_t &lhs,
+  const trans_spec_view_t &rhs) const
+{
+  transaction_id_t l = ((lhs.t == nullptr) ? 0 : lhs.t->get_trans_id());
+  transaction_id_t r = ((rhs.t == nullptr) ? 0 : rhs.t->get_trans_id());
+  return l < r;
+}
+
+bool trans_spec_view_t::cmp_t::operator()(
+  const transaction_id_t &lhs,
+  const trans_spec_view_t &rhs) const
+{
+  transaction_id_t r = ((rhs.t == nullptr) ? 0 : rhs.t->get_trans_id());
+  return lhs < r;
+}
+
+bool trans_spec_view_t::cmp_t::operator()(
+  const trans_spec_view_t &lhs,
+  const transaction_id_t &rhs) const
+{
+  transaction_id_t l = ((lhs.t == nullptr) ? 0 : lhs.t->get_trans_id());
+  return l < rhs;
+}
+
 CachedExtent::~CachedExtent()
 {
   if (parent_index) {
@@ -106,6 +154,44 @@ CachedExtent* CachedExtent::maybe_get_transactional_view(Transaction &t) {
   return this;
 }
 
+bool CachedExtent::is_pending_in_trans(transaction_id_t id) const {
+  auto trans_id = ((t == nullptr) ? 0 : t->get_trans_id());
+  return is_pending() && trans_id == id;
+}
+
+std::ostream &CachedExtent::print(std::ostream &out) const {
+  std::string prior_poffset_str = prior_poffset
+    ? fmt::format("{}", *prior_poffset)
+    : "nullopt";
+  out << "CachedExtent(addr=" << this
+      << ", type=" << get_type()
+      << ", trans=" << ((t == nullptr) ? 0 : t->get_trans_id())
+      << ", version=" << version
+      << ", dirty_from=" << dirty_from
+      << ", modify_time=" << sea_time_point_printer_t{modify_time}
+      << ", paddr=" << get_paddr()
+      << ", prior_paddr=" << prior_poffset_str
+      << std::hex << ", length=0x" << get_length()
+      << ", loaded=0x" << get_loaded_length() << std::dec
+      << ", state=" << state
+      << ", pin_state=" << pin_state
+      << ", last_committed_crc=" << last_committed_crc
+      << ", refcount=" << use_count()
+      << ", user_hint=" << user_hint
+      << ", write_policy=" << write_policy
+      << ", rewrite_gen=" << rewrite_gen_printer_t{rewrite_generation}
+      << ", pending_io=";
+  if (is_pending_io()) {
+    out << io_wait->from_state;
+  } else {
+    out << "N/A";
+  }
+  if (is_valid() && is_fully_loaded() && !is_stable_clean_pending()) {
+    print_detail(out);
+  }
+  return out << ")";
+}
+
 std::ostream &LogicalCachedExtent::print_detail(std::ostream &out) const
 {
   out << ", laddr=" << laddr
@@ -392,13 +478,17 @@ void ExtentCommitter::commit_state() {
   SUBTRACET(seastore_cache, "{} prior={}",
     t, extent, *extent.prior_instance);
   auto &prior = *extent.prior_instance;
-  prior.pending_for_transaction = extent.pending_for_transaction;
+  prior.t = extent.t;
   prior.modify_time = extent.modify_time;
   prior.last_committed_crc = extent.last_committed_crc;
   prior.dirty_from = extent.dirty_from;
   prior.length = extent.length;
-  prior.loaded_length = extent.loaded_length;
-  prior.buffer_space = std::move(extent.buffer_space);
+  // XXX: at present, zero loaded_length extents here
+  // must have been created by promoting/demoting them.
+  if (likely(extent.loaded_length != 0)) {
+    assert(prior.loaded_length == extent.loaded_length);
+    prior.buffer_space = std::move(extent.buffer_space);
+  }
   // XXX: We can go ahead and change the prior's version because
   // transactions don't hold a local view of the version field,
   // unlike FixedKVLeafNode::modifications
@@ -414,8 +504,29 @@ void ExtentCommitter::maybe_sync_copied_lba_key() {
   auto &lextent = static_cast<LogicalChildNode&>(extent);
   auto &prior = *extent.prior_instance;
   for (auto &item : prior.read_transactions) {
-    item.t->maybe_sync_copied_lba_key(
-      lextent.get_laddr(), lextent.get_paddr());
+    switch (t.get_src()) {
+    case transaction_type_t::PROMOTE:
+      {
+        auto &shadow = *lextent.get_shadow();
+        item.t->maybe_sync_copied_lba_key(
+          lextent.get_laddr(),
+          lextent.get_paddr(),
+          shadow.get_paddr());
+        break;
+      }
+    case transaction_type_t::DEMOTE:
+      item.t->maybe_sync_copied_lba_key(
+        lextent.get_laddr(),
+        lextent.get_paddr(),
+        P_ADDR_NULL);
+      break;
+    default:
+      item.t->maybe_sync_copied_lba_key(
+        lextent.get_laddr(),
+        lextent.get_paddr(),
+        std::nullopt);
+      break;
+    }
   }
 }
 
@@ -464,7 +575,7 @@ void ExtentCommitter::_share_prior_data_to_mutations() {
           if (it != merged.end()) {
             TRACE("{} -> {}, {} -> {}",
               me, mextent, (pladdr_t)buf.val.pladdr, it->second);
-            buf.val.pladdr = pladdr_le_t(it->second);
+            buf.val = lba::lba_map_val_le_t(it->second);
           }
         }
       });
@@ -515,4 +626,48 @@ void CachedExtent::new_committer(Transaction &t) {
   prior_instance->committer = committer;
 }
 
+void ExtentCommitter::commit_shadow_demote(Transaction &t) {
+  LOG_PREFIX(ExtentCommitter::commit_shadow_demote);
+  assert(t.get_src() == transaction_type_t::DEMOTE);
+  auto &prior = *extent.prior_instance->template cast<LogicalChildNode>();
+  auto shadow = prior.get_shadow();
+  assert(shadow);
+  for (auto &trans_view : prior.retired_transactions) {
+    assert(trans_view.t != nullptr);
+    auto view_tid = trans_view.t->get_trans_id();
+    if (view_tid == t.get_trans_id()) {
+      continue;
+    }
+    TRACET("removing shadow {} from retired_set of t.{}", t, *shadow, view_tid);
+    [[maybe_unused]] bool removed =
+      trans_view.t->remove_from_retired_set(*shadow);
+    assert(removed);
+    trans_view.t->remove_shadow_from_write_set(
+      shadow->get_paddr(), shadow->get_length());
+  }
+}
+
+void ExtentCommitter::commit_shadow_promote(Transaction &t) {
+  LOG_PREFIX(ExtentCommitter::commit_shadow_promote);
+  assert(t.get_src() == transaction_type_t::PROMOTE);
+  assert(extent.is_logical());
+  auto &lprior = static_cast<LogicalChildNode&>(*extent.prior_instance);
+  ceph_assert(lprior.get_pin_state() == extent_pin_state_t::Promoting);
+  auto &lext = static_cast<LogicalChildNode&>(extent);
+  auto shadow = lext.get_shadow();
+  assert(shadow);
+  assert(shadow->is_shadow_extent());
+  lprior.set_shadow(shadow);
+  for (auto &trans_view : lprior.retired_transactions) {
+    assert(trans_view.t != nullptr);
+    auto view_tid = trans_view.t->get_trans_id();
+    if (view_tid == t.get_trans_id()) {
+      continue;
+    }
+    TRACET("adding shadow {} from t.{}", t, *shadow, view_tid);
+    trans_view.t->add_absent_to_retired_set(shadow);
+  }
+  lprior.set_pin_state(extent_pin_state_t::Fresh);
+}
+
 }

src/crimson/os/seastore/cached_extent.h

@@ -141,32 +141,22 @@ template <typename T>
 using read_trans_set_t = typename read_set_item_t<T>::trans_set_t;
 
 struct trans_spec_view_t {
-  // if the extent is pending, contains the id of the owning transaction;
-  // TRANS_ID_NULL otherwise
-  transaction_id_t pending_for_transaction = TRANS_ID_NULL;
+  // if the extent is pending, this points to the transaction
+  Transaction *t = nullptr;
   trans_spec_view_t() = default;
-  trans_spec_view_t(transaction_id_t id) : pending_for_transaction(id) {}
+  trans_spec_view_t(Transaction &t);
   virtual ~trans_spec_view_t() = default;
 
   struct cmp_t {
     bool operator()(
       const trans_spec_view_t &lhs,
-      const trans_spec_view_t &rhs) const
-    {
-      return lhs.pending_for_transaction < rhs.pending_for_transaction;
-    }
+      const trans_spec_view_t &rhs) const;
     bool operator()(
       const transaction_id_t &lhs,
-      const trans_spec_view_t &rhs) const
-    {
-      return lhs < rhs.pending_for_transaction;
-    }
+      const trans_spec_view_t &rhs) const;
     bool operator()(
       const trans_spec_view_t &lhs,
-      const transaction_id_t &rhs) const
-    {
-      return lhs.pending_for_transaction < rhs;
-    }
+      const transaction_id_t &rhs) const;
   };
 
   using trans_view_hook_t =
@@ -266,12 +256,17 @@ private:
   map_t buffer_map;
 };
 
-enum class extent_2q_state_t : uint8_t {
+enum class extent_pin_state_t : uint8_t {
+  // shared state between LRU and 2Q impl
   Fresh = 0,
+  PendingPromote,
+  Promoting,
+  // 2Q impl only
   WarmIn,
   Hot,
   Max
 };
+std::ostream &operator<<(std::ostream &, const extent_pin_state_t &);
 
 class ExtentCommitter : public boost::intrusive_ref_counter<
   ExtentCommitter, boost::thread_unsafe_counter> {
@@ -295,6 +290,8 @@ public:
   void commit_and_share_paddr();
 
   void maybe_sync_copied_lba_key();
+  void commit_shadow_demote(Transaction&);
+  void commit_shadow_promote(Transaction&);
 private:
   // the rewritten extent
   CachedExtent &extent;
@@ -356,12 +353,14 @@ public:
             paddr_t paddr,
             placement_hint_t hint,
             rewrite_gen_t gen,
-	    transaction_id_t trans_id) {
+            Transaction *trans,
+            write_policy_t policy) {
     state = _state;
     set_paddr(paddr);
     user_hint = hint;
     rewrite_generation = gen;
-    pending_for_transaction = trans_id;
+    t = trans;
+    write_policy = policy;
   }
 
   void set_modify_time(sea_time_point t) {
@@ -514,36 +513,7 @@ public:
 
   friend std::ostream &operator<<(std::ostream &, extent_state_t);
   virtual std::ostream &print_detail(std::ostream &out) const { return out; }
-  std::ostream &print(std::ostream &out) const {
-    std::string prior_poffset_str = prior_poffset
-      ? fmt::format("{}", *prior_poffset)
-      : "nullopt";
-    out << "CachedExtent(addr=" << this
-	<< ", type=" << get_type()
-	<< ", trans=" << pending_for_transaction
-	<< ", version=" << version
-	<< ", dirty_from=" << dirty_from
-	<< ", modify_time=" << sea_time_point_printer_t{modify_time}
-	<< ", paddr=" << get_paddr()
-	<< ", prior_paddr=" << prior_poffset_str
-	<< std::hex << ", length=0x" << get_length()
-	<< ", loaded=0x" << get_loaded_length() << std::dec
-	<< ", state=" << state
-	<< ", last_committed_crc=" << last_committed_crc
-	<< ", refcount=" << use_count()
-	<< ", user_hint=" << user_hint
-	<< ", rewrite_gen=" << rewrite_gen_printer_t{rewrite_generation}
-	<< ", pending_io=";
-    if (is_pending_io()) {
-      out << io_wait->from_state;
-    } else {
-      out << "N/A";
-    }
-    if (is_valid() && is_fully_loaded() && !is_stable_clean_pending()) {
-      print_detail(out);
-    }
-    return out << ")";
-  }
+  std::ostream &print(std::ostream &out) const;
 
   /**
    * get_delta
@@ -806,8 +776,14 @@ public:
   }
 
   /// assign the target rewrite generation for the followup rewrite
-  void set_target_rewrite_generation(rewrite_gen_t gen) {
-    user_hint = placement_hint_t::REWRITE;
+  void set_target_rewrite_generation(
+    rewrite_gen_t gen,
+    placement_hint_t hint = PLACEMENT_HINT_NULL) {
+    if (hint != PLACEMENT_HINT_NULL) {
+      user_hint = hint;
+    } else {
+      user_hint = placement_hint_t::REWRITE;
+    }
     rewrite_generation = gen;
   }
 
@@ -842,9 +818,7 @@ public:
   }
 
   /// Returns true if the extent part of the open transaction
-  bool is_pending_in_trans(transaction_id_t id) const {
-    return is_pending() && pending_for_transaction == id;
-  }
+  bool is_pending_in_trans(transaction_id_t id) const;
 
   enum class viewable_state_t {
     stable,                // viewable
@@ -864,17 +838,34 @@ public:
   std::pair<bool, viewable_state_t>
   is_viewable_by_trans(Transaction &t);
 
-  extent_2q_state_t get_2q_state() const {
-    assert("2Q" == crimson::common::get_conf<std::string>
-	   ("seastore_cachepin_type"));
-    return cache_state;
+  extent_pin_state_t get_pin_state() const {
+#ifndef NDEBUG
+    using crimson::common::get_conf;
+    auto type = get_conf<std::string>("seastore_cachepin_type");
+    if (type == "LRU") {
+      assert(pin_state <= extent_pin_state_t::Promoting);
+    } else if (type == "2Q") {
+      assert(pin_state < extent_pin_state_t::Max);
+    } else {
+      ceph_abort("invalid seastore_cachepin_type(LRU or 2Q)");
+    }
+#endif
+    return pin_state;
   }
 
-  void set_2q_state(extent_2q_state_t state) {
-    assert("2Q" == crimson::common::get_conf<std::string>
-	   ("seastore_cachepin_type"));
-    assert(state < extent_2q_state_t::Max);
-    cache_state = state;
+  void set_pin_state(extent_pin_state_t state) {
+    pin_state = state;
+#ifndef NDEBUG
+    using crimson::common::get_conf;
+    auto type = get_conf<std::string>("seastore_cachepin_type");
+    if (type == "LRU") {
+      assert(pin_state <= extent_pin_state_t::Promoting);
+    } else if (type == "2Q") {
+      assert(pin_state < extent_pin_state_t::Max);
+    } else {
+      ceph_abort("invalid seastore_cachepin_type(LRU or 2Q)");
+    }
+#endif
   }
 
   extent_len_t get_last_touch_end() const {
@@ -886,6 +877,26 @@ public:
     last_touch_end = touch_end;
   }
 
+  bool is_shadow_extent() const {
+    return is_shadow;
+  }
+
+  void set_shadow_extent(bool b) {
+    is_shadow = b;
+  }
+
+  write_policy_t get_write_policy() const {
+    return write_policy;
+  }
+
+  void set_write_policy(write_policy_t w) {
+    write_policy = w;
+  }
+
+  void reset_write_policy() {
+    write_policy = write_policy_t::WRITE_BACK;
+  }
+
 private:
   template <typename T>
   friend class read_set_item_t;
@@ -997,6 +1008,8 @@ private:
 
   placement_hint_t user_hint = PLACEMENT_HINT_NULL;
 
+  write_policy_t write_policy = write_policy_t::WRITE_BACK;
+
   // the target rewrite generation for the followup rewrite
   // or the rewrite generation for the fresh write
   rewrite_gen_t rewrite_generation = NULL_GENERATION;
@@ -1005,8 +1018,8 @@ private:
   // see Cache::touch_extent_by_range() and ExtentPinboardTwoQ.
   extent_len_t last_touch_end = 0;
 
-  // This field is unused when the ExtentPinboard use LRU algorithm
-  extent_2q_state_t cache_state = extent_2q_state_t::Fresh;
+  // This field is used by ExtentPinboard
+  extent_pin_state_t pin_state = extent_pin_state_t::Fresh;
 
   ExtentCommitterRef committer;
 
@@ -1014,6 +1027,8 @@ private:
 
   seastar::shared_mutex commit_lock;
 
+  bool is_shadow = false;
+
 protected:
   trans_view_set_t mutation_pending_extents;
   trans_view_set_t retired_transactions;
@@ -1096,6 +1111,7 @@ protected:
   friend class ExtentQueue;
   friend class ExtentPinboardLRU;
   friend class ExtentPinboardTwoQ;
+  friend class ExtentPromoter;
   template <typename T, typename... Args>
   static TCachedExtentRef<T> make_cached_extent_ref(
     Args&&... args) {
@@ -1279,8 +1295,8 @@ struct trans_retired_extent_link_t {
   // Otherwise, we have to search through each extent's "retired_transactions"
   // to remove the transaction
   trans_spec_view_t trans_view;
-  trans_retired_extent_link_t(CachedExtentRef extent, transaction_id_t id)
-    : extent(extent), trans_view{id}
+  trans_retired_extent_link_t(CachedExtentRef extent, Transaction &t)
+    : extent(extent), trans_view{t}
   {
     assert(extent->is_stable());
     extent->retired_transactions.insert(trans_view);
@@ -1621,25 +1637,5 @@ using lextent_list_t = addr_extent_list_base_t<
 template <> struct fmt::formatter<crimson::os::seastore::CachedExtent> : fmt::ostream_formatter {};
 template <> struct fmt::formatter<crimson::os::seastore::CachedExtent::viewable_state_t> : fmt::ostream_formatter {};
 template <> struct fmt::formatter<crimson::os::seastore::LogicalCachedExtent> : fmt::ostream_formatter {};
+template <> struct fmt::formatter<crimson::os::seastore::extent_pin_state_t> : fmt::ostream_formatter {};
 #endif
-
-template <>
-struct fmt::formatter<crimson::os::seastore::extent_2q_state_t>
-    : public fmt::formatter<std::string_view> {
-  using State = crimson::os::seastore::extent_2q_state_t;
-  auto format(const State &s, auto &ctx) const {
-    switch (s) {
-    case State::Fresh:
-      return fmt::format_to(ctx.out(), "Fresh");
-    case State::WarmIn:
-      return fmt::format_to(ctx.out(), "WarmIn");
-    case State::Hot:
-      return fmt::format_to(ctx.out(), "Hot");
-    case State::Max:
-      return fmt::format_to(ctx.out(), "Max");
-    default:
-      __builtin_unreachable();
-      return ctx.out();
-    }
-  }
-};

src/crimson/os/seastore/device.cc

@@ -58,6 +58,8 @@ void device_superblock_t::validate() const
   }
   ceph_assert(block_size > 0);
   ceph_assert(config.spec.magic != 0);
+  // allow HDD devices use segmented backend
+  ceph_assert(config.spec.btype != backend_type_t::NONE);
   ceph_assert(config.spec.id <= DEVICE_ID_MAX_VALID);
   if (!config.major_dev) {
     ceph_assert(config.secondary_devices.empty());
@@ -106,7 +108,8 @@ void device_superblock_t::validate() const
       ceph_assert(shard_infos[i].size > block_size &&
                   shard_infos[i].size % block_size == 0);
       ceph_assert_always(shard_infos[i].size <= DEVICE_OFF_MAX);
-      ceph_assert(journal_size > 0 && journal_size % block_size == 0);
+      ceph_assert((journal_size > 0 && journal_size % block_size == 0) ||
+                   config.spec.dtype == device_type_t::RANDOM_BLOCK_HDD);
       ceph_assert(shard_infos[i].start_offset < total_size &&
                   shard_infos[i].start_offset % block_size == 0);
     }
@@ -114,19 +117,23 @@ void device_superblock_t::validate() const
 }
 
 seastar::future<DeviceRef>
-Device::make_device(const std::string& device, device_type_t dtype)
+Device::make_device(
+  const std::string& device,
+  device_type_t dtype,
+  backend_type_t btype)
 {
-  if (get_default_backend_of_device(dtype) == backend_type_t::SEGMENTED) {
+  if (btype == backend_type_t::SEGMENTED) {
     return SegmentManager::get_segment_manager(device, dtype
     ).then([](DeviceRef ret) {
       return ret;
     });
-  } 
-  assert(get_default_backend_of_device(dtype) == backend_type_t::RANDOM_BLOCK);
-  return get_rb_device(device
-  ).then([](DeviceRef ret) {
-    return ret;
-  });
+  } else {
+    ceph_assert(btype != backend_type_t::NONE);
+    return get_rb_device(device, dtype
+    ).then([](DeviceRef ret) {
+      return ret;
+    });
+  }
 }
 
 check_create_device_ret check_create_device(

src/crimson/os/seastore/device.h

@@ -23,11 +23,13 @@ using magic_t = uint64_t;
 struct device_spec_t {
   magic_t magic = 0;
   device_type_t dtype = device_type_t::NONE;
+  backend_type_t btype = backend_type_t::NONE;
   device_id_t id = DEVICE_ID_NULL;
   DENC(device_spec_t, v, p) {
     DENC_START(1, 1, p);
     denc(v.magic, p);
     denc(v.dtype, p);
+    denc(v.btype, p);
     denc(v.id, p);
     DENC_FINISH(p);
   }
@@ -55,12 +57,14 @@ struct device_config_t {
     uuid_d new_osd_fsid,
     device_id_t id,
     device_type_t d_type,
+    backend_type_t b_type,
     secondary_device_set_t sds) {
     return device_config_t{
              true,
              device_spec_t{
                (magic_t)std::rand(),
 		d_type,
+                b_type,
 		id},
              seastore_meta_t{new_osd_fsid},
              sds};
@@ -69,12 +73,14 @@ struct device_config_t {
     uuid_d new_osd_fsid,
     device_id_t id,
     device_type_t d_type,
+    backend_type_t b_type,
     magic_t magic) {
     return device_config_t{
              false,
              device_spec_t{
                magic,
                d_type,
+               b_type,
                id},
              seastore_meta_t{new_osd_fsid},
              secondary_device_set_t()};
@@ -329,7 +335,8 @@ public:
 
   static seastar::future<DeviceRef> make_device(
     const std::string &device,
-    device_type_t dtype);
+    device_type_t dtype,
+    backend_type_t btype);
 
 // interfaces used by each device shard
 public:

src/crimson/os/seastore/extent_pinboard.cc

@@ -3,6 +3,7 @@
 
 #include "crimson/os/seastore/extent_pinboard.h"
 #include "crimson/os/seastore/transaction.h"
+#include "crimson/os/seastore/transaction_manager.h"
 
 #include <boost/unordered/unordered_flat_map.hpp>
 
@@ -266,8 +267,183 @@ void ExtentQueue::get_stats(
   last_overall_io = overall_io;
 }
 
+class ExtentPromoter {
+public:
+  ExtentPromoter(
+    size_t promotion_size,
+    ExtentPlacementManager &epm)
+    : promotion_size(promotion_size),
+      epm(epm),
+      test_workload(crimson::common::get_conf<bool>(
+        "seastore_logical_bucket_cache_test_stress"))
+  {}
+
+  ~ExtentPromoter() {
+    clear();
+  }
+
+  bool enabled() const {
+    return ecb != nullptr;
+  }
+
+  bool should_promote_extent(const CachedExtent &extent) {
+    return enabled() && epm.is_cold_device(extent.get_paddr().get_device_id());
+  }
+
+  size_t get_promotion_size() const {
+    if (unlikely(test_workload)) {
+      return current_contents >= promotion_size
+        ? current_contents
+        : promotion_size;
+    } else {
+      return current_contents;
+    }
+  }
+
+  void set_background_callback(BackgroundListener *l) {
+    listener = l;
+  }
+
+  void set_extent_callback(ExtentCallbackInterface *cb) {
+    ecb = cb;
+  }
+
+  bool should_run_promote() const {
+    return enabled() &&
+      (current_contents >= promotion_size
+       || test_workload);
+  }
+
+  std::size_t get_promoted_size() const {
+    return promoted_size;
+  }
+
+  std::size_t get_promoted_count() const {
+    return promoted_count;
+  }
+
+  void add_extent(CachedExtent &extent) {
+    assert(!extent.is_linked_to_list());
+    assert(extent.is_stable_clean());
+    extent.set_pin_state(extent_pin_state_t::PendingPromote);
+    list.push_back(extent);
+    current_contents += extent.get_length();
+    intrusive_ptr_add_ref(&extent);
+    while (current_contents > promotion_size) {
+      remove_extent(list.front(), extent_pin_state_t::Fresh);
+    }
+    if (should_run_promote()) {
+      assert(listener);
+      listener->maybe_wake_promote();
+    }
+  }
+
+  void remove_extent(CachedExtent &extent, extent_pin_state_t new_state) {
+    assert(extent.is_linked_to_list());
+    assert(extent.get_pin_state() == extent_pin_state_t::PendingPromote);
+    assert(current_contents >= extent.get_length());
+    extent.set_pin_state(new_state);
+    list.erase(list.s_iterator_to(extent));
+    current_contents -= extent.get_length();
+    intrusive_ptr_release(&extent);
+  }
+
+  void clear() {
+    for (auto iter = list.begin(); iter != list.end();) {
+      remove_extent(*(iter++), extent_pin_state_t::Fresh);
+    }
+  }
+
+  using run_promote_ret = base_iertr::future<>;
+  run_promote_ret run_promote(
+    Transaction &t,
+    std::list<CachedExtentRef> &promoting_extents)
+  {
+    LOG_PREFIX(ExtentPromoter::run_promote);
+    std::size_t promote_size = 0;
+    DEBUGT("start promote", t);
+    if (unlikely(test_workload &&
+                 current_contents < promotion_size &&
+                 // if promoting_extents is not empty, it means the last
+                 // round of promotion was not a test workload and was
+                 // interrupted, so this round shouldn't be a test workload
+                 // too.
+                 promoting_extents.empty())) {
+      auto id = epm.get_cold_device_id();
+      paddr_t start = P_ADDR_NULL;
+      if (device_id_to_paddr_type(id) == paddr_types_t::SEGMENT) {
+	start = paddr_t::make_seg_paddr(id, 0, 0);
+      } else {
+	start = paddr_t::make_blk_paddr(id, 0);
+      }
+      co_await ecb->promote_extents_from_disk(t, start);
+    } else {
+      std::list<CachedExtentRef> extents;
+      for (auto &extent : list) {
+        DEBUGT("promote {} to the hot tier", t, extent);
+        ceph_assert(extent.is_stable_clean());
+        ceph_assert(extent.get_pin_state() == extent_pin_state_t::PendingPromote);
+        extents.emplace_back(&extent);
+      }
+      for (auto &extent : extents) {
+        remove_extent(*extent, extent_pin_state_t::Promoting);
+      }
+      promoting_extents.insert(
+        promoting_extents.end(),
+        extents.begin(),
+        extents.end());
+      for (auto it = promoting_extents.begin();
+           it != promoting_extents.end();) {
+        auto &extent = *it;
+        if (!extent->is_valid()) {
+          it = promoting_extents.erase(it);
+          continue;
+        }
+        promote_size += extent->get_length();
+        t.add_to_read_set(extent);
+        co_await trans_intr::make_interruptible(extent->wait_io());
+        co_await ecb->promote_extent(t, extent);
+        it++;
+      }
+    }
+    // existing extents in lru will be retired after transaction submitted
+    co_await ecb->submit_transaction_direct(t);
+    promoted_count += promoting_extents.size();
+    promoted_size += promote_size;
+    DEBUGT("finish promoting {} {}B extents",
+      t, promoting_extents.size(), promote_size);
+    co_return;
+  }
+
+  seastar::future<> promote() {
+    assert(enabled());
+    std::list<CachedExtentRef> promoting_extents;
+    co_await repeat_eagain([this, &promoting_extents] {
+      return ecb->with_transaction_intr(
+        Transaction::src_t::PROMOTE,
+	"promote", cache_hint_t::get_nocache(),
+        [this, &promoting_extents](auto &t) {
+	  return run_promote(t, promoting_extents);
+	});
+    }).handle_error(crimson::ct_error::assert_all("error occupied during promotion"));
+  }
+
+private:
+  const size_t promotion_size = 0;
+  ExtentPlacementManager &epm;
+  ExtentCallbackInterface *ecb = nullptr;
+  BackgroundListener *listener = nullptr;
+  CachedExtent::primary_ref_list list;
+  size_t current_contents = 0;
+
+  size_t promoted_count = 0;
+  size_t promoted_size = 0;
+  bool test_workload = false;
+};
+
 class ExtentPinboardLRU : public ExtentPinboard {
   ExtentQueue lru;
+  ExtentPromoter promoter;
   seastar::metrics::metric_group metrics;
 
   // hit and miss indicates if an extent is linked when touching it
@@ -275,9 +451,11 @@ class ExtentPinboardLRU : public ExtentPinboard {
   uint64_t miss = 0;
 
 public:
-  ExtentPinboardLRU(std::size_t capacity) : lru(capacity) {
+  ExtentPinboardLRU(std::size_t capacity, size_t promotion_size, ExtentPlacementManager &epm)
+      : lru(capacity), promoter(promotion_size, epm) {
     LOG_PREFIX(ExtentPinboardLRU::ExtentPinboardLRU);
-    INFO("created, lru_capacity=0x{:x}B", capacity);
+    INFO("created, lru_capacity=0x{:x}B, promotion_size=0x{:x}B",
+	 capacity, promotion_size);
   }
 
   std::size_t get_capacity_bytes() const {
@@ -325,6 +503,26 @@ public:
         ),
       }
     );
+    if (promoter.enabled()) {
+      metrics.add_group(
+	"cache",
+	{
+	  sm::make_counter(
+	    "promoted_size",
+	    [this] {
+	      return promoter.get_promoted_size();
+	    },
+	    sm::description("total bytes promoted by the lru"),
+            {sm::label_instance("shard_store_index", std::to_string(store_index))}),
+	  sm::make_counter(
+	    "promoted_count",
+	    [this] {
+	      return promoter.get_promoted_count();
+	    },
+	    sm::description("total extents promoted by the lru"),
+            {sm::label_instance("shard_store_index", std::to_string(store_index))}),
+	});
+    }
   }
 
   void get_stats(
@@ -335,8 +533,29 @@ public:
   }
 
   void remove(CachedExtent &extent) final {
+    auto s = extent.get_pin_state();
     if (extent.is_linked_to_list()) {
-      lru.remove(extent);
+      if (s == extent_pin_state_t::Fresh) {
+	lru.remove(extent);
+      } else {
+	promoter.remove_extent(extent, extent_pin_state_t::Fresh);
+      }
+    } else {
+      ceph_assert(s == extent_pin_state_t::Fresh ||
+                  s == extent_pin_state_t::Promoting);
+    }
+  }
+
+  void add_to_top(
+    CachedExtent &extent,
+    const Transaction::src_t* p_src) {
+    auto trimmed = lru.add_to_top(extent, p_src);
+    if (promoter.enabled()) {
+      for (auto &extent : trimmed) {
+	if (promoter.should_promote_extent(*extent)) {
+	  promoter.add_extent(*extent);
+	}
+      }
     }
   }
 
@@ -345,11 +564,20 @@ public:
     const Transaction::src_t* p_src,
     extent_len_t /*load_start*/,
     extent_len_t /*load_length*/) final {
-    if (extent.is_linked_to_list()) {
-      lru.move_to_top(extent, p_src);
+    if (auto s = extent.get_pin_state();
+        s == extent_pin_state_t::Promoting) {
+      assert(!extent.is_linked_to_list());
+    } else if (extent.is_linked_to_list()) {
+      assert(s <= extent_pin_state_t::PendingPromote);
+      if (s == extent_pin_state_t::Fresh) {
+	lru.move_to_top(extent, p_src);
+      } else {
+	promoter.remove_extent(extent, extent_pin_state_t::Fresh);
+	add_to_top(extent, p_src);
+      }
       hit++;
     } else {
-      lru.add_to_top(extent, p_src);
+      add_to_top(extent, p_src);
       miss++;
     }
   }
@@ -358,13 +586,38 @@ public:
     CachedExtent &extent,
     extent_len_t increased_length,
     const Transaction::src_t* p_src) final {
-    if (extent.is_linked_to_list()) {
+    if (extent.is_linked_to_list() &&
+	extent.get_pin_state() == extent_pin_state_t::Fresh) {
       lru.increase_cached_size(extent, increased_length, p_src);
+    } else {
+      // promoter take the complete extent size for content size calculation
+      assert(extent.get_pin_state() <= extent_pin_state_t::Promoting);
     }
   }
 
   void clear() final {
     lru.clear();
+    promoter.clear();
+  }
+
+  void set_background_callback(BackgroundListener *listener) final {
+    promoter.set_background_callback(listener);
+  }
+
+  void set_extent_callback(ExtentCallbackInterface *cb) final {
+    promoter.set_extent_callback(cb);
+  }
+
+  std::size_t get_promotion_size() const final {
+    return promoter.get_promotion_size();
+  }
+
+  bool should_promote() const final {
+    return promoter.should_run_promote();
+  }
+
+  seastar::future<> promote() final {
+    return promoter.promote();
   }
 
   ~ExtentPinboardLRU() {
@@ -482,15 +735,22 @@ public:
   ExtentPinboardTwoQ(
     std::size_t warm_in_capacity,
     std::size_t warm_out_capacity,
-    std::size_t hot_capacity)
+    std::size_t hot_capacity,
+    std::size_t promotion_size,
+    ExtentPlacementManager &epm)
       : warm_in(warm_in_capacity),
 	warm_out(warm_out_capacity),
-	hot(hot_capacity)
+	hot(hot_capacity),
+	promoter(promotion_size, epm),
+        test_workload(crimson::common::get_conf<bool>(
+          "seastore_logical_bucket_cache_test_stress")),
+        TwoQ_promote_probability(crimson::common::get_conf<double>(
+          "seastore_test_workload_2Q_promote_probability"))
   {
     LOG_PREFIX(ExtentPinboardTwoQ::ExtentPinboardTwoQ);
-    INFO("created, warm_in_capacity=0x{:x}B, "
-	 "warm_out_capacity=0x{:x}B, hot_capacity=0x{:x}B",
-	 warm_in_capacity, warm_out_capacity, hot_capacity);
+    INFO("created, warm_in_capacity=0x{:x}B, warm_out_capacity=0x{:x}B, "
+	 "hot_capacity=0x{:x}B, promotion_size=0x{:x}B",
+	 warm_in_capacity, warm_out_capacity, hot_capacity, promotion_size);
   }
 
   std::size_t get_capacity_bytes() const {
@@ -513,17 +773,20 @@ public:
     double seconds) const final;
 
   void remove(CachedExtent &extent) final {
-    auto s = extent.get_2q_state();
+    auto s = extent.get_pin_state();
     if (extent.is_linked_to_list()) {
-      if (s == extent_2q_state_t::WarmIn) {
+      if (s == extent_pin_state_t::WarmIn) {
 	warm_in.remove(extent);
+      } else if (s == extent_pin_state_t::PendingPromote) {
+	promoter.remove_extent(extent, extent_pin_state_t::Fresh);
       } else {
-	ceph_assert(s == extent_2q_state_t::Hot);
+	ceph_assert(s == extent_pin_state_t::Hot);
 	hot.remove(extent);
       }
-      extent.set_2q_state(extent_2q_state_t::Fresh);
+      extent.set_pin_state(extent_pin_state_t::Fresh);
     } else {
-      ceph_assert(s == extent_2q_state_t::Fresh);
+      ceph_assert(s == extent_pin_state_t::Fresh ||
+                  s == extent_pin_state_t::Promoting);
     }
   }
 
@@ -532,14 +795,23 @@ public:
     const Transaction::src_t* p_src,
     extent_len_t load_start,
     extent_len_t load_length) final {
-    auto state = extent.get_2q_state();
+    auto state = extent.get_pin_state();
     auto type = extent.get_type();
+    if (state == extent_pin_state_t::Promoting) {
+      assert(!extent.is_linked_to_list());
+      return;
+    }
     if (extent.is_linked_to_list()) {
-      if (state == extent_2q_state_t::Hot) {
+      if (state == extent_pin_state_t::Hot) {
 	hot.move_to_top(extent, p_src);
 	hit_queue(overall_hits.hot_hits, p_src, type);
+      } else if (state == extent_pin_state_t::PendingPromote) {
+	promoter.remove_extent(extent, extent_pin_state_t::Hot);
+	auto trimmed_extents = hot.add_to_top(extent, p_src);
+	on_update_hot(trimmed_extents);
+	hit_queue(overall_hits.hot_hits, p_src, type);
       } else {
-	ceph_assert(state == extent_2q_state_t::WarmIn);
+	ceph_assert(state == extent_pin_state_t::WarmIn);
 	hit_queue(overall_hits.warm_in_hits, p_src, type);
 	// warm_in is a FIFO queue, do nothing here
 	// In the standard 2Q algorithm, the extent won't be considerred
@@ -549,28 +821,28 @@ public:
       hit++;
     } else if (!is_logical_type(extent.get_type())) {
       // put physical extents to hot queue directly
-      ceph_assert(state == extent_2q_state_t::Fresh);
-      extent.set_2q_state(extent_2q_state_t::Hot);
+      ceph_assert(state == extent_pin_state_t::Fresh);
+      extent.set_pin_state(extent_pin_state_t::Hot);
       auto trimmed_extents = hot.add_to_top(extent, p_src);
       on_update_hot(trimmed_extents);
       hit_queue(overall_hits.absent, p_src, type);
       miss++;
     } else { // the logical extent which is not in warm_in and not in hot
-      ceph_assert(state == extent_2q_state_t::Fresh);
+      ceph_assert(state == extent_pin_state_t::Fresh);
       auto lext = extent.cast<LogicalCachedExtent>();
       auto m = warm_out.accessed_recently(lext->get_laddr(), load_start);
       using AccessMode = IndexedFifoQueue::AccessMode;
       if (m == AccessMode::Again) {
 	// This extent was accessed recently, consider it's hot enough to
 	// promote to hot queue.
-	extent.set_2q_state(extent_2q_state_t::Hot);
+	extent.set_pin_state(extent_pin_state_t::Hot);
 	auto trimmed_extents = hot.add_to_top(extent, p_src);
 	on_update_hot(trimmed_extents);
 	hit_queue(overall_hits.hot_absent, p_src, type);
       } else {
 	// This extent didn't be accessed recently, put it warm_in queue
 	// by default.
-	extent.set_2q_state(extent_2q_state_t::WarmIn);
+	extent.set_pin_state(extent_pin_state_t::WarmIn);
 	auto trimmed_extents = warm_in.add_to_top(extent, p_src);
 	on_update_warm_in(trimmed_extents);
 	if (m == AccessMode::Missing) {
@@ -593,16 +865,18 @@ public:
     extent_len_t increased_length,
     const Transaction::src_t* p_src) final {
     if (extent.is_linked_to_list()) {
-      auto state = extent.get_2q_state();
-      if (state == extent_2q_state_t::WarmIn) {
+      auto state = extent.get_pin_state();
+      if (state == extent_pin_state_t::WarmIn) {
 	auto trimmed_extents = warm_in.increase_cached_size(
 	  extent, increased_length, p_src);
 	on_update_warm_in(trimmed_extents);
-      } else {
-	ceph_assert(state == extent_2q_state_t::Hot);
+      } else if (state == extent_pin_state_t::Hot) {
 	auto trimmed_extents = hot.increase_cached_size(
 	  extent, increased_length, p_src);
 	on_update_hot(trimmed_extents);
+      } else {
+	ceph_assert(extent.get_pin_state() ==
+		    extent_pin_state_t::PendingPromote);
       }
     }
   }
@@ -616,6 +890,23 @@ public:
     warm_in.clear();
     warm_out.clear();
     hot.clear();
+    promoter.clear();
+  }
+
+  void set_background_callback(BackgroundListener *listener) final {
+    promoter.set_background_callback(listener);
+  }
+  void set_extent_callback(ExtentCallbackInterface *cb) final {
+    promoter.set_extent_callback(cb);
+  }
+  std::size_t get_promotion_size() const final {
+    return promoter.get_promotion_size();
+  }
+  bool should_promote() const final {
+    return promoter.should_run_promote();
+  }
+  seastar::future<> promote() final {
+    return promoter.promote();
   }
 
   ~ExtentPinboardTwoQ() {
@@ -624,13 +915,17 @@ public:
 private:
   void on_update_hot(std::list<CachedExtentRef> &extents) {
     for (auto extent : extents) {
-      extent->set_2q_state(extent_2q_state_t::Fresh);
+      if (promoter.should_promote_extent(*extent)) {
+	promoter.add_extent(*extent);
+      } else {
+	extent->set_pin_state(extent_pin_state_t::Fresh);
+      }
     }
   }
   void on_update_warm_in(std::list<CachedExtentRef> &extents) {
     for (auto extent : extents) {
       ceph_assert(is_logical_type(extent->get_type()));
-      extent->set_2q_state(extent_2q_state_t::Fresh);
+      extent->set_pin_state(extent_pin_state_t::Fresh);
       auto len = extent->get_loaded_length();
       if (len == 0) {
         // The extent is possibly empty after being initially split/remapped
@@ -638,6 +933,12 @@ private:
         // to the warm out queue.
         continue;
       }
+      if (promoter.should_promote_extent(*extent)
+	  && test_workload
+	  && (double(std::rand() % 100) / 100.0) <= TwoQ_promote_probability) {
+	promoter.add_extent(*extent);
+	continue;
+      }
       auto lext = extent->cast<LogicalCachedExtent>();
       auto laddr = lext->get_laddr();
       auto end = extent->get_last_touch_end();
@@ -663,6 +964,7 @@ private:
   ExtentQueue warm_in;
   IndexedFifoQueue warm_out;
   ExtentQueue hot;
+  ExtentPromoter promoter;
   seastar::metrics::metric_group metrics;
 
   struct QueueCounter {
@@ -723,6 +1025,8 @@ private:
   // hit and miss indicates if an extent is linked when touching it
   uint64_t hit = 0;
   uint64_t miss = 0;
+  bool test_workload = false;
+  double TwoQ_promote_probability = 0;
 };
 
 void ExtentPinboardTwoQ::get_stats(
@@ -852,13 +1156,34 @@ void ExtentPinboardTwoQ::register_metrics(store_index_t store_index) {
       ),
     }
   );
+  if (promoter.enabled()) {
+    metrics.add_group(
+      "cache",
+      {
+	sm::make_counter(
+	  "promoted_size",
+	  [this] {
+	    return promoter.get_promoted_size();
+	  },
+	  sm::description("total bytes promoted by the lru"),
+          {sm::label_instance("shard_store_index", std::to_string(store_index))}),
+	sm::make_counter(
+	  "promoted_count",
+	  [this] {
+	    return promoter.get_promoted_count();
+	  },
+	  sm::description("total extents promoted by the lru"),
+          {sm::label_instance("shard_store_index", std::to_string(store_index))}),
+      });
+  }
 }
 
-ExtentPinboardRef create_extent_pinboard(std::size_t capacity) {
+ExtentPinboardRef create_extent_pinboard(std::size_t capacity, ExtentPlacementManager *epm) {
   using crimson::common::get_conf;
+  size_t promotion_size = get_conf<Option::size_t>("seastore_cache_promotion_size");
   auto algorithm = get_conf<std::string>("seastore_cachepin_type");
   if (algorithm == "LRU") {
-    return std::make_unique<ExtentPinboardLRU>(capacity);
+    return std::make_unique<ExtentPinboardLRU>(capacity, promotion_size, *epm);
   } else if (algorithm == "2Q") {
     auto warm_in_ratio = get_conf<double>("seastore_cachepin_2q_in_ratio");
     auto warm_out_ratio = get_conf<double>("seastore_cachepin_2q_out_ratio");
@@ -867,7 +1192,8 @@ ExtentPinboardRef create_extent_pinboard(std::size_t capacity) {
     return std::make_unique<ExtentPinboardTwoQ>(
       capacity * warm_in_ratio,
       capacity * warm_out_ratio,
-      capacity * (1 - warm_in_ratio));
+      capacity * (1 - warm_in_ratio),
+      promotion_size, *epm);
   } else {
     ceph_abort("invalid seastore_cachepin_type(LRU or 2Q)");
     return nullptr;

src/crimson/os/seastore/extent_pinboard.h

@@ -7,6 +7,9 @@
 #include "crimson/os/seastore/transaction.h"
 
 namespace crimson::os::seastore {
+class BackgroundListener;
+class ExtentCallbackInterface;
+class ExtentPlacementManager;
 
 struct ExtentPinboard {
   virtual ~ExtentPinboard() = default;
@@ -28,8 +31,14 @@ struct ExtentPinboard {
     extent_len_t increased_length,
     const Transaction::src_t *p_src) = 0;
   virtual void clear() = 0;
+  virtual void set_background_callback(BackgroundListener *listener) = 0;
+  virtual void set_extent_callback(ExtentCallbackInterface *cb) = 0;
+  virtual std::size_t get_promotion_size() const = 0;
+  virtual bool should_promote() const = 0;
+  virtual seastar::future<> promote() = 0;
 };
 using ExtentPinboardRef = std::unique_ptr<ExtentPinboard>;
-ExtentPinboardRef create_extent_pinboard(std::size_t capacity);
+ExtentPinboardRef create_extent_pinboard(
+  std::size_t capacity, ExtentPlacementManager *epm);
 
 } // namespace crimson::os::seastore

src/crimson/os/seastore/extent_placement_manager.cc

@@ -1,6 +1,9 @@
 // -*- mode:C++; tab-width:8; c-basic-offset:2; indent-tabs-mode:nil -*-
 // vim: ts=8 sw=2 sts=2 expandtab expandtab
 
+#include <chrono>
+#include <seastar/core/sleep.hh>
+
 #include "crimson/os/seastore/extent_placement_manager.h"
 
 #include "crimson/common/errorator-utils.h"
@@ -16,7 +19,8 @@ SegmentedOolWriter::SegmentedOolWriter(
   data_category_t category,
   rewrite_gen_t gen,
   SegmentProvider& sp,
-  SegmentSeqAllocator &ssa)
+  SegmentSeqAllocator &ssa,
+  TokenBucket &bucket)
   : store_index(store_index),
     segment_allocator(nullptr, category, gen, sp, ssa),
     record_submitter(crimson::common::get_conf<uint64_t>(
@@ -27,7 +31,8 @@ SegmentedOolWriter::SegmentedOolWriter(
                        "seastore_journal_batch_flush_size"),
                      crimson::common::get_conf<double>(
                        "seastore_journal_batch_preferred_fullness"),
-                     segment_allocator)
+                     segment_allocator),
+    token_bucket(bucket)
 {
 }
 
@@ -182,23 +187,36 @@ SegmentedOolWriter::alloc_write_ool_extents(
   std::list<CachedExtentRef>& extents)
 {
   if (extents.empty()) {
-    return alloc_write_iertr::now();
+    co_return;
   }
-  return seastar::with_gate(write_guard, [this, &t, &extents] {
-    return do_write(t, extents);
+  co_await seastar::with_gate(
+    write_guard,
+    [this, &t, &extents] -> alloc_write_iertr::future<> {
+    uint64_t size = 0;
+    for (auto &e : extents) {
+      size += e->get_length();
+    }
+    try {
+      co_await trans_intr::make_interruptible(
+        token_bucket.get(size));
+      co_await do_write(t, extents);
+    } catch (...) {
+      token_bucket.release(size);
+      throw;
+    }
   });
 }
 
 void ExtentPlacementManager::init(
     JournalTrimmerImplRef &&trimmer,
     AsyncCleanerRef &&cleaner,
-    AsyncCleanerRef &&cold_cleaner)
+    AsyncCleanerRef &&cold_cleaner,
+    ExtentPinboard *pinboard)
 {
   LOG_PREFIX(ExtentPlacementManager::init);
   writer_refs.clear();
-  auto cold_segment_cleaner = dynamic_cast<SegmentCleaner*>(cold_cleaner.get());
   dynamic_max_rewrite_generation = hot_tier_generations - 1;
-  if (cold_segment_cleaner) {
+  if (cold_cleaner) {
     dynamic_max_rewrite_generation = hot_tier_generations + cold_tier_generations - 1;
   }
   DEBUG("dynamic_max_rewrite_generation: {}, "
@@ -207,6 +225,14 @@ void ExtentPlacementManager::init(
         cold_tier_generations);
   ceph_assert(dynamic_max_rewrite_generation > MIN_REWRITE_GENERATION);
 
+  auto main_bw_limit = crimson::common::get_conf<
+    Option::size_t>("seastore_hot_backend_bw_throttle");
+  auto secondary_bw_limit = crimson::common::get_conf<
+    Option::size_t>("seastore_cold_backend_bw_throttle");
+
+  token_buckets.emplace_back(std::make_unique<TokenBucket>(main_bw_limit));
+  token_buckets.back()->start();
+
   if (trimmer->get_backend_type() == backend_type_t::SEGMENTED) {
     DEBUG("initiating SegmentCleaner");
     auto segment_cleaner = dynamic_cast<SegmentCleaner*>(cleaner.get());
@@ -219,7 +245,7 @@ void ExtentPlacementManager::init(
     for (rewrite_gen_t gen = OOL_GENERATION; gen < hot_tier_generations; ++gen) {
       writer_refs.emplace_back(std::make_unique<SegmentedOolWriter>(store_index,
 	    data_category_t::DATA, gen, *segment_cleaner,
-            *ool_segment_seq_allocator));
+            *ool_segment_seq_allocator, *token_buckets.back()));
       data_writers_by_gen[generation_to_writer(gen)] = writer_refs.back().get();
     }
 
@@ -228,7 +254,7 @@ void ExtentPlacementManager::init(
     for (rewrite_gen_t gen = OOL_GENERATION; gen < hot_tier_generations; ++gen) {
       writer_refs.emplace_back(std::make_unique<SegmentedOolWriter>(store_index,
 	    data_category_t::METADATA, gen, *segment_cleaner,
-            *ool_segment_seq_allocator));
+            *ool_segment_seq_allocator, *token_buckets.back()));
       md_writers_by_gen[generation_to_writer(gen)] = writer_refs.back().get();
     }
 
@@ -246,42 +272,67 @@ void ExtentPlacementManager::init(
     data_writers_by_gen.resize(num_writers, nullptr);
     md_writers_by_gen.resize(num_writers, {});
     writer_refs.emplace_back(std::make_unique<RandomBlockOolWriter>(
-	    rb_cleaner));
+	    rb_cleaner, *token_buckets.back()));
     // TODO: implement eviction in RBCleaner and introduce further writers
     data_writers_by_gen[generation_to_writer(OOL_GENERATION)] = writer_refs.back().get();
     md_writers_by_gen[generation_to_writer(OOL_GENERATION)] = writer_refs.back().get();
     for (auto *rb : rb_cleaner->get_rb_group()->get_rb_managers()) {
       add_device(rb->get_device());
     }
-  }
-
-  if (cold_segment_cleaner) {
-    // Cold DATA Segments
-    for (rewrite_gen_t gen = hot_tier_generations; gen <= dynamic_max_rewrite_generation; ++gen) {
-      writer_refs.emplace_back(std::make_unique<SegmentedOolWriter>(store_index,
-            data_category_t::DATA, gen, *cold_segment_cleaner,
-            *ool_segment_seq_allocator));
+    for (rewrite_gen_t gen = OOL_GENERATION; gen < hot_tier_generations; ++gen) {
       data_writers_by_gen[generation_to_writer(gen)] = writer_refs.back().get();
     }
-    for (rewrite_gen_t gen = hot_tier_generations; gen <= dynamic_max_rewrite_generation; ++gen) {
-      // Cold METADATA Segments
-      writer_refs.emplace_back(std::make_unique<SegmentedOolWriter>(store_index,
-            data_category_t::METADATA, gen, *cold_segment_cleaner,
-            *ool_segment_seq_allocator));
+    for (rewrite_gen_t gen = OOL_GENERATION; gen < hot_tier_generations; ++gen) {
       md_writers_by_gen[generation_to_writer(gen)] = writer_refs.back().get();
     }
-    for (auto *device : cold_segment_cleaner->get_segment_manager_group()
-                                            ->get_segment_managers()) {
-      add_device(device);
-    }
   }
 
+  if (cold_cleaner) {
+    token_buckets.emplace_back(std::make_unique<TokenBucket>(secondary_bw_limit));
+    token_buckets.back()->start();
+    if (cold_cleaner->get_backend_type() == backend_type_t::SEGMENTED) {
+      auto cold_segment_cleaner = static_cast<SegmentCleaner*>(cold_cleaner.get());
+      for (rewrite_gen_t gen = hot_tier_generations; gen <= dynamic_max_rewrite_generation; ++gen) {
+        writer_refs.emplace_back(std::make_unique<SegmentedOolWriter>(store_index,
+              data_category_t::DATA, gen, *cold_segment_cleaner,
+              *ool_segment_seq_allocator, *token_buckets.back()));
+        data_writers_by_gen[generation_to_writer(gen)] = writer_refs.back().get();
+      }
+      for (rewrite_gen_t gen = hot_tier_generations; gen <= dynamic_max_rewrite_generation; ++gen) {
+        writer_refs.emplace_back(std::make_unique<SegmentedOolWriter>(store_index,
+              data_category_t::METADATA, gen, *cold_segment_cleaner,
+              *ool_segment_seq_allocator, *token_buckets.back()));
+        md_writers_by_gen[generation_to_writer(gen)] = writer_refs.back().get();
+      }
+      for (auto *device : cold_segment_cleaner->get_segment_manager_group()
+                                              ->get_segment_managers()) {
+        add_device(device);
+      }
+    } else {
+      ceph_assert(cold_cleaner->get_backend_type() == backend_type_t::RANDOM_BLOCK);
+      auto rb_cleaner = static_cast<RBMCleaner*>(cold_cleaner.get());
+      ceph_assert(rb_cleaner);
+      writer_refs.emplace_back(std::make_unique<RandomBlockOolWriter>(rb_cleaner, *token_buckets.back()));
+      for (rewrite_gen_t gen = hot_tier_generations; gen <= dynamic_max_rewrite_generation; ++gen) {
+        data_writers_by_gen[generation_to_writer(gen)] = writer_refs.back().get();
+      }
+      for (rewrite_gen_t gen = hot_tier_generations; gen <= dynamic_max_rewrite_generation; ++gen) {
+        md_writers_by_gen[generation_to_writer(gen)] = writer_refs.back().get();
+      }
+      for (auto *rb : rb_cleaner->get_rb_group()->get_rb_managers()) {
+        add_device(rb->get_device());
+      }
+     }
+   }
+
+  auto cold_cleaner_ = cold_cleaner.get();
   background_process.init(std::move(trimmer),
                           std::move(cleaner),
                           std::move(cold_cleaner),
-                          hot_tier_generations);
-  if (cold_segment_cleaner) {
-    ceph_assert(get_main_backend_type() == backend_type_t::SEGMENTED);
+                          hot_tier_generations,
+                          pinboard);
+  ceph_assert(get_main_backend_type() != backend_type_t::NONE);
+  if (cold_cleaner_) {
     ceph_assert(background_process.has_cold_tier());
   } else {
     ceph_assert(!background_process.has_cold_tier());
@@ -570,6 +621,9 @@ ExtentPlacementManager::close()
 {
   LOG_PREFIX(ExtentPlacementManager::close);
   INFO("started");
+  for (auto &token_bucket : token_buckets) {
+    token_bucket->stop();
+  }
   return crimson::do_for_each(data_writers_by_gen, [](auto &writer) {
     if (writer) {
       return writer->close();
@@ -607,6 +661,9 @@ ExtentPlacementManager::mount_ret ExtentPlacementManager::BackgroundProcess::mou
   trimmer->reset();
   stats = {};
   register_metrics(store_index);
+  if (logical_bucket) {
+    logical_bucket->mount(store_index);
+  }
   DEBUG("mounting main cleaner");
   co_await main_cleaner->mount();
   if (has_cold_tier()) {
@@ -629,41 +686,49 @@ void ExtentPlacementManager::BackgroundProcess::start_background()
   ceph_assert(state == state_t::SCAN_SPACE);
   assert(!is_running());
   process_join = seastar::now();
+  promote_process_join = seastar::now();
   state = state_t::RUNNING;
   assert(is_running());
   process_join = run();
+  if (has_cold_tier()) {
+    promote_process_join = run_promote();
+  }
 }
 
 seastar::future<>
 ExtentPlacementManager::BackgroundProcess::stop_background()
 {
   LOG_PREFIX(BackgroundProcess::stop_background);
-  return seastar::futurize_invoke([this, FNAME] {
-    if (!is_running()) {
-      if (state != state_t::HALT) {
-        INFO("isn't RUNNING or HALT, STOP");
-        state = state_t::STOP;
-      } else {
-        INFO("isn't RUNNING, already HALT");
-      }
-      return seastar::now();
+  if (!is_running()) {
+    if (state != state_t::HALT) {
+      INFO("isn't RUNNING or HALT, STOP");
+      state = state_t::STOP;
+    } else {
+      INFO("isn't RUNNING, already HALT");
     }
-    INFO("is RUNNING, going to HALT...");
-    auto ret = std::move(*process_join);
-    process_join.reset();
-    state = state_t::HALT;
-    assert(!is_running());
-    do_wake_background();
-    return ret;
-  }).then([this, FNAME] {
-    INFO("done, {}, {}",
-         JournalTrimmerImpl::stat_printer_t{*trimmer, true},
-         AsyncCleaner::stat_printer_t{*main_cleaner, true});
-    if (has_cold_tier()) {
-      INFO("done, cold_cleaner: {}",
-           AsyncCleaner::stat_printer_t{*cold_cleaner, true});
-    }
-  });
+    co_return;
+  }
+  INFO("is RUNNING, going to HALT...");
+  std::vector<seastar::future<>> futs;
+  futs.emplace_back(std::move(*process_join));
+  process_join.reset();
+  if (promote_process_join) {
+    futs.emplace_back(std::move(*promote_process_join));
+    promote_process_join.reset();
+  }
+  state = state_t::HALT;
+  assert(!is_running());
+  do_wake_background();
+  do_wake_promote();
+  co_await seastar::when_all(futs.begin(), futs.end());
+  INFO("done, {}, {}",
+       JournalTrimmerImpl::stat_printer_t{*trimmer, true},
+       AsyncCleaner::stat_printer_t{*main_cleaner, true});
+  if (has_cold_tier()) {
+    INFO("done, cold_cleaner: {}",
+         AsyncCleaner::stat_printer_t{*cold_cleaner, true});
+  }
+  co_return;
 }
 
 seastar::future<>
@@ -729,7 +794,6 @@ ExtentPlacementManager::BackgroundProcess::reserve_projected_usage(
   if (!is_ready()) {
     co_return;
   }
-  ceph_assert(!blocking_io);
   // The pipeline configuration prevents another IO from entering
   // prepare until the prior one exits and clears this.
   ++stats.io_count;
@@ -759,15 +823,16 @@ ExtentPlacementManager::BackgroundProcess::reserve_projected_usage(
     // IO blocked -> needs cleaner -> cleaner sleeping -> nothing runs -> deadlock.
     // Kick the background so it can free space and call maybe_wake_blocked_io().
     auto arm_blocking_io_and_wake = [this] {
-      blocking_io = seastar::promise<>();
+      if (!blocking_io) {
+        blocking_io = seastar::shared_promise<>();
+      }
       do_wake_background();
     };
     arm_blocking_io_and_wake();
     // we just blocked this IO, now wait until
     // maybe_wake_blocked_io will set value to blocking_io
-    do {
-      co_await blocking_io->get_future();
-      ceph_assert(!blocking_io);
+    while (true) {
+      co_await blocking_io->get_shared_future();
       auto res = try_reserve_io(usage);
       if (res.is_successful()) {
         DEBUG("unblocked");
@@ -777,6 +842,7 @@ ExtentPlacementManager::BackgroundProcess::reserve_projected_usage(
         auto duration = end_time - begin_time;
         stats.io_blocked_time += std::chrono::duration_cast<
         std::chrono::milliseconds>(duration).count();
+        break;
       } else {
         DEBUG("blocked again: inline={}, main={}, cold={}, usage={}",
               res.reserve_inline_success,
@@ -792,7 +858,7 @@ ExtentPlacementManager::BackgroundProcess::reserve_projected_usage(
         }
         arm_blocking_io_and_wake();
       }
-    } while (blocking_io);
+    }
   }
 }
 
@@ -820,7 +886,8 @@ ExtentPlacementManager::BackgroundProcess::run()
 {
   assert(is_running());
   while (is_running()) {
-    if (background_should_run()) {
+    if (background_should_run()
+        || force_run_background()) {
       log_state("run(background)");
       co_await do_background_cycle();
       // Edge-triggered: yield only when a blocked IO was actually woken, so the
@@ -836,9 +903,13 @@ ExtentPlacementManager::BackgroundProcess::run()
       if (cold_cleaner) {
         cold_cleaner->maybe_adjust_thresholds();
       }
+      maybe_reschedule_force_process();
     } else {
       log_state("run(block)");
       assert(!blocking_background);
+      if (unlikely(test_workload)) {
+        set_next_force_process();
+      }
       blocking_background = seastar::promise<>();
       co_await blocking_background->get_future();
       // After waking (typically because arm_blocking_io_and_wake() kicked us),
@@ -910,23 +981,35 @@ void ExtentPlacementManager::BackgroundProcess::abort_cold_usage(
   }
 }
 
+bool ExtentPlacementManager::BackgroundProcess::try_reserve_main(
+  std::size_t usage)
+{
+  return main_cleaner->try_reserve_projected_usage(usage);
+}
+
 reserve_cleaner_result_t
 ExtentPlacementManager::BackgroundProcess::try_reserve_cleaner(
   const cleaner_usage_t &usage)
 {
   return {
-    main_cleaner->try_reserve_projected_usage(usage.main_usage),
+    try_reserve_main(usage.main_usage),
     try_reserve_cold(usage.cold_ool_usage)
   };
 }
 
+void ExtentPlacementManager::BackgroundProcess::abort_main_usage(
+  std::size_t usage, bool success)
+{
+  if (success) {
+    main_cleaner->release_projected_usage(usage);
+  }
+}
+
 void ExtentPlacementManager::BackgroundProcess::abort_cleaner_usage(
   const cleaner_usage_t &usage,
   const reserve_cleaner_result_t &result)
 {
-  if (result.reserve_main_success) {
-    main_cleaner->release_projected_usage(usage.main_usage);
-  }
+  abort_main_usage(usage.main_usage, result.reserve_main_success);
   abort_cold_usage(usage.cold_ool_usage, result.reserve_cold_success);
 }
 
@@ -980,12 +1063,24 @@ ExtentPlacementManager::BackgroundProcess::do_background_cycle()
     }
   }
 
+  bool force_trim = false;
+  bool should_abort_cleaner_usage = true;
+  if (unlikely(should_force_trim())) {
+    if (!proceed_trim) {
+      should_abort_cleaner_usage = false;
+    }
+    proceed_trim = true;
+    force_trim = true;
+  }
+
   if (proceed_trim) {
     DEBUG("started trimming...");
-    return trimmer->trim(
-    ).finally([this, trim_usage, FNAME] {
+    return trimmer->trim(force_trim
+    ).finally([this, trim_usage, should_abort_cleaner_usage, FNAME] {
       DEBUG("finished trimming");
-      abort_cleaner_usage(trim_usage, {true, true});
+      if (should_abort_cleaner_usage) {
+        abort_cleaner_usage(trim_usage, {true, true});
+      }
     });
   } else {
     assert(!proceed_trim);
@@ -1015,11 +1110,39 @@ ExtentPlacementManager::BackgroundProcess::do_background_cycle()
       proceed_clean_cold = true;
     }
 
-    if (!proceed_clean_main && !proceed_clean_cold) {
-      ceph_abort_msg("no background process will start");
+    bool proceed_demote = false;
+    if (demote_should_run()) {
+      proceed_demote = true;
+    }
+
+    bool abort_cold_cleaner_usage = true;
+    if (unlikely(should_force_clean())) {
+      if (!proceed_clean_main) {
+        abort_cold_cleaner_usage = false;
+      }
+      proceed_clean_main = main_cleaner->can_clean_space();
+      if (has_cold_tier()) {
+        proceed_clean_cold = cold_cleaner->can_clean_space();
+      }
+      if (logical_bucket) {
+        proceed_demote = logical_bucket->could_demote();
+      }
+    }
+
+    if (proceed_demote &&
+        !try_reserve_cold(logical_bucket_demote_size_per_cycle)) {
+      abort_cold_usage(logical_bucket_demote_size_per_cycle, false);
+      proceed_demote = false;
+    }
+
+    if (!proceed_clean_main && !proceed_clean_cold && !proceed_demote) {
+      // abort when the system is full, following the enospc handling
+      // in ObjectDataHandler
+      ceph_abort_msg("no background process will start, "
+                     "this probably means the underlying disks are full");
     }
     return seastar::when_all(
-      [this, FNAME, proceed_clean_main,
+      [this, FNAME, proceed_clean_main, abort_cold_cleaner_usage,
        should_clean_main_for_trim, main_cold_usage] {
         if (!proceed_clean_main) {
           return seastar::now();
@@ -1034,9 +1157,11 @@ ExtentPlacementManager::BackgroundProcess::do_background_cycle()
           crimson::ct_error::assert_all(
             "do_background_cycle encountered invalid error in main clean_space"
           )
-        ).finally([this, main_cold_usage, FNAME] {
+        ).finally([this, main_cold_usage, abort_cold_cleaner_usage, FNAME] {
           DEBUG("finished clean main");
-          abort_cold_usage(main_cold_usage, true);
+          if (abort_cold_cleaner_usage) {
+            abort_cold_usage(main_cold_usage, true);
+          }
         });
       },
       [this, FNAME, proceed_clean_cold,
@@ -1057,11 +1182,60 @@ ExtentPlacementManager::BackgroundProcess::do_background_cycle()
         ).finally([FNAME] {
           DEBUG("finished clean cold");
         });
+      },
+      [this, proceed_demote] {
+        if (!proceed_demote) {
+          return seastar::now();
+        }
+        return logical_bucket->demote(
+        ).finally([this] {
+          abort_cold_usage(logical_bucket_demote_size_per_cycle, true);
+        });
       }
     ).discard_result();
   }
 }
 
+seastar::future<> ExtentPlacementManager::BackgroundProcess::run_promote()
+{
+  assert(pinboard);
+  assert(is_running());
+  return seastar::repeat([this] {
+    if (!is_running()) {
+      return seastar::make_ready_future<seastar::stop_iteration>(
+          seastar::stop_iteration::yes);
+    }
+
+    return seastar::futurize_invoke([this] {
+      if (pinboard->should_promote()) {
+        auto usage = pinboard->get_promotion_size();
+        auto reserved = try_reserve_main(usage);
+        if (reserved) {
+          return pinboard->promote(
+          ).finally([this, usage] {
+            abort_main_usage(usage, true);
+          });
+        } else {
+          // reserve usage failed, block
+          abort_main_usage(usage, false);
+        }
+      } // shouldn't promote, block
+
+      ceph_assert(!blocking_promote);
+      blocking_promote = seastar::promise<>();
+      return blocking_promote->get_future();
+    }).then([this] {
+      if (unlikely(test_workload)) {
+        return seastar::sleep(std::chrono::seconds(
+          force_process_half_life));
+      }
+      return seastar::now();
+    }).then([] {
+      return seastar::stop_iteration::no;
+    });
+  });
+}
+
 void ExtentPlacementManager::BackgroundProcess::register_metrics(store_index_t store_index)
 {
   namespace sm = seastar::metrics;
@@ -1099,22 +1273,35 @@ RandomBlockOolWriter::alloc_write_ool_extents(
   std::list<CachedExtentRef>& extents)
 {
   if (extents.empty()) {
-    return alloc_write_iertr::now();
+    co_return;
   }
-  return seastar::with_gate(write_guard, [this, &t, &extents] {
-    seastar::lw_shared_ptr<rbm_pending_ool_t> ptr =
-      seastar::make_lw_shared<rbm_pending_ool_t>();
-    ptr->pending_extents = t.get_pre_alloc_list();
-    assert(!t.is_conflicted());
-    t.set_pending_ool(ptr);
-    return do_write(t, extents
-    ).finally([this, ptr=ptr] {
-      if (ptr->is_conflicted) {
-	for (auto &e : ptr->pending_extents) {
-	  rb_cleaner->mark_space_free(e->get_paddr(), e->get_length());
-	}
-      }
-    });
+  co_await seastar::with_gate(
+    write_guard,
+    [this, &t, &extents] -> alloc_write_iertr::future<> {
+    uint64_t size = 0;
+    for (auto &extent : extents) {
+      size += extent->get_length();
+    }
+    try {
+      co_await trans_intr::make_interruptible(
+        token_bucket.get(size));
+      seastar::lw_shared_ptr<rbm_pending_ool_t> ptr =
+          seastar::make_lw_shared<rbm_pending_ool_t>();
+      ptr->pending_extents = t.get_pre_alloc_list();
+      assert(!t.is_conflicted());
+      t.set_pending_ool(ptr);
+      co_await do_write(t, extents
+      ).finally([this, ptr=ptr] {
+        if (ptr->is_conflicted) {
+          for (auto &e : ptr->pending_extents) {
+            rb_cleaner->mark_space_free(e->get_paddr(), e->get_length());
+          }
+        }
+      });
+    } catch (...) {
+      token_bucket.release(size);
+      throw;
+    }
   });
 }
 

src/crimson/os/seastore/extent_placement_manager.h

@@ -11,6 +11,8 @@
 #include "crimson/os/seastore/journal/segment_allocator.h"
 #include "crimson/os/seastore/journal/record_submitter.h"
 #include "crimson/os/seastore/transaction.h"
+#include "crimson/os/seastore/extent_pinboard.h"
+#include "crimson/os/seastore/logical_bucket.h"
 #include "crimson/os/seastore/random_block_manager.h"
 #include "crimson/os/seastore/random_block_manager/block_rb_manager.h"
 #include "crimson/os/seastore/randomblock_manager_group.h"
@@ -21,6 +23,89 @@ namespace crimson::os::seastore {
 
 class Cache;
 
+class TokenBucket {
+  struct Blocker {
+    uint64_t size = 0;
+    seastar::promise<> pr;
+  };
+public:
+  TokenBucket(uint64_t mt) :
+    tokens(mt), max_tokens(mt), timer() {}
+
+  void start() {
+    if (max_tokens != 0) {
+      tokens = max_tokens;
+      timer.set_callback([this] {
+        if (tokens == max_tokens) {
+          return;
+        }
+        assert(tokens < max_tokens);
+        tokens += std::min(
+          max_tokens / 10 + 1,
+          max_tokens - tokens);
+        do_wake();
+      });
+      if (!timer.armed()) {
+        timer.arm_periodic(std::chrono::milliseconds(100));
+      }
+    }
+  }
+
+  void stop() {
+    if (max_tokens != 0) {
+      timer.cancel();
+      tokens = std::numeric_limits<uint64_t>::max();
+      do_wake();
+    }
+  }
+
+  seastar::future<> get(uint64_t size) {
+    if (max_tokens == 0) {
+      return seastar::now();
+    }
+    if (tokens < size) {
+      size -= tokens;
+      tokens = 0;
+      blockers.emplace_back(size);
+      return blockers.back().pr.get_future();
+    } else {
+      tokens -= size;
+      return seastar::now();
+    }
+  }
+
+  void release(uint64_t size) {
+    if (tokens == max_tokens) {
+      return;
+    }
+    assert(tokens < max_tokens);
+    tokens += std::min(size, max_tokens - tokens);
+    do_wake();
+  }
+
+private:
+  void do_wake() {
+    while (!blockers.empty()) {
+      auto &next = blockers.front();
+      if (tokens < next.size) {
+        next.size -= tokens;
+        tokens = 0;
+        break;
+      } else {
+        tokens -= next.size;
+        next.pr.set_value();
+        blockers.pop_front();
+      }
+    }
+  }
+  uint64_t tokens;
+  const uint64_t max_tokens;
+  seastar::timer<seastar::steady_clock_type> timer;
+  std::list<Blocker> blockers;
+};
+
+using TokenBucketRef = std::unique_ptr<TokenBucket>;
+
 /**
  * ExtentOolWriter
  *
@@ -41,7 +126,9 @@ public:
 
   virtual paddr_t alloc_paddr(extent_len_t length) = 0;
 
-  virtual std::list<alloc_paddr_result> alloc_paddrs(extent_len_t length) = 0;
+  virtual std::list<alloc_paddr_result> alloc_paddrs(
+    extent_len_t length,
+    paddr_t hint) = 0;
 
   using alloc_write_ertr = base_ertr;
   using alloc_write_iertr = trans_iertr<alloc_write_ertr>;
@@ -73,7 +160,8 @@ public:
                      data_category_t category,
                      rewrite_gen_t gen,
                      SegmentProvider &sp,
-                     SegmentSeqAllocator &ssa);
+                     SegmentSeqAllocator &ssa,
+                     TokenBucket &buckets);
 
   backend_type_t get_type() const final {
     return backend_type_t::SEGMENTED;
@@ -103,7 +191,7 @@ public:
     return make_delayed_temp_paddr(0);
   }
 
-  std::list<alloc_paddr_result> alloc_paddrs(extent_len_t length) final {
+  std::list<alloc_paddr_result> alloc_paddrs(extent_len_t length, paddr_t) final {
     return {alloc_paddr_result{make_delayed_temp_paddr(0), length}};
   }
 
@@ -127,13 +215,14 @@ private:
   journal::SegmentAllocator segment_allocator;
   journal::RecordSubmitter record_submitter;
   seastar::gate write_guard;
+  TokenBucket &token_bucket;
 };
 
 
 class RandomBlockOolWriter : public ExtentOolWriter {
 public:
-  RandomBlockOolWriter(RBMCleaner* rb_cleaner) :
-    rb_cleaner(rb_cleaner) {}
+  RandomBlockOolWriter(RBMCleaner* rb_cleaner, TokenBucket &bucket) :
+    rb_cleaner(rb_cleaner), token_bucket(bucket) {}
 
   backend_type_t get_type() const final {
     return backend_type_t::RANDOM_BLOCK;
@@ -169,9 +258,10 @@ public:
     return rb_cleaner->alloc_paddr(length);
   }
 
-  std::list<alloc_paddr_result> alloc_paddrs(extent_len_t length) final {
+  std::list<alloc_paddr_result> alloc_paddrs(
+    extent_len_t length, paddr_t hint) final {
     assert(rb_cleaner);
-    return rb_cleaner->alloc_paddrs(length);
+    return rb_cleaner->alloc_paddrs(length, hint);
   }
 
   bool can_inplace_rewrite(Transaction& t,
@@ -179,7 +269,9 @@ public:
     if (!extent->is_stable_dirty()) {
       return false;
     }
-    assert(t.get_src() == transaction_type_t::TRIM_DIRTY);
+    assert((t.get_src() == transaction_type_t::TRIM_DIRTY) ||
+           (t.get_src() == transaction_type_t::DEMOTE) ||
+           (t.get_src() == transaction_type_t::PROMOTE));
     ceph_assert_always(is_root_type(extent->get_type()) ||
 	extent->get_paddr().is_absolute());
     return crimson::os::seastore::can_inplace_rewrite(extent->get_type());
@@ -218,6 +310,7 @@ private:
   seastar::gate write_guard;
   writer_stats_t w_stats;
   mutable writer_stats_t last_w_stats;
+  TokenBucket &token_bucket;
 };
 
 struct cleaner_usage_t {
@@ -280,7 +373,13 @@ public:
       ool_segment_seq_allocator(
           std::make_unique<SegmentSeqAllocator>(segment_type_t::OOL)),
       max_data_allocation_size(crimson::common::get_conf<Option::size_t>(
-	  "seastore_max_data_allocation_size"))
+	  "seastore_max_data_allocation_size")),
+      write_through_size(crimson::common::get_conf<Option::size_t>(
+	  "seastore_write_through_size")),
+      test_workload(crimson::common::get_conf<bool>(
+          "seastore_logical_bucket_cache_test_stress")),
+      write_through_probability(crimson::common::get_conf<double>(
+          "seastore_test_workload_write_through_probability"))
   {
     LOG_PREFIX(ExtentPlacementManager::ExtentPlacementManager);
     devices_by_id.resize(DEVICE_ID_MAX, nullptr);
@@ -288,7 +387,8 @@ public:
       cold_tier_generations, hot_tier_generations);
   }
 
-  void init(JournalTrimmerImplRef &&, AsyncCleanerRef &&, AsyncCleanerRef &&);
+  void init(JournalTrimmerImplRef &&, AsyncCleanerRef &&, AsyncCleanerRef &&,
+            ExtentPinboard *pinboard);
 
   SegmentSeqAllocator &get_ool_segment_seq_allocator() const {
     return *ool_segment_seq_allocator;
@@ -296,6 +396,18 @@ public:
 
   void set_primary_device(Device *device);
 
+  bool is_full() const {
+    return background_process.is_full();
+  }
+
+  void maybe_wake_background() {
+    background_process.maybe_wake_background();
+  }
+
+  seastar::future<> wait_background() {
+    return background_process.wait_background();
+  }
+
   void set_extent_callback(ExtentCallbackInterface *cb) {
     background_process.set_extent_callback(cb);
   }
@@ -331,6 +443,10 @@ public:
     bool report_detail,
     double seconds) const;
 
+  LogicalBucket *get_logical_bucket() {
+    return background_process.get_logical_bucket();
+  }
+
   using mount_ertr = crimson::errorator<
       crimson::ct_error::input_output_error>;
   using mount_ret = mount_ertr::future<>;
@@ -349,6 +465,16 @@ public:
     return background_process.start_background();
   }
 
+  struct alloc_option_t {
+    placement_hint_t hint;
+    rewrite_gen_t gen;
+    bool is_tracked;
+    paddr_t paddr_hint = P_ADDR_NULL;
+    write_policy_t write_policy = write_policy_t::WRITE_BACK;
+#ifdef UNIT_TESTS_BUILT
+    std::optional<paddr_t> external_paddr = std::nullopt;
+#endif
+  };
   struct alloc_result_t {
     paddr_t paddr;
     bufferptr bp;
@@ -358,34 +484,30 @@ public:
     Transaction& t,
     extent_types_t type,
     extent_len_t length,
-    placement_hint_t hint,
-#ifdef UNIT_TESTS_BUILT
-    rewrite_gen_t gen,
-    std::optional<paddr_t> external_paddr = std::nullopt
-#else
-    rewrite_gen_t gen
-#endif
+    alloc_option_t opt
   ) {
-    assert(hint < placement_hint_t::NUM_HINTS);
-    assert(is_target_rewrite_generation(gen, dynamic_max_rewrite_generation));
-    assert(gen == INIT_GENERATION || hint == placement_hint_t::REWRITE);
+    assert(opt.hint < placement_hint_t::NUM_HINTS);
+    assert(is_target_rewrite_generation(opt.gen, dynamic_max_rewrite_generation));
+    assert(opt.gen == INIT_GENERATION ||
+           opt.hint == placement_hint_t::REWRITE ||
+           opt.hint == placement_hint_t::COLD);
 
     data_category_t category = get_extent_category(type);
-    gen = adjust_generation(category, type, hint, gen);
+    opt.gen = adjust_generation(
+      category, type, opt.hint, opt.gen, opt.write_policy, opt.is_tracked);
 
     paddr_t addr;
 #ifdef UNIT_TESTS_BUILT
-    if (unlikely(external_paddr.has_value())) {
-      assert(external_paddr->is_fake());
-      addr = *external_paddr;
-    } else if (gen == INLINE_GENERATION) {
-#else
-    if (gen == INLINE_GENERATION) {
+    if (unlikely(opt.external_paddr.has_value())) {
+      assert(opt.external_paddr->is_fake());
+      addr = *opt.external_paddr;
+    } else
 #endif
+      if (opt.gen == INLINE_GENERATION) {
       addr = make_record_relative_paddr(0);
     } else {
       assert(category == data_category_t::METADATA);
-      addr = get_writer(hint, category, gen)->alloc_paddr(length);
+      addr = get_writer(opt.hint, category, opt.gen)->alloc_paddr(length);
     }
     assert(!(category == data_category_t::DATA));
 
@@ -397,44 +519,41 @@ public:
     // according to the allocator.
     auto bp = create_extent_ptr_zero(length);
 
-    return alloc_result_t{addr, std::move(bp), gen};
+    return alloc_result_t{addr, std::move(bp), opt.gen};
   }
 
   std::list<alloc_result_t> alloc_new_data_extents(
     Transaction& t,
     extent_types_t type,
     extent_len_t length,
-    placement_hint_t hint,
-#ifdef UNIT_TESTS_BUILT
-    rewrite_gen_t gen,
-    std::optional<paddr_t> external_paddr = std::nullopt
-#else
-    rewrite_gen_t gen
-#endif
+    alloc_option_t opt
   ) {
     LOG_PREFIX(ExtentPlacementManager::alloc_new_data_extents);
-    assert(hint < placement_hint_t::NUM_HINTS);
-    assert(is_target_rewrite_generation(gen, dynamic_max_rewrite_generation));
-    assert(gen == INIT_GENERATION || hint == placement_hint_t::REWRITE);
+    assert(opt.hint < placement_hint_t::NUM_HINTS);
+    assert(is_target_rewrite_generation(opt.gen, dynamic_max_rewrite_generation));
+    assert(opt.gen == INIT_GENERATION ||
+           opt.hint == placement_hint_t::REWRITE ||
+           opt.hint == placement_hint_t::COLD);
 
     data_category_t category = get_extent_category(type);
-    gen = adjust_generation(category, type, hint, gen);
-    assert(gen != INLINE_GENERATION);
+    opt.gen = adjust_generation(
+      category, type, opt.hint, opt.gen, opt.write_policy, opt.is_tracked);
+    assert(opt.gen != INLINE_GENERATION);
 
     // XXX: bp might be extended to point to different memory (e.g. PMem)
     // according to the allocator.
     std::list<alloc_result_t> allocs;
 #ifdef UNIT_TESTS_BUILT
-    if (unlikely(external_paddr.has_value())) {
-      assert(external_paddr->is_fake());
+    if (unlikely(opt.external_paddr.has_value())) {
+      assert(opt.external_paddr->is_fake());
       auto bp = create_extent_ptr_zero(length);
-      allocs.emplace_back(alloc_result_t{*external_paddr, std::move(bp), gen});
-    } else {
-#else
-    {
+      allocs.emplace_back(alloc_result_t{*opt.external_paddr, std::move(bp), opt.gen});
+    } else
 #endif
+    {
       assert(category == data_category_t::DATA);
-      auto addrs = get_writer(hint, category, gen)->alloc_paddrs(length);
+      auto addrs = get_writer(opt.hint, category, opt.gen)->alloc_paddrs(
+        length, opt.paddr_hint);
       for (auto &ext : addrs) {
         auto left = ext.len;
         while (left > 0) {
@@ -446,10 +565,10 @@ public:
           auto start = ext.start.is_delayed()
                         ? ext.start
                         : ext.start + (ext.len - left);
-          allocs.emplace_back(alloc_result_t{start, std::move(bp), gen});
+          allocs.emplace_back(alloc_result_t{start, std::move(bp), opt.gen});
           SUBDEBUGT(seastore_epm,
-                    "allocated {} 0x{:x}B extent at {}, hint={}, gen={}",
-                    t, type, len, start, hint, gen);
+                    "allocated {} 0x{:x}B extent at {}, opt.hint={}, opt.gen={}",
+                    t, type, len, start, opt.hint, opt.gen);
           left -= len;
         }
       }
@@ -457,6 +576,19 @@ public:
     return allocs;
   }
 
+  write_policy_t get_write_policy(extent_types_t type, extent_len_t length) const {
+    if (has_cold_tier() &&  is_data_type(type)) {
+      if (length >= write_through_size
+          || (test_workload
+              && (double(std::rand() % 100) / 100.0) <=
+                  write_through_probability))
+      {
+        return write_policy_t::WRITE_THROUGH;
+      }
+    }
+    return write_policy_t::WRITE_BACK;
+  }
+
 #ifdef UNIT_TESTS_BUILT
   void prefill_fragmented_devices() {
     LOG_PREFIX(ExtentPlacementManager::prefill_fragmented_devices);
@@ -562,6 +694,11 @@ public:
     background_process.release_projected_usage(usage);
   }
 
+  bool is_storage_full() const {
+    auto *cleaner = background_process.get_main_cleaner();
+    return cleaner ? cleaner->is_storage_full() : false;
+  }
+
   backend_type_t get_main_backend_type() const {
     if (!background_process.is_no_background()) {
       return background_process.get_main_backend_type();
@@ -571,11 +708,19 @@ public:
     return primary_device->get_backend_type();
   }
 
-
   bool is_pure_rbm() const {
     return get_main_backend_type() == backend_type_t::RANDOM_BLOCK &&
-      // as of now, cold tier can only be segmented.
-      !background_process.has_cold_tier();
+      (!background_process.has_cold_tier() ||
+       (background_process.get_cold_tier_backend_type() ==
+        backend_type_t::RANDOM_BLOCK));
+  }
+
+  bool has_cold_tier() const {
+    return background_process.has_cold_tier();
+  }
+
+  bool is_cold_device(device_id_t id) const {
+    return background_process.is_cold_device(id);
   }
 
   // Testing interfaces
@@ -609,12 +754,22 @@ public:
     return !devices_by_id[addr.get_device_id()]->is_end_to_end_data_protection();
   }
 
+  rewrite_gen_t get_max_hot_gen() const {
+    return hot_tier_generations - 1;
+  }
+
+  device_id_t get_cold_device_id() const {
+    return background_process.get_cold_device_id();
+  }
+
 private:
   rewrite_gen_t adjust_generation(
       data_category_t category,
       extent_types_t type,
       placement_hint_t hint,
-      rewrite_gen_t gen) {
+      rewrite_gen_t gen,
+      write_policy_t policy,
+      bool is_tracked) {
     assert(is_real_type(type));
     if (is_root_type(type)) {
       gen = INLINE_GENERATION;
@@ -622,7 +777,6 @@ private:
                is_lba_backref_node(type)) {
       gen = INLINE_GENERATION;
     } else if (hint == placement_hint_t::COLD) {
-      assert(gen == INIT_GENERATION);
       if (background_process.has_cold_tier()) {
         gen = hot_tier_generations;
       } else {
@@ -643,10 +797,26 @@ private:
         }
       } else {
         assert(category == data_category_t::DATA);
-        gen = OOL_GENERATION;
+        if (background_process.has_cold_tier() &&
+            policy == write_policy_t::WRITE_THROUGH) {
+          gen = hot_tier_generations;
+        } else {
+          assert(policy != write_policy_t::WRITE_THROUGH);
+          gen = OOL_GENERATION;
+        }
       }
     } else if (background_process.has_cold_tier()) {
       gen = background_process.adjust_generation(gen);
+      if (gen <= hot_tier_generations &&
+          policy == write_policy_t::WRITE_THROUGH) {
+        gen = hot_tier_generations;
+      }
+    }
+
+    if (is_tracked && gen >= hot_tier_generations &&
+        hint != placement_hint_t::REWRITE &&
+        hint != placement_hint_t::COLD) {
+      gen = hot_tier_generations - 1;
     }
 
     if (gen > dynamic_max_rewrite_generation) {
@@ -729,7 +899,8 @@ private:
     void init(JournalTrimmerImplRef &&_trimmer,
               AsyncCleanerRef &&_cleaner,
               AsyncCleanerRef &&_cold_cleaner,
-              rewrite_gen_t hot_tier_generations) {
+              rewrite_gen_t hot_tier_generations,
+              ExtentPinboard *_pinboard) {
       trimmer = std::move(_trimmer);
       trimmer->set_background_callback(this);
       main_cleaner = std::move(_cleaner);
@@ -746,6 +917,7 @@ private:
           cleaners_by_device_id[id] = cold_cleaner.get();
         }
 
+        using crimson::common::get_conf;
         eviction_state.init(
           crimson::common::get_conf<double>(
             "seastore_multiple_tiers_stop_evict_ratio"),
@@ -754,7 +926,34 @@ private:
           crimson::common::get_conf<double>(
             "seastore_multiple_tiers_fast_evict_ratio"),
           hot_tier_generations);
+
+        pinboard = _pinboard;
+        ceph_assert(pinboard != nullptr);
+        pinboard->set_background_callback(this);
+
+        logical_bucket_demote_size_per_cycle =
+          get_conf<Option::size_t>("seastore_logical_bucket_proceed_size_per_cycle");
+        logical_bucket = create_logical_bucket(
+          get_conf<Option::size_t>("seastore_logical_bucket_capacity"),
+          logical_bucket_demote_size_per_cycle);
+        logical_bucket->set_background_callback(this);
       }
+      LOG_PREFIX(BackgroundProcess::init);
+      test_workload = crimson::common::get_conf<bool>(
+        "seastore_logical_bucket_cache_test_stress");
+      force_process_half_life = crimson::common::get_conf<uint64_t>(
+        "seastore_test_workload_force_process_background_tasks_period");
+      force_background_timer.set_callback([this] { wake_half_life(); });
+      write_through_probability = crimson::common::get_conf<double>(
+        "seastore_test_workload_write_through_probability");
+      SUBINFO(seastore_epm, "crimson test workload supported, enabled: {}", test_workload);
+      if (test_workload) {
+        set_next_force_process();
+      }
+    }
+
+    LogicalBucket *get_logical_bucket() {
+      return logical_bucket.get();
     }
 
     backend_type_t get_backend_type() const {
@@ -770,11 +969,26 @@ private:
       return cold_cleaner.get() != nullptr;
     }
 
+    backend_type_t get_cold_tier_backend_type() const {
+      assert(cold_cleaner);
+      return cold_cleaner->get_backend_type();
+    }
+
+    bool is_cold_device(device_id_t id) const {
+      if (!has_cold_tier()) {
+        return false;
+      }
+      assert(cleaners_by_device_id[id]);
+      return cleaners_by_device_id[id] != main_cleaner.get();
+    }
+
     void set_extent_callback(ExtentCallbackInterface *cb) {
       trimmer->set_extent_callback(cb);
       main_cleaner->set_extent_callback(cb);
       if (has_cold_tier()) {
         cold_cleaner->set_extent_callback(cb);
+        pinboard->set_extent_callback(cb);
+        logical_bucket->set_extent_callback(cb);
       }
     }
 
@@ -850,13 +1064,20 @@ private:
     }
 
     rewrite_gen_t adjust_generation(rewrite_gen_t gen) {
-      if (has_cold_tier()) {
+      if (has_cold_tier() &&
+          get_main_backend_type() == backend_type_t::SEGMENTED) {
         return eviction_state.adjust_generation_with_eviction(gen);
       } else {
         return gen;
       }
     }
 
+
+    device_id_t get_cold_device_id() const {
+      assert(has_cold_tier());
+      return *cold_cleaner->get_device_ids().begin();
+    }
+
     seastar::future<> reserve_projected_usage(io_usage_t usage);
 
     void release_projected_usage(const io_usage_t &usage) {
@@ -889,9 +1110,15 @@ private:
       return !trimmer || !main_cleaner;
     }
 
-  protected:
-    state_t get_state() const final {
-      return state;
+    bool is_full() const {
+      if (has_cold_tier()) {
+        return cold_cleaner->get_alive_ratio() >= 0.99;
+      }
+      return main_cleaner->get_alive_ratio() >= 0.99;
+    }
+
+    const AsyncCleaner* get_main_cleaner() const {
+      return main_cleaner.get();
     }
 
     void maybe_wake_background() final {
@@ -903,12 +1130,35 @@ private:
       }
     }
 
+    seastar::future<> wait_background() {
+      if (!blocking_io) {
+        blocking_io = seastar::shared_promise<>();
+      }
+      return blocking_io->get_shared_future();
+    }
+
+  protected:
+    state_t get_state() const final {
+      return state;
+    }
+
     void maybe_wake_blocked_io() final;
 
+    void maybe_wake_promote() final {
+      if (!is_ready()) {
+        return;
+      }
+      if (pinboard && pinboard->should_promote()) {
+        do_wake_promote();
+      }
+    }
+
   private:
     // reserve helpers
     bool try_reserve_cold(std::size_t usage);
+    bool try_reserve_main(std::size_t usage);
     void abort_cold_usage(std::size_t usage, bool success);
+    void abort_main_usage(std::size_t usage, bool success);
 
     reserve_cleaner_result_t try_reserve_cleaner(const cleaner_usage_t &usage);
     void abort_cleaner_usage(const cleaner_usage_t &usage,
@@ -939,6 +1189,13 @@ private:
       }
     }
 
+    void do_wake_promote() {
+      if (blocking_promote) {
+        blocking_promote->set_value();
+        blocking_promote = std::nullopt;
+      }
+    }
+
     // background_should_run() should be atomic with do_background_cycle()
     // to make sure the condition is consistent.
     bool background_should_run() {
@@ -946,12 +1203,14 @@ private:
       maybe_update_eviction_mode();
       return main_cleaner_should_run()
         || cold_cleaner_should_run()
-        || trimmer->should_trim();
+        || trimmer->should_trim()
+        || demote_should_run();
     }
 
     bool main_cleaner_should_fast_evict() const {
       return has_cold_tier() &&
-         main_cleaner->can_clean_space() &&
+          (main_cleaner->can_clean_space() ||
+           (logical_bucket && logical_bucket->could_demote())) &&
          eviction_state.is_fast_mode();
     }
 
@@ -967,6 +1226,13 @@ private:
         cold_cleaner->should_clean_space();
     }
 
+    bool demote_should_run() const {
+      return has_cold_tier() &&
+        logical_bucket->could_demote() &&
+        (eviction_state.is_fast_mode() ||
+         logical_bucket->should_demote());
+    }
+
     bool should_block_io() const {
       assert(is_ready());
       return trimmer->should_block_io_on_trim() ||
@@ -1096,6 +1362,7 @@ private:
     };
 
     seastar::future<> do_background_cycle();
+    seastar::future<> run_promote();
 
     void register_metrics(store_index_t store_index);
 
@@ -1114,6 +1381,9 @@ private:
 
     JournalTrimmerImplRef trimmer;
     AsyncCleanerRef main_cleaner;
+    ExtentPinboard *pinboard = nullptr;
+    LogicalBucketRef logical_bucket;
+    std::size_t logical_bucket_demote_size_per_cycle = 0;
 
     /*
      * cold tier (optional, see has_cold_tier())
@@ -1123,16 +1393,69 @@ private:
 
     std::optional<seastar::future<>> process_join;
     std::optional<seastar::promise<>> blocking_background;
-    std::optional<seastar::promise<>> blocking_io;
+    std::optional<seastar::shared_promise<>> blocking_io;
     // Set by maybe_wake_blocked_io() whenever it actually unblocks a
     // user IO; consumed by run() to yield exactly once on that edge,
     // giving the woken continuation a chance to retry the reservation
     // before the next background cycle.
     bool pending_user_io_wake = false;
+    std::optional<seastar::future<>> promote_process_join;
+    std::optional<seastar::promise<>> blocking_promote;
     bool is_running_until_halt = false;
     state_t state = state_t::STOP;
     eviction_state_t eviction_state;
 
+    enum class ForceProcessState : uint8_t{
+      STOP,
+      TRIM,
+      CLEAN,
+    };
+    bool test_workload = false;
+    double write_through_probability = 0;
+    ForceProcessState force_process_state = ForceProcessState::STOP;
+    ForceProcessState last_process_state = ForceProcessState::STOP;
+    seastar::timer<seastar::steady_clock_type> force_background_timer;
+    int force_process_half_life;
+
+    void set_next_force_process() {
+      assert(test_workload);
+      force_background_timer.rearm(
+        seastar::steady_clock_type::now() +
+        std::chrono::seconds(force_process_half_life));
+    }
+
+    void wake_half_life() {
+      assert(test_workload);
+      if (last_process_state == ForceProcessState::TRIM) {
+        force_process_state = ForceProcessState::CLEAN;
+      } else {
+        force_process_state = ForceProcessState::TRIM;
+      }
+
+      do_wake_background();
+    }
+
+    void maybe_reschedule_force_process() {
+      if (unlikely(test_workload &&
+                   force_process_state != ForceProcessState::STOP)) {
+        last_process_state = force_process_state;
+        force_process_state = ForceProcessState::STOP;
+        set_next_force_process();
+      }
+    }
+
+    bool should_force_trim() const {
+      return test_workload && force_process_state == ForceProcessState::TRIM;
+    }
+
+    bool should_force_clean() const {
+      return test_workload && force_process_state == ForceProcessState::CLEAN;
+    }
+
+    bool force_run_background() const {
+      return test_workload && force_process_state != ForceProcessState::STOP
+        && (logical_bucket && logical_bucket->could_demote());
+    }
     friend class ::transaction_manager_test_t;
   };
 
@@ -1140,6 +1463,7 @@ private:
   std::vector<ExtentOolWriter*> data_writers_by_gen;
   // gen 0 METADATA writer is the journal writer
   std::vector<ExtentOolWriter*> md_writers_by_gen;
+  std::vector<TokenBucketRef> token_buckets;
 
   std::vector<Device*> devices_by_id;
   Device* primary_device = nullptr;
@@ -1154,6 +1478,9 @@ private:
   // TODO: drop once paddr->journal_seq_t is introduced
   SegmentSeqAllocatorRef ool_segment_seq_allocator;
   extent_len_t max_data_allocation_size = 0;
+  std::size_t write_through_size = 0;
+  bool test_workload = false;
+  double write_through_probability = 0;
 
   friend class ::transaction_manager_test_t;
   friend class Cache;

src/crimson/os/seastore/journal.cc

@@ -5,14 +5,44 @@
 #include "journal/segmented_journal.h"
 #include "journal/circular_bounded_journal.h"
 
-namespace crimson::os::seastore::journal {
+namespace crimson::os::seastore {
+
+Journal::scan_alloc_map_ret
+Journal::scan_alloc_map() {
+  alloc_map_t map;
+  journal_seq_t tail = get_dirty_tail() <= get_alloc_tail() ?
+    get_dirty_tail() : get_alloc_tail();
+  auto build_paddr_seq_map = [&map](
+    const auto &offsets,
+    const auto &e,
+    sea_time_point modify_time)
+  {
+    if (e.type == extent_types_t::ALLOC_INFO) {
+      alloc_delta_t alloc_delta;
+      decode(alloc_delta, e.bl);
+      if (alloc_delta.op == alloc_delta_t::op_types_t::CLEAR) {
+        for (auto &alloc_blk : alloc_delta.alloc_blk_ranges) {
+          map[alloc_blk.paddr] = offsets.write_result.start_seq;
+        }
+      }
+    }
+    return replay_ertr::make_ready_future<bool>(true);
+  };
+  // build the paddr->journal_seq_t map from extent allocations
+  co_await scan_valid_record_delta(std::move(build_paddr_seq_map), tail);
+  co_return map;
+}
+
+namespace journal {
 
 JournalRef make_segmented(
   store_index_t store_index,
   SegmentProvider &provider,
-  JournalTrimmer &trimmer)
+  JournalTrimmer &trimmer,
+  bool scan_alloc_on_boot)
 {
-  return std::make_unique<SegmentedJournal>(store_index, provider, trimmer);
+  return std::make_unique<SegmentedJournal>(
+    store_index, provider, trimmer, scan_alloc_on_boot);
 }
 
 JournalRef make_circularbounded(
@@ -24,4 +54,6 @@ JournalRef make_circularbounded(
   return std::make_unique<CircularBoundedJournal>(store_index, trimmer, device, path);
 }
 
-}
+} // namespace journal
+
+} // namespace crimson::os::seastore

src/crimson/os/seastore/journal.h

@@ -106,6 +106,24 @@ public:
   virtual backend_type_t get_type() = 0;
 
   virtual bool is_checksum_needed() = 0; 
+
+protected:
+  using alloc_map_t = std::map<paddr_t, journal_seq_t>;
+  using scan_alloc_map_ertr = replay_ertr;
+  using scan_alloc_map_ret = scan_alloc_map_ertr::future<alloc_map_t>;
+  scan_alloc_map_ret scan_alloc_map();
+
+  using scan_delta_handler_t = std::function<
+    replay_ertr::future<bool>(
+      const record_locator_t&,
+      const delta_info_t&,
+      sea_time_point modify_time)>;
+  virtual replay_ret scan_valid_record_delta(
+    scan_delta_handler_t &&delta_handler,
+    journal_seq_t tail) = 0;
+
+  virtual journal_seq_t get_dirty_tail() const = 0;
+  virtual journal_seq_t get_alloc_tail() const = 0;
 };
 using JournalRef = std::unique_ptr<Journal>;
 
@@ -114,7 +132,8 @@ namespace journal {
 JournalRef make_segmented(
   store_index_t store_index,
   SegmentProvider &provider,
-  JournalTrimmer &trimmer);
+  JournalTrimmer &trimmer,
+  bool scan_alloc_on_boot);
 
 JournalRef make_circularbounded(
   store_index_t store_index,

src/crimson/os/seastore/journal/circular_bounded_journal.cc

@@ -141,7 +141,7 @@ CircularBoundedJournal::submit_record(
 }
 
 Journal::replay_ret CircularBoundedJournal::replay_segment(
-   cbj_delta_handler_t &handler, scan_valid_records_cursor& cursor)
+   scan_delta_handler_t &handler, scan_valid_records_cursor& cursor)
 {
   LOG_PREFIX(Journal::replay_segment);
   return seastar::do_with(
@@ -232,7 +232,7 @@ Journal::replay_ret CircularBoundedJournal::replay_segment(
 
 
 Journal::replay_ret CircularBoundedJournal::scan_valid_record_delta(
-   cbj_delta_handler_t &&handler, journal_seq_t tail)
+   scan_delta_handler_t &&handler, journal_seq_t tail)
 {
   LOG_PREFIX(Journal::scan_valid_record_delta);
   INFO("starting at {} ", tail);
@@ -341,7 +341,6 @@ Journal::replay_ret CircularBoundedJournal::replay(
   cjs.set_cbj_header(head);
   DEBUG("header : {}", cjs.get_cbj_header());
   cjs.set_initialized(true);
-  std::map<paddr_t, journal_seq_t> map;
   std::map<paddr_t, std::pair<CachedExtentRef, uint32_t>> crc_info;
   auto tail = get_dirty_tail() <= get_alloc_tail() ?
     get_dirty_tail() : get_alloc_tail();
@@ -360,27 +359,9 @@ Journal::replay_ret CircularBoundedJournal::replay(
     return replay_ertr::make_ready_future<bool>(true);
   };
   co_await scan_valid_record_delta(std::move(find_tail), tail);
-  tail = get_dirty_tail() <= get_alloc_tail() ?
-    get_dirty_tail() : get_alloc_tail();
-  auto build_paddr_seq_map = [&map](
-    const auto &offsets,
-    const auto &e,
-    sea_time_point modify_time)
-  {
-    if (e.type == extent_types_t::ALLOC_INFO) {
-      alloc_delta_t alloc_delta;
-      decode(alloc_delta, e.bl);
-      if (alloc_delta.op == alloc_delta_t::op_types_t::CLEAR) {
-        for (auto &alloc_blk : alloc_delta.alloc_blk_ranges) {
-          map[alloc_blk.paddr] = offsets.write_result.start_seq;
-        }
-      }
-    }
-    return replay_ertr::make_ready_future<bool>(true);
-  };
   // The second pass to build the paddr->journal_seq_t map
   // from extent allocations
-  co_await scan_valid_record_delta(std::move(build_paddr_seq_map), tail);
+  alloc_map_t map = co_await scan_alloc_map();
   auto call_d_handler_if_valid = [this, &map, &d_handler, &crc_info](
     const auto &offsets,
     const auto &e,

src/crimson/os/seastore/journal/circular_bounded_journal.h

@@ -147,14 +147,8 @@ public:
     return cjs.get_records_start();
   }
 
-  using cbj_delta_handler_t = std::function<
-  replay_ertr::future<bool>(
-    const record_locator_t&,
-    const delta_info_t&,
-    sea_time_point modify_time)>;
-
   Journal::replay_ret scan_valid_record_delta(
-    cbj_delta_handler_t &&delta_handler,
+    scan_delta_handler_t &&delta_handler,
     journal_seq_t tail);
 
   void try_read_rolled_header(scan_valid_records_cursor &cursor) {
@@ -170,7 +164,7 @@ public:
   };
 
   Journal::replay_ret replay_segment(
-    cbj_delta_handler_t &handler, scan_valid_records_cursor& cursor);
+    scan_delta_handler_t &handler, scan_valid_records_cursor& cursor);
 
   read_ret read(paddr_t start, size_t len) final;
 

src/crimson/os/seastore/journal/segmented_journal.cc

@@ -10,6 +10,7 @@
 #include "segmented_journal.h"
 
 #include "crimson/common/config_proxy.h"
+#include "crimson/common/coroutine.h"
 #include "crimson/os/seastore/logging.h"
 
 SET_SUBSYS(seastore_journal);
@@ -29,7 +30,8 @@ namespace crimson::os::seastore::journal {
 SegmentedJournal::SegmentedJournal(
   store_index_t store_index,
   SegmentProvider &segment_provider,
-  JournalTrimmer &trimmer)
+  JournalTrimmer &trimmer,
+  bool scan_alloc_on_startup)
   : store_index(store_index),
     segment_seq_allocator(
       new SegmentSeqAllocator(segment_type_t::JOURNAL)),
@@ -48,7 +50,8 @@ SegmentedJournal::SegmentedJournal(
                        "seastore_journal_batch_preferred_fullness"),
                      journal_segment_allocator),
     sm_group(*segment_provider.get_segment_manager_group()),
-    trimmer{trimmer}
+    trimmer{trimmer},
+    scan_alloc_on_startup(scan_alloc_on_startup)
 {
 }
 
@@ -108,8 +111,8 @@ SegmentedJournal::prep_replay_segments(
   return scan_last_segment(last_segment_id, last_header
   ).safe_then([this, FNAME, segments=std::move(segments)] {
     INFO("dirty_tail={}, alloc_tail={}",
-         trimmer.get_dirty_tail(),
-         trimmer.get_alloc_tail());
+         get_dirty_tail(),
+         get_alloc_tail());
     auto journal_tail = trimmer.get_journal_tail();
     auto journal_tail_paddr = journal_tail.offset;
     ceph_assert(journal_tail != JOURNAL_SEQ_NULL);
@@ -129,9 +132,9 @@ SegmentedJournal::prep_replay_segments(
 
     auto num_segments = segments.end() - from;
     INFO("{} segments to replay", num_segments);
-    auto ret = replay_segments_t(num_segments);
+    replay_segments.resize(num_segments);
     std::transform(
-      from, segments.end(), ret.begin(),
+      from, segments.end(), replay_segments.begin(),
       [this](const auto &p) {
         auto ret = journal_seq_t{
           p.second.segment_seq,
@@ -141,10 +144,9 @@ SegmentedJournal::prep_replay_segments(
         };
         return std::make_pair(ret, p.second);
       });
-    ret[0].first.offset = journal_tail_paddr;
+    replay_segments[0].first.offset = journal_tail_paddr;
     return prep_replay_segments_fut(
-      replay_ertr::ready_future_marker{},
-      std::move(ret));
+      replay_ertr::ready_future_marker{});
   });
 }
 
@@ -229,15 +231,14 @@ SegmentedJournal::replay_ertr::future<>
 SegmentedJournal::replay_segment(
   journal_seq_t seq,
   segment_header_t header,
-  delta_handler_t &handler,
-  replay_stats_t &stats)
+  scan_delta_handler_t &handler)
 {
   LOG_PREFIX(Journal::replay_segment);
   INFO("starting at {} -- {}", seq, header);
   return seastar::do_with(
     scan_valid_records_cursor(seq),
     SegmentManagerGroup::found_record_handler_t(
-      [&handler, this, &stats](
+      [&handler, this](
       record_locator_t locator,
       const record_group_header_t& header,
       const bufferlist& mdbuf)
@@ -259,16 +260,14 @@ SegmentedJournal::replay_segment(
         [write_result=locator.write_result,
          this,
          FNAME,
-         &handler,
-         &stats](auto& record_deltas_list)
+         &handler](auto& record_deltas_list)
       {
         return crimson::do_for_each(
           record_deltas_list,
           [write_result,
            this,
            FNAME,
-           &handler,
-           &stats](record_deltas_t& record_deltas)
+           &handler](record_deltas_t& record_deltas)
         {
           ++stats.num_records;
           auto locator = record_locator_t{
@@ -281,30 +280,11 @@ SegmentedJournal::replay_segment(
           return crimson::do_for_each(
             record_deltas.deltas,
             [locator,
-             this,
-             &handler,
-             &stats](auto &p)
+             &handler](auto &p)
           {
 	    auto& modify_time = p.first;
 	    auto& delta = p.second;
-	    return handler(
-	      locator,
-	      delta,
-	      trimmer.get_dirty_tail(),
-	      trimmer.get_alloc_tail(),
-              modify_time
-            ).safe_then([&stats, delta_type=delta.type](auto ret) {
-	      auto [is_applied, ext] = ret;
-              if (is_applied) {
-                // see Cache::replay_delta()
-                assert(delta_type != extent_types_t::JOURNAL_TAIL);
-                if (delta_type == extent_types_t::ALLOC_INFO) {
-                  ++stats.num_alloc_deltas;
-                } else {
-                  ++stats.num_dirty_deltas;
-                }
-              }
-            });
+            return handler(locator, delta, modify_time).discard_result();
           });
         });
       });
@@ -325,36 +305,65 @@ SegmentedJournal::replay_segment(
   );
 }
 
+SegmentedJournal::replay_ret
+SegmentedJournal::scan_valid_record_delta(
+  scan_delta_handler_t &&delta_handler,
+  journal_seq_t tail)
+{
+  auto handler = std::move(delta_handler);
+  for (auto &[seq, header] : replay_segments) {
+    co_await replay_segment(seq, header, handler);
+  }
+}
+
 SegmentedJournal::replay_ret SegmentedJournal::replay(
   delta_handler_t &&delta_handler)
 {
   LOG_PREFIX(Journal::replay);
-  return sm_group.find_journal_segment_headers(
-  ).safe_then([this, FNAME, delta_handler=std::move(delta_handler)]
-    (auto &&segment_headers) mutable -> replay_ret {
-    INFO("got {} segments", segment_headers.size());
-    return seastar::do_with(
-      std::move(delta_handler),
-      replay_segments_t(),
-      replay_stats_t(),
-      [this, segment_headers=std::move(segment_headers), FNAME]
-      (auto &handler, auto &segments, auto &stats) mutable -> replay_ret {
-	return prep_replay_segments(std::move(segment_headers)
-	).safe_then([this, &handler, &segments, &stats](auto replay_segs) mutable {
-	  segments = std::move(replay_segs);
-	  return crimson::do_for_each(segments,[this, &handler, &stats](auto i) mutable {
-	    return replay_segment(i.first, i.second, handler, stats);
-	  });
-        }).safe_then([&stats, FNAME] {
-          INFO("replay done, record_groups={}, records={}, "
-               "alloc_deltas={}, dirty_deltas={}",
-               stats.num_record_groups,
-               stats.num_records,
-               stats.num_alloc_deltas,
-               stats.num_dirty_deltas);
-        });
-      });
-  });
+  auto handler = std::move(delta_handler);
+  auto segment_headers = co_await sm_group.find_journal_segment_headers();
+  INFO("got {} segments", segment_headers.size());
+  co_await prep_replay_segments(std::move(segment_headers));
+  alloc_map_t alloc_map;
+  if (scan_alloc_on_startup) {
+    alloc_map = co_await scan_alloc_map();
+  }
+  auto d_handler = [&handler, this, &alloc_map](
+    const record_locator_t &locator,
+    const delta_info_t &delta,
+    sea_time_point modify_time) -> replay_ertr::future<bool> {
+    if (auto it = alloc_map.find(delta.paddr);
+        it != alloc_map.end() &&
+        it->second > locator.write_result.start_seq) {
+      co_return true;
+    }
+    auto ret = co_await handler(
+      locator,
+      delta,
+      get_dirty_tail(),
+      get_alloc_tail(),
+      modify_time);
+    auto [is_applied, ext] = ret;
+    if (is_applied) {
+      // see Cache::replay_delta()
+      assert(delta.type != extent_types_t::JOURNAL_TAIL);
+      if (delta.type == extent_types_t::ALLOC_INFO) {
+        ++stats.num_alloc_deltas;
+      } else {
+        ++stats.num_dirty_deltas;
+      }
+    }
+    co_return true;
+  };
+  journal_seq_t tail = get_dirty_tail() <= get_alloc_tail() ?
+    get_dirty_tail() : get_alloc_tail();
+  co_return co_await scan_valid_record_delta(std::move(d_handler), tail);
+  INFO("replay done, record_groups={}, records={}, "
+       "alloc_deltas={}, dirty_deltas={}",
+       stats.num_record_groups,
+       stats.num_records,
+       stats.num_alloc_deltas,
+       stats.num_dirty_deltas);
 }
 
 seastar::future<> SegmentedJournal::flush(OrderingHandle &handle)

src/crimson/os/seastore/journal/segmented_journal.h

@@ -28,7 +28,8 @@ public:
   SegmentedJournal(
       store_index_t store_index,
       SegmentProvider &segment_provider,
-      JournalTrimmer &trimmer);
+      JournalTrimmer &trimmer,
+      bool scan_alloc_on_startup);
   ~SegmentedJournal() {}
 
   JournalTrimmer &get_trimmer() final {
@@ -83,10 +84,7 @@ private:
   WritePipeline* write_pipeline = nullptr;
 
   /// return ordered vector of segments to replay
-  using replay_segments_t = std::vector<
-    std::pair<journal_seq_t, segment_header_t>>;
-  using prep_replay_segments_fut = replay_ertr::future<
-    replay_segments_t>;
+  using prep_replay_segments_fut = replay_ertr::future<>;
   prep_replay_segments_fut prep_replay_segments(
     std::vector<std::pair<segment_id_t, segment_header_t>> segments);
 
@@ -100,16 +98,32 @@ private:
     std::size_t num_records = 0;
     std::size_t num_alloc_deltas = 0;
     std::size_t num_dirty_deltas = 0;
-  };
+  } stats;
+
+  const bool scan_alloc_on_startup = false;
+  using replay_segments_t = std::vector<
+    std::pair<journal_seq_t, segment_header_t>>;
+  replay_segments_t replay_segments;
 
   /// replays records starting at start through end of segment
   replay_ertr::future<>
   replay_segment(
     journal_seq_t start,             ///< [in] starting addr, seq
     segment_header_t header,         ///< [in] segment header
-    delta_handler_t &delta_handler,  ///< [in] processes deltas in order
-    replay_stats_t &stats            ///< [out] replay stats
+    scan_delta_handler_t &delta_handler  ///< [in] processes deltas in order
   );
+
+  journal_seq_t get_dirty_tail() const final {
+    return trimmer.get_dirty_tail();
+  }
+
+  journal_seq_t get_alloc_tail() const final {
+    return trimmer.get_alloc_tail();
+  }
+
+  replay_ret scan_valid_record_delta(
+    scan_delta_handler_t &&delta_handler,
+    journal_seq_t tail) final;
 };
 
 }

src/crimson/os/seastore/lba/btree_lba_manager.cc

@@ -7,6 +7,7 @@
 #include <seastar/core/metrics.hh>
 
 #include "include/buffer.h"
+#include "crimson/common/coroutine.h"
 #include "crimson/os/seastore/lba/btree_lba_manager.h"
 #include "crimson/os/seastore/lba/lba_btree_node.h"
 #include "crimson/os/seastore/logging.h"
@@ -172,7 +173,12 @@ BtreeLBAManager::mkfs(
   auto croot = co_await cache.get_root(t);
   assert(croot);
   assert(croot->is_mutation_pending());
-  croot->get_root().lba_root = LBABtree::mkfs(croot, get_context(t));
+  croot->get_root().lba_root =
+    co_await LBABtree::mkfs(croot, get_context(t)
+    ).handle_error_interruptible(
+      mkfs_iertr::pass_further{},
+      crimson::ct_error::assert_all("unexpected error")
+    );
 }
 
 /**
@@ -300,6 +306,7 @@ BtreeLBAManager::resolve_indirect_cursor(
   const LBACursor &indirect_cursor)
 {
   ceph_assert(indirect_cursor.is_indirect());
+  ceph_assert(!indirect_cursor.has_shadow_paddr());
   return get_cursors(
     c,
     btree,
@@ -329,6 +336,102 @@ BtreeLBAManager::lower_bound(
   co_return iter.get_cursor(c);
 }
 
+BtreeLBAManager::upper_bound_right_ret
+BtreeLBAManager::upper_bound_right(
+  Transaction &t,
+  laddr_t laddr)
+{
+  auto c = get_context(t);
+  auto btree = co_await get_btree<LBABtree>(c);
+  auto iter = co_await btree.upper_bound_right(c, laddr);
+  if (iter.is_end()) {
+    co_await upper_bound_right_iertr::future<LBACursorRef>(
+      crimson::ct_error::enoent::make());
+  }
+  assert(iter.get_key() >= laddr);
+  co_return iter.get_cursor(c);
+}
+
+BtreeLBAManager::promote_extent_ret
+BtreeLBAManager::promote_extent(
+  Transaction &t,
+  LBACursor &cursor,
+  std::vector<LogicalChildNodeRef> extents)
+{
+  LOG_PREFIX(BtreeLBAManager::promote_extent);
+  auto laddr = cursor.get_laddr();
+  ceph_assert(!extents.empty());
+  ceph_assert(!cursor.is_indirect());
+  ceph_assert(laddr == extents.front()->get_laddr());
+  DEBUGT("promote cursor {} with {} extents",
+	 t, cursor, extents.size());
+  auto c = get_context(t);
+  auto btree = co_await get_btree<LBABtree>(c);
+  auto iter = btree.make_partial_iter(c, cursor);
+  auto orig_val = iter.get_val();
+  if (extents.size() == 1) {
+    auto new_val = orig_val;
+    ceph_assert(new_val.pladdr.is_paddr());
+    new_val.shadow_paddr = new_val.pladdr.get_paddr();
+    auto extent = extents.front().get();
+    auto paddr = extent->get_paddr();
+    new_val.pladdr = pladdr_t(paddr);
+    TRACET("promote {} from {} to {}",
+	   t, iter.get_key(), new_val.shadow_paddr, paddr);
+    assert(extent->is_pending());
+    assert(!extent->has_parent_tracker());
+    iter = btree.update(c, iter, new_val, extent);
+    assert(extent->has_parent_tracker());
+  } else {
+    auto insert_iter = co_await btree.remove(c, std::move(iter));
+    for (auto &extent : extents) {
+      auto offset = extent->get_laddr().get_byte_distance<extent_len_t>(laddr);
+      auto new_val = orig_val;
+      new_val.shadow_paddr = orig_val.pladdr.get_paddr().add_offset(offset);
+      new_val.pladdr = pladdr_t(extent->get_paddr());
+      new_val.len = extent->get_length();
+      new_val.checksum = extent->get_last_committed_crc();
+      TRACET("insert promoted cursor {} {}",
+	     c.trans, extent->get_laddr(), new_val);
+      assert(!extent->has_parent_tracker());
+      auto [iter, inserted] = co_await btree.insert(
+	c, std::move(insert_iter), extent->get_laddr(), new_val, extent.get());
+      ceph_assert(inserted);
+      assert(extent->has_parent_tracker());
+      insert_iter = co_await iter.next(c);
+    }
+  }
+  co_return;
+}
+BtreeLBAManager::demote_extent_ret
+BtreeLBAManager::demote_extent(
+  Transaction &t,
+  LBACursor &cursor,
+  LogicalChildNode &extent)
+{
+  assert(cursor.is_viewable());
+  assert(!cursor.is_end());
+  assert(!cursor.is_indirect());
+  assert(cursor.has_shadow_paddr());
+  auto c = get_context(t);
+  auto btree = co_await get_btree<LBABtree>(c);
+  auto ret = co_await _update_mapping(
+    t,
+    cursor,
+    [&extent](lba_map_val_t val) {
+      assert(val.pladdr.is_paddr());
+      assert(val.shadow_paddr == extent.get_paddr());
+      val.pladdr = pladdr_t(val.shadow_paddr);
+      val.shadow_paddr = P_ADDR_NULL;
+      return val;
+    },
+    &extent
+  ).handle_error_interruptible(
+    demote_extent_iertr::pass_further{},
+    crimson::ct_error::assert_all("unexpected enoent"));
+  co_return ret;
+}
+
 /**
  * reserve_region: insert a zero-mapping (P_ADDR_ZERO) at the specified laddr.
  * Uses the cursor as a btree insertion hint.  The reserved_ptr child pointer
@@ -351,6 +454,7 @@ BtreeLBAManager::reserve_region(
   lba_map_val_t val{
     len,
     pladdr_t{P_ADDR_ZERO},
+    P_ADDR_NULL,
     EXTENT_DEFAULT_REF_COUNT,
     0,
     type};
@@ -392,6 +496,7 @@ BtreeLBAManager::alloc_extents(
       lba_map_val_t{
 	ext->get_length(),
 	pladdr_t{ext->get_paddr()},
+        P_ADDR_NULL,
 	EXTENT_DEFAULT_REF_COUNT,
 	ext->get_last_committed_crc(),
         ext->get_type()},
@@ -451,6 +556,7 @@ BtreeLBAManager::clone_mapping(
     lba_map_val_t{
       len,
       pladdr_t{inter_key.get_local_clone_id()},
+      P_ADDR_NULL,
       EXTENT_DEFAULT_REF_COUNT,
       0,
       mapping->get_extent_type()},
@@ -880,7 +986,8 @@ BtreeLBAManager::scan_mappings(
 	  }
 	  ceph_assert((pos.get_key() + pos.get_val().len) > begin);
 	  if (pos.get_val().pladdr.is_paddr()) {
-	    f(pos.get_key(), pos.get_val().pladdr.get_paddr(), pos.get_val().len);
+	    f(pos.get_key(), pos.get_val().pladdr.get_paddr(),
+	      pos.get_val().shadow_paddr, pos.get_val().len);
 	  }
 	  return LBABtree::iterate_repeat_ret_inner(
 	    interruptible::ready_future_marker{},
@@ -956,11 +1063,19 @@ BtreeLBAManager::update_mapping(
       assert(!addr.is_null());
       lba_map_val_t ret = in;
       ceph_assert(in.pladdr.is_paddr());
-      ceph_assert(in.pladdr.get_paddr() == prev_addr);
       ceph_assert(in.len == prev_len);
-      ret.pladdr = addr;
-      ret.len = len;
-      ret.checksum = checksum;
+      if (prev_addr == in.pladdr.get_paddr()) {
+        ret.pladdr = addr;
+        ret.len = len;
+        ret.checksum = checksum;
+        if (ret.shadow_paddr != P_ADDR_NULL) {
+          ceph_assert(
+            addr.get_device_id() != ret.shadow_paddr.get_device_id());
+        }
+      } else {
+        ceph_assert(in.shadow_paddr == prev_addr);
+        ret.shadow_paddr = addr;
+      }
       return ret;
     },
     &nextent
@@ -1017,13 +1132,27 @@ BtreeLBAManager::update_mappings(
 	  return this->_update_mapping(
 	    c.trans,
 	    *cursor,
-	    [prev_addr, addr, len, checksum](
+	    [prev_addr, addr, len, checksum, extent, c](
 	      const lba_map_val_t &in) {
 	      lba_map_val_t ret = in;
 	      ceph_assert(in.pladdr.is_paddr());
-	      ceph_assert(in.pladdr.get_paddr() == prev_addr);
 	      ceph_assert(in.len == len);
-	      ret.pladdr = addr;
+	      if (likely(in.pladdr.get_paddr() == prev_addr)) {
+                ret.pladdr = addr;
+                if (c.trans.get_src() == transaction_type_t::TRIM_DIRTY) {
+                  // This is a dirty onode/omap extent that are rewritten,
+                  // the shadow extent should be removed.
+                  ret.shadow_paddr = P_ADDR_NULL;
+                }
+              } else {
+                // this can only happen when the extent is EXIST_CLEAN
+                // and is demoted onto the cold tier by a DEMOTE trans.
+                assert(in.shadow_paddr == P_ADDR_NULL);
+                assert(extent->is_exist_clean());
+                assert(extent->get_paddr() == in.pladdr.get_paddr());
+                assert(c.cache.is_on_cold_tier(extent->get_paddr()));
+                assert(!c.cache.is_on_cold_tier(prev_addr));
+              }
 	      ret.checksum = checksum;
 	      return ret;
 	    },
@@ -1314,6 +1443,9 @@ BtreeLBAManager::remap_mappings(
       } else {
         auto paddr = val.pladdr.get_paddr();
         val.pladdr = paddr + cur_off;
+        if (val.shadow_paddr != P_ADDR_NULL) {
+          val.shadow_paddr = val.shadow_paddr.add_offset(cur_off);
+        }
       }
       val.len = remap.len;
       val.refcount = EXTENT_DEFAULT_REF_COUNT;
@@ -1363,35 +1495,82 @@ BtreeLBAManager::remap_mappings(
 void BtreeLBAManager::update_paddr_sync(
   Transaction &t,
   laddr_t laddr,
-  paddr_t paddr)
+  paddr_t paddr,
+  extent_len_t len,
+  std::optional<paddr_t> shadow)
 {
   LOG_PREFIX(BtreeLBAManager::update_paddr_sync);
   DEBUGT("laddr={}, paddr={}", t, laddr, paddr);
   auto c = get_context(t);
   auto btree = get_btree_sync<LBABtree>(c);
   auto iter = btree.lower_bound_sync(c, laddr);
-  assert(iter.get_leaf_node()->is_pending());
-  auto child = iter.get_leaf_node()->get_child_sync<LogicalChildNode>(
-    c.trans, c.cache, iter.get_leaf_pos(), iter.get_key());
-  ceph_assert(child);
-  if (child->is_initial_pending()) {
-    TRACET("{} is initial_pending, skipping", t, *child);
-    return;
+  while (iter.get_key() + iter.get_val().len <= laddr + len) {
+    assert(iter.get_leaf_node()->is_pending());
+    if (iter.get_val().pladdr.is_laddr() ||
+        iter.get_val().pladdr.get_paddr().is_zero()) {
+      TRACET("skipping mapping {}~{}", t, iter.get_key(), iter.get_val());
+      if (!iter.next_sync(c)) {
+        // can't reach the next mapping, which means the next
+        // mapping mustn't have been touched by the current
+        // transactions, we don't need to continue, just leave
+        return;
+      }
+      continue;
+    }
+    auto child = iter.get_leaf_node()->get_child_sync<LogicalChildNode>(
+      c.trans, c.cache, iter.get_leaf_pos(), iter.get_key());
+    ceph_assert(is_valid_child_ptr(child.get()));
+    if (child->is_initial_pending()) {
+      TRACET("{} is initial_pending, skipping", t, *child);
+      if (!iter.next_sync(c)) {
+        // can't reach the next mapping, which means the next
+        // mapping mustn't have been touched by the current
+        // transactions, we don't need to continue, just leave
+        return;
+      }
+      continue;
+    }
+    ceph_assert(child->is_exist_clean());
+    auto cursor = iter.get_cursor(c);
+    extent_len_t off = cursor->get_laddr().get_byte_distance<
+      extent_len_t>(laddr);
+    paddr_t shadow_paddr;
+    if (shadow) {
+      // the committing txn changed the shadow
+      // to *shadow
+      shadow_paddr = *shadow;
+      if (shadow_paddr != P_ADDR_NULL) {
+        shadow_paddr = shadow_paddr + off;
+      }
+    } else if (cursor->has_shadow_paddr()) {
+      // shadow is preserved by the committer,
+      // and the copy inherited one, so the source
+      // was promoted when it's copied
+      shadow_paddr = cursor->get_shadow_paddr();
+    } else {
+      // shadow is preserved, and nothing is
+      // inherited: not currently promoted
+      shadow_paddr = P_ADDR_NULL;
+    }
+    iter = btree.update(
+      c,
+      std::move(iter),
+      lba_map_val_t{
+        cursor->get_length(),
+        pladdr_t{paddr + off},
+        shadow_paddr,
+        cursor->get_refcount(),
+        cursor->get_checksum(),
+        cursor->get_extent_type()},
+      nullptr,
+      modification_t::TRANS_SYNC);
+    if (!iter.next_sync(c)) {
+      // can't reach the next mapping, which means the next
+      // mapping mustn't have been touched by the current
+      // transactions, we don't need to continue, just leave
+      return;
+    }
   }
-  ceph_assert(child->is_exist_clean());
-  auto cursor = iter.get_cursor(c);
-  assert(cursor->get_laddr() == laddr);
-  btree.update(
-    c,
-    std::move(iter),
-    lba_map_val_t{
-      cursor->get_length(),
-      pladdr_t{std::move(paddr)},
-      cursor->get_refcount(),
-      cursor->get_checksum(),
-      cursor->get_extent_type()},
-    nullptr,
-    modification_t::TRANS_SYNC);
 }
 
 // ---------------------------------------------------------------------------
@@ -1445,8 +1624,10 @@ BtreeLBAManager::_copy_mapping(
   c.trans.new_lba_key_copied(
     ret.src->get_key(),
     dest_laddr,
-    [this, c](laddr_t laddr, paddr_t paddr) {
-      update_paddr_sync(c.trans, laddr, paddr);
+    ret.src->get_length(),
+    [this, c](laddr_t laddr, paddr_t paddr,
+              extent_len_t len, std::optional<paddr_t> shadow) {
+      update_paddr_sync(c.trans, laddr, paddr, len, shadow);
     });
   auto [niter, inserted] = co_await btree.copy(
       c,
@@ -1539,6 +1720,7 @@ BtreeLBAManager::move_and_clone_direct_mapping(
       lba_map_val_t val = in;
       val.pladdr = ret.dest->get_key().get_local_clone_id();
       val.checksum = 0;
+      val.shadow_paddr = P_ADDR_NULL;
       return val;
     },
     nullptr

src/crimson/os/seastore/lba/btree_lba_manager.h

@@ -148,6 +148,20 @@ public:
     Transaction &t,
     laddr_t laddr) final;
 
+  upper_bound_right_ret upper_bound_right(
+    Transaction &t,
+    laddr_t laddr) final;
+
+  promote_extent_ret promote_extent(
+    Transaction &t,
+    LBACursor &cursor,
+    std::vector<LogicalChildNodeRef> extents) final;
+
+  demote_extent_ret demote_extent(
+    Transaction &t,
+    LBACursor &cursor,
+    LogicalChildNode &extent) final;
+
    // ---------------------------------------------------------------------------
    // Allocation operations - insert new mappings into the LBA tree.
    // ---------------------------------------------------------------------------
@@ -209,6 +223,7 @@ public:
     laddr_t dest_laddr,
     LBACursorRef dest) final {
     assert(src->is_indirect());
+    assert(!src->has_shadow_paddr());
     return _move_mapping(
       t, std::move(src), dest_laddr, std::move(dest), nullptr);
   }
@@ -503,6 +518,7 @@ private:
 	{
 	  len,
 	  pladdr_t(P_ADDR_ZERO),
+	  P_ADDR_NULL,
 	  EXTENT_DEFAULT_REF_COUNT,
 	  0,
 	  type
@@ -517,6 +533,7 @@ private:
 	{
 	  len,
 	  pladdr_t(intermediate_key.get_local_clone_id()),
+	  P_ADDR_NULL,
 	  EXTENT_DEFAULT_REF_COUNT,
 	  0,	// crc will only be used and checked with LBA direct mappings
 		// also see pin_to_extent(_by_type)
@@ -533,7 +550,7 @@ private:
       LogicalChildNode& extent) {
       return {
 	laddr,
-	{len, pladdr_t(paddr), refcount, checksum, extent.get_type()},
+	{len, pladdr_t(paddr), P_ADDR_NULL, refcount, checksum, extent.get_type()},
 	&extent
       };
     }
@@ -600,7 +617,9 @@ private:
   void update_paddr_sync(
     Transaction &t,
     laddr_t laddr,
-    paddr_t paddr);
+    paddr_t paddr,
+    extent_len_t len,
+    std::optional<paddr_t> shadow);
 
 
   /**

src/crimson/os/seastore/lba/lba_btree_node.cc

@@ -50,8 +50,9 @@ void LBALeafNode::update(
   modification_t mod)
 {
   LOG_PREFIX(LBALeafNode::update);
+  assert(this->t != nullptr);
   SUBTRACE(seastore_fixedkv_tree, "trans.{}, pos {}",
-    this->pending_for_transaction,
+    this->t->get_trans_id(),
     iter.get_offset());
   if (likely(mod == modification_t::USER_MODIFY)) {
     this->on_modify();
@@ -71,8 +72,9 @@ LBALeafNode::internal_const_iterator_t LBALeafNode::insert(
   lba_map_val_t val)
 {
   LOG_PREFIX(LBALeafNode::insert);
+  assert(this->t != nullptr);
   SUBTRACE(seastore_fixedkv_tree, "trans.{}, pos {}, key {}",
-    this->pending_for_transaction,
+    this->t->get_trans_id(),
     iter.get_offset(),
     addr);
   this->on_modify();

src/crimson/os/seastore/lba/lba_btree_node.h

@@ -101,14 +101,14 @@ using LBAInternalNodeRef = LBAInternalNode::Ref;
  *   checksum   : ceph_le32[1]                4B
  *   size       : ceph_le32[1]                4B
  *   meta       : lba_node_meta_le_t[1]       36B
- *   keys       : laddr_le_t[CAPACITY]        (106*16)B
- *   values     : lba_map_val_le_t[CAPACITY]  (106*21)B
- *                                            = 4077B
+ *   keys       : laddr_le_t[CAPACITY]        (88*16)B
+ *   values     : lba_map_val_le_t[CAPACITY]  (88*30)B
+ *                                            = 4092B
  *
  * TODO: update FixedKVNodeLayout to handle the above calculation
  * TODO: the above alignment probably isn't portable without further work
  */
-constexpr size_t LEAF_NODE_CAPACITY = 106;
+constexpr size_t LEAF_NODE_CAPACITY = 88;
 
 struct LBALeafNode
   : FixedKVLeafNode<
@@ -165,8 +165,9 @@ struct LBALeafNode
     laddr_t pivot,
     lba_map_val_t val) {
     LOG_PREFIX(FixedKVInternalNode::replace);
+    assert(this->t != nullptr);
     SUBTRACE(seastore_fixedkv_tree, "trans.{}, pos {}, old key {}, key {}",
-      this->pending_for_transaction,
+      this->t->get_trans_id(),
       iter.get_offset(),
       iter.get_key(),
       pivot);
@@ -180,8 +181,9 @@ struct LBALeafNode
 
   void remove(internal_const_iterator_t iter) final {
     LOG_PREFIX(LBALeafNode::remove);
+    assert(this->t != nullptr);
     SUBTRACE(seastore_fixedkv_tree, "trans.{}, pos {}, key {}",
-      this->pending_for_transaction,
+      this->t->get_trans_id(),
       iter.get_offset(),
       iter.get_key());
     assert(iter != this->end());
@@ -318,13 +320,14 @@ struct LBALeafNode
 
   std::ostream &print_detail(std::ostream &out) const final;
 
-  std::map<laddr_t, pladdr_t> merge_content_to(
+  std::map<laddr_t, lba_map_val_t> merge_content_to(
     Transaction &t,
     LBALeafNode &pending_version,
     iterator &iter)
   {
     LOG_PREFIX(LBALeafNode::merge_content_to);
-    std::map<laddr_t, pladdr_t> modified;
+    SUBTRACET(seastore_lba, "merging with {}", t, pending_version);
+    std::map<laddr_t, lba_map_val_t> modified;
     auto it = pending_version.begin();
     while (it != pending_version.end() && iter != this->end()) {
       const auto &v1 = iter->get_val();
@@ -345,14 +348,24 @@ struct LBALeafNode
         ceph_abort();
       }
       if (is_valid_child_ptr(child)) {
-        if ((child->_is_mutable() || child->_is_pending_io())) {
-          // skip the ones that the pending version is also modifying
+        if (// skip the ones that the pending version is also modifying
+           (child->_is_mutable() || child->_is_pending_io()) ||
+           // EXIST_CLEAN extents created by DEMOTE transactions also
+           // updates their paddrs, so they should also be skpped.
+           (pending_version.t->get_src() == transaction_type_t::DEMOTE)) {
+          SUBTRACET(seastore_lba, "skipping {}~{}", t, it->get_key(), it->get_val());
           it++;
           continue;
         } else {
-          assert(child->_is_exist_clean() || child->_is_exist_mutation_pending());
+          assert(child->_is_exist_clean() ||
+                 child->_is_exist_mutation_pending() ||
+                 // a pending child might have been invalidated and
+                 // its corresponding mapping has not been removed
+                 !child->_is_valid());
         }
       }
+      SUBTRACET(seastore_lba, "examing v2: {}~{}, v1: {}~{}",
+        t, it->get_key(), it->get_val(), iter->get_key(), iter->get_val());
       auto pending_key = it->get_key();
       auto stable_key = iter->get_key();
       auto stable_end = stable_key + v1.len;
@@ -362,13 +375,21 @@ struct LBALeafNode
         if (pending_key != stable_key) {
           assert(v2.pladdr != v1.pladdr);
           assert(is_valid_child_ptr(child));
+          assert(child->_is_exist_clean());
         }
         if (v2.pladdr != v1.pladdr) {
           auto m_v2 = v2;
           auto off = pending_key.get_byte_distance<extent_len_t>(stable_key);
           auto paddr = v1.pladdr.get_paddr();
           paddr = paddr + off;
+          SUBTRACET(seastore_lba, "merging to {}, paddr: {} -> {}",
+            t, pending_version, m_v2.pladdr, paddr);
           m_v2.pladdr = paddr;
+          if (v1.shadow_paddr == P_ADDR_NULL) {
+            m_v2.shadow_paddr = P_ADDR_NULL;
+          } else {
+            m_v2.shadow_paddr = (v1.shadow_paddr + off);
+          }
           SUBTRACET(seastore_lba, "merging to {}, paddr: {} -> {}",
             t, pending_version, m_v2.pladdr, paddr);
           if (!is_valid_child_ptr(child)) {
@@ -378,7 +399,7 @@ struct LBALeafNode
             m_v2.checksum = v1.checksum;
           }
           it->set_val(m_v2);
-          auto [_it, inserted] = modified.emplace(it->get_key(), paddr);
+          auto [_it, inserted] = modified.emplace(it->get_key(), m_v2);
           ceph_assert(inserted);
         }
         it++;
@@ -480,12 +501,22 @@ struct LBACursor : BtreeCursor<laddr_t, lba::lba_map_val_t, LBALeafNode> {
   }
 
   extent_types_t get_extent_type() const {
+    assert(iter.get_val().type != extent_types_t::NONE);
     assert(is_viewable());
     assert(!is_end());
     assert(iter.get_val().type != extent_types_t::NONE);
     return iter.get_val().type;
   }
 
+  bool has_shadow_paddr() const {
+    return iter.get_val().shadow_paddr != P_ADDR_NULL;
+  }
+
+  paddr_t get_shadow_paddr() const {
+    assert(has_shadow_paddr());
+    return iter.get_val().shadow_paddr;
+  }
+
   base_iertr::future<> refresh();
 private:
 
@@ -500,7 +531,6 @@ using LBACursorRef = boost::intrusive_ptr<LBACursor>;
 
 #if FMT_VERSION >= 90000
 template <> struct fmt::formatter<crimson::os::seastore::lba::lba_node_meta_t> : fmt::ostream_formatter {};
-template <> struct fmt::formatter<crimson::os::seastore::lba::lba_map_val_t> : fmt::ostream_formatter {};
 template <> struct fmt::formatter<crimson::os::seastore::lba::LBAInternalNode> : fmt::ostream_formatter {};
 template <> struct fmt::formatter<crimson::os::seastore::lba::LBACursor> : fmt::ostream_formatter {};
 #endif

src/crimson/os/seastore/lba_manager.h

@@ -60,6 +60,13 @@ public:
     Transaction &t,
     laddr_t laddr) = 0;
 
+  using upper_bound_right_iertr = base_iertr::extend<
+    crimson::ct_error::enoent>;
+  using upper_bound_right_ret = upper_bound_right_iertr::future<LBACursorRef>;
+  virtual upper_bound_right_ret upper_bound_right(
+    Transaction &t,
+    laddr_t laddr) = 0;
+
 #ifdef UNIT_TESTS_BUILT
   using get_end_mapping_iertr = base_iertr;
   using get_end_mapping_ret = get_end_mapping_iertr::future<LBACursorRef>;
@@ -167,6 +174,20 @@ public:
     LBACursorRef dest,
     LogicalChildNode &extent) = 0;
 
+  using promote_extent_iertr = base_iertr;
+  using promote_extent_ret = promote_extent_iertr::future<>;
+  virtual promote_extent_ret promote_extent(
+    Transaction &t,
+    LBACursor &cursor,
+    std::vector<LogicalChildNodeRef> extents) = 0;
+
+  using demote_extent_iertr = base_iertr;
+  using demote_extent_ret = demote_extent_iertr::future<LBACursorRef>;
+  virtual demote_extent_ret demote_extent(
+    Transaction &t,
+    LBACursor &cursor,
+    LogicalChildNode &extent) = 0;
+
   virtual alloc_extent_ret reserve_region(
     Transaction &t,
     laddr_hint_t hint,
@@ -253,7 +274,7 @@ public:
   using scan_mappings_iertr = base_iertr;
   using scan_mappings_ret = scan_mappings_iertr::future<>;
   using scan_mappings_func_t = std::function<
-    void(laddr_t, paddr_t, extent_len_t)>;
+    void(laddr_t, paddr_t, paddr_t, extent_len_t)>;
   virtual scan_mappings_ret scan_mappings(
     Transaction &t,
     laddr_t begin,

src/crimson/os/seastore/lba_mapping.h

@@ -132,6 +132,18 @@ public:
     return direct_cursor->get_length();
   }
 
+  bool has_shadow_val() const {
+    assert(is_linked_direct());
+    assert(!direct_cursor->is_end());
+    return direct_cursor->has_shadow_paddr();
+  }
+
+  paddr_t get_shadow_val() const {
+    assert(is_linked_direct());
+    assert(!direct_cursor->is_end());
+    return direct_cursor->get_shadow_paddr();
+  }
+
   checksum_t get_checksum() const {
     assert(is_linked_direct());
     assert(!direct_cursor->is_end());

src/crimson/os/seastore/linked_tree_node.h

@@ -117,7 +117,7 @@ protected:
     auto pi = me.get_prior_instance();
     auto &prior = *pi->template cast<T>();
     ceph_assert(prior.parent_of_root);
-    ceph_assert(me.pending_for_transaction);
+    ceph_assert(me.t != nullptr);
     parent_of_root = prior.parent_of_root;
     TreeRootLinker<ParentT, T>::link_root(parent_of_root, &me);
     return;
@@ -215,9 +215,9 @@ public:
   virtual bool _is_mutable() const = 0;
   virtual bool _is_exist_clean() const = 0;
   virtual bool _is_exist_mutation_pending() const = 0;
+  virtual bool _is_valid() const = 0;
 protected:
   parent_tracker_ref<ParentT> parent_tracker;
-  virtual bool _is_valid() const = 0;
   virtual bool _is_stable() const = 0;
   template <typename, typename, typename>
   friend class ParentNode;
@@ -350,7 +350,6 @@ public:
       auto &sparent = me.get_stable_for_key(key);
       auto spos = sparent.lower_bound(key).get_offset();
       child = sparent.children[spos];
-      assert(is_valid_child_ptr(child));
       auto ret = etvr.get_extent_viewable_by_trans_sync(
         t, static_cast<ChildT*>(child));
       return ret->template cast<ChildT>();
@@ -441,7 +440,7 @@ public:
   // see Transaction::views
   struct copy_dests_t : trans_spec_view_t {
     std::set<TCachedExtentRef<T>, Comparator> dests_by_key;
-    copy_dests_t(Transaction &t) : trans_spec_view_t{t.get_trans_id()} {}
+    copy_dests_t(Transaction &t) : trans_spec_view_t(t) {}
     ~copy_dests_t() {
       LOG_PREFIX(~copy_dests_t);
       SUBTRACE(seastore_fixedkv_tree, "copy_dests_t destroyed");
@@ -480,8 +479,9 @@ protected:
       auto mut_iter = me.mutation_pending_extents.find(
 	t.get_trans_id(), trans_spec_view_t::cmp_t());
       assert(mut_iter != me.mutation_pending_extents.end());
-      assert(copy_dests_by_trans.find(t.get_trans_id()) ==
-	copy_dests_by_trans.end());
+      assert(copy_dests_by_trans.find(
+        t.get_trans_id(), trans_spec_view_t::cmp_t()) ==
+          copy_dests_by_trans.end());
       return static_cast<T*>(&(*mut_iter));
     }
     ceph_assert(hint == CachedExtent::viewable_state_t::stable_become_retired);
@@ -501,7 +501,8 @@ protected:
   template <typename Func>
   void for_each_copy_dest_set(Transaction &t, Func &&f) {
     for (auto &dests : copy_dests_by_trans) {
-      if (dests.pending_for_transaction == t.get_trans_id()) {
+      assert(dests.t != nullptr);
+      if (dests.t->get_trans_id() == t.get_trans_id()) {
         continue;
       }
       auto &copy_dests = static_cast<copy_dests_t&>(dests);
@@ -515,8 +516,11 @@ protected:
     auto tid = t.get_trans_id();
     auto iter = copy_dests_by_trans.lower_bound(
       tid, trans_spec_view_t::cmp_t());
+    if (iter != copy_dests_by_trans.end()) {
+      assert(iter->t != nullptr);
+    }
     if (iter == copy_dests_by_trans.end() ||
-	iter->pending_for_transaction != tid) {
+	iter->t->get_trans_id() != tid) {
       iter = copy_dests_by_trans.insert_before(
 	iter, t.add_transactional_view<copy_dests_t>(t));
     }
@@ -548,8 +552,9 @@ protected:
     auto &me = down_cast();
     LOG_PREFIX(ParentNode::remove_child_ptr);
     auto raw_children = children.data();
+    assert(me.t != nullptr);
     SUBTRACE(seastore_fixedkv_tree, "trans.{}, pos {}, total size {}, extent {}",
-      me.pending_for_transaction,
+      me.t->get_trans_id(),
       offset,
       me.get_size(),
       (void*)raw_children[offset]);

src/crimson/os/seastore/logical_bucket.cc

@@ -0,0 +1,237 @@
+// -*- mode:C++; tab-width:8; c-basic-offset:2; indent-tabs-mode:t -*-
+// vim: ts=8 sw=2 smarttab
+
+#include "seastar/core/metrics.hh"
+
+#include "crimson/common/coroutine.h"
+#include "crimson/os/seastore/logging.h"
+#include "crimson/os/seastore/logical_bucket.h"
+#include "crimson/os/seastore/transaction_manager.h"
+
+#include <boost/unordered/unordered_flat_map.hpp>
+
+namespace crimson::os::seastore {
+
+SET_SUBSYS(seastore_cache);
+
+struct bucket_t {
+  laddr_t laddr = L_ADDR_NULL;
+  bool demoting = false;
+};
+
+class LogicalBucketCache : public LogicalBucket {
+public:
+  LogicalBucketCache(std::size_t memory_capacity,
+                     std::size_t demote_size_per_cycle)
+    : memory_capacity(memory_capacity),
+      demote_size_per_cycle(demote_size_per_cycle) {
+    LOG_PREFIX(LogicalBucketCache);
+    INFO("init memory_capacity={}, demote_size_per_cycle={}",
+	 memory_capacity, demote_size_per_cycle);
+  }
+
+  void mount(store_index_t store_index) final {
+    register_metrics(store_index);
+  }
+
+  ~LogicalBucketCache() {
+    clear();
+  }
+
+  void move_to_top(laddr_t laddr) final {
+    LOG_PREFIX(LogicalBucketCache::move_to_top);
+    assert(laddr != L_ADDR_NULL);
+    assert(laddr == laddr.get_object_prefix());
+    auto iter = index.find(laddr);
+    if (iter != index.end()) {
+      TRACE("find bucket: {}", iter->first);
+      iter->second->demoting = false;
+      lru.splice(lru.end(), lru, iter->second);
+    } else {
+      TRACE("create bucket: {}", laddr);
+      index[laddr] = lru.emplace(lru.end(), laddr);
+      if (should_demote()) {
+	assert(listener);
+	listener->maybe_wake_background();
+      }
+    }
+  }
+
+  bool is_cached(laddr_t laddr) final {
+    assert(laddr != L_ADDR_NULL);
+    assert(laddr == laddr.get_object_prefix());
+    return index.contains(laddr);
+  }
+
+  void clear() final {
+    index.clear();
+    lru.clear();
+  }
+
+  void set_background_callback(BackgroundListener *l) final {
+    listener = l;
+  }
+
+  void set_extent_callback(ExtentCallbackInterface *cb) final {
+    ecb = cb;
+  }
+
+  bool could_demote() const final {
+    return !lru.empty();
+  }
+
+  bool should_demote() const final {
+    // lru element: laddr_t + pointer * 2
+    // index element: laddr_t + lru iterator(void*)
+    auto element_size = (sizeof(laddr_t) * 2 + sizeof(void*) * 3);
+    return element_size * lru.size() > memory_capacity;
+  }
+
+  using run_demote_iertr = base_iertr;
+  using run_demote_ret = run_demote_iertr::future<>;
+  run_demote_ret run_demote(Transaction &t) {
+    LOG_PREFIX(LogicalBucketCache::run_demote);
+    std::vector<laddr_t> pending_buckets;
+    std::vector<laddr_t> completed_buckets;
+    ceph_assert(pending_buckets_target > 0);
+    ceph_assert(!lru.empty());
+    for (auto &b : lru) {
+      if (pending_buckets.size() == (uint32_t)pending_buckets_target) {
+	break;
+      }
+      b.demoting = true;
+      pending_buckets.push_back(b.laddr);
+    }
+
+    DEBUGT("start demote {} buckets", t, pending_buckets.size());
+    std::size_t demoted_size = 0;
+    std::size_t evicted_size = 0;
+    for (auto &bucket : pending_buckets) {
+      TRACET("start demote {}", t, bucket);
+      auto res = co_await ecb->demote_region(
+	t,
+	bucket,
+	demote_size_per_cycle - demoted_size - evicted_size);
+
+      TRACET("demote_size: {}, evicted_size: {}, complete: {}",
+	     t, res.demoted_size, res.evicted_size, res.complete);
+      demoted_size += res.demoted_size;
+      evicted_size += res.evicted_size;
+      if (res.complete) {
+	completed_buckets.push_back(bucket);
+      }
+      if (demoted_size + evicted_size >= demote_size_per_cycle) {
+	break;
+      }
+    }
+
+    co_await ecb->submit_transaction_direct(t);
+
+    DEBUGT("finish demoting {} buckets with {} bytes evicted and {} bytes demoted",
+	   t, completed_buckets.size(), evicted_size, demoted_size);
+    stat.demoted_bucket_count += completed_buckets.size();
+    stat.demoted_size += demoted_size;
+    stat.evicted_size += evicted_size;
+    for (auto &p : completed_buckets) {
+      maybe_remove(p);
+    }
+    auto old_count = pending_buckets_target;
+    if (demoted_size != 0 && !completed_buckets.empty()) {
+      auto demote_ratio = (double)(demoted_size + evicted_size) /
+	  (double)completed_buckets.size();
+      assert(!std::isnan(demote_ratio));
+      pending_buckets_target = (demote_size_per_cycle / demote_ratio) + 1;
+    }
+    DEBUGT("update init buckets count {} -> {}",
+	   t, old_count, pending_buckets_target);
+    co_return;
+  }
+
+  seastar::future<> demote() final {
+    return repeat_eagain([this] {
+      return ecb->with_transaction_intr(
+        Transaction::src_t::DEMOTE,
+        "demote", cache_hint_t::get_nocache(),
+        [this](auto &t) {
+	  return run_demote(t);
+	});
+    }).handle_error(crimson::ct_error::assert_all("impossible"));
+  }
+
+private:
+  using laddr_lru_t = std::list<bucket_t>;
+  laddr_lru_t lru;
+  boost::unordered_flat_map<laddr_t, laddr_lru_t::iterator> index;
+  int32_t pending_buckets_target = 20;
+
+  void maybe_remove(laddr_t laddr) {
+    LOG_PREFIX(LogicalBucketCache::maybe_remove);
+    TRACE("laddr: {}", laddr);
+    assert(laddr != L_ADDR_NULL);
+    assert(laddr == laddr.get_object_prefix());
+    auto iter = index.find(laddr);
+    if (iter != index.end()) {
+      if (iter->second->demoting) {
+	DEBUG("remove bucket: {}", laddr);
+	lru.erase(iter->second);
+	index.erase(iter);
+      } else {
+	DEBUG("bucket {} not removed, user accesses to the "
+	      "bucket must have happened during the demotion", laddr);
+      }
+    }
+  }
+
+
+  void register_metrics(store_index_t store_index) {
+    namespace sm = seastar::metrics;
+    metrics.add_group(
+      "cache",
+      {
+	sm::make_gauge(
+	  "non_volatile_cache_buckets_count",
+	  [this] { return lru.size(); },
+	  sm::description("the count of laddr bucket used by non volatile cache"),
+	  {sm::label_instance("shard_store_index", std::to_string(store_index))}),
+	sm::make_counter(
+	  "non_volatile_cache_evicted_size",
+	  [this] { return stat.evicted_size; },
+	  sm::description("total bytes of extents evicted by non volatile cache"),
+	  {sm::label_instance("shard_store_index", std::to_string(store_index))}),
+	sm::make_counter(
+	  "non_volatile_cache_demoted_size",
+	  [this] { return stat.demoted_size; },
+	  sm::description("total bytes of extents demoted by non volatile cache"),
+	  {sm::label_instance("shard_store_index", std::to_string(store_index))}),
+	sm::make_counter(
+	  "non_volatile_cache_demoted_bucket_count",
+	  [this] { return stat.demoted_bucket_count; },
+	  sm::description("the count of laddr bucket demoted by non volatile cache"),
+	  {sm::label_instance("shard_store_index", std::to_string(store_index))}),
+      });
+  }
+
+  struct {
+    uint64_t evicted_size = 0;
+    uint64_t demoted_size = 0;
+    uint64_t demoted_bucket_count = 0;
+  } stat;
+
+  seastar::metrics::metric_group metrics;
+
+  const std::size_t memory_capacity = 0;
+  const std::size_t demote_size_per_cycle = 0;
+
+  ExtentCallbackInterface *ecb = nullptr;
+  BackgroundListener *listener = nullptr;
+};
+
+LogicalBucketRef create_logical_bucket(
+  std::size_t memory_capacity,
+  std::size_t demote_size_per_cycle)
+{
+  return std::make_unique<LogicalBucketCache>(
+    memory_capacity, demote_size_per_cycle);
+}
+
+}

src/crimson/os/seastore/logical_bucket.h

@@ -0,0 +1,29 @@
+// -*- mode:C++; tab-width:8; c-basic-offset:2; indent-tabs-mode:t -*-
+// vim: ts=8 sw=2 smarttab expandtab
+
+#pragma once
+
+#include "crimson/common/smp_helpers.h"
+#include "crimson/os/seastore/seastore_types.h"
+
+namespace crimson::os::seastore {
+class BackgroundListener;
+class ExtentCallbackInterface;
+
+struct LogicalBucket {
+  virtual ~LogicalBucket() = default;
+  virtual void move_to_top(laddr_t laddr) = 0;
+  virtual bool is_cached(laddr_t laddr) = 0;
+  virtual void clear() = 0;
+  virtual void set_background_callback(BackgroundListener *listener) = 0;
+  virtual void set_extent_callback(ExtentCallbackInterface *cb) = 0;
+  virtual bool could_demote() const = 0;
+  virtual bool should_demote() const = 0;
+  virtual void mount(store_index_t store_index) = 0;
+  virtual seastar::future<> demote() = 0;
+};
+using LogicalBucketRef = std::unique_ptr<LogicalBucket>;
+LogicalBucketRef create_logical_bucket(
+  std::size_t memory_capacity,
+  std::size_t demote_size_per_cycle);
+}

src/crimson/os/seastore/logical_child_node.h

@@ -28,7 +28,7 @@ public:
   }
 
   virtual ~LogicalChildNode() {
-    if (this->is_stable()) {
+    if (this->is_stable() && !is_shadow_extent()) {
       lba_child_node_t::destroy();
     }
   }
@@ -44,6 +44,20 @@ public:
   laddr_t get_end() const {
     return (get_laddr() + get_length()).checked_to_laddr();
   }
+
+  TCachedExtentRef<LogicalChildNode> get_shadow() const {
+    return shadow;
+  }
+
+  void set_shadow(TCachedExtentRef<LogicalChildNode> &s) {
+    assert(!shadow);
+    shadow = s;
+  }
+  
+  void reset_shadow() {
+    shadow.reset();
+  }
+
 protected:
   void on_replace_prior(Transaction &t) final {
     assert(is_seen_by_users());
@@ -56,6 +70,8 @@ protected:
   void on_data_commit() final {
     ceph_abort("impossible");
   }
+private:
+  TCachedExtentRef<LogicalChildNode> shadow;
 };
 using LogicalChildNodeRef = TCachedExtentRef<LogicalChildNode>;
 } // namespace crimson::os::seastore

src/crimson/os/seastore/object_data_handler.cc

@@ -1578,6 +1578,7 @@ ObjectDataHandler::read_ret ObjectDataHandler::read(
       read_len,
       pin_start,
       pin_len);
+
     rpins.emplace_back(
       pin, read_start_aligned, read_len_aligned,
       unalign_start_offset, read_len);
@@ -1590,6 +1591,8 @@ ObjectDataHandler::read_ret ObjectDataHandler::read(
       "ObjectDataHandler::read hit invalid error"
     )
   );
+  auto prefix = l_start.get_laddr().get_object_prefix();
+  bool all_cold = true;
   for (auto &pin : rpins) {
     if (pin.mapping.is_zero_reserved()) {
       ret.append_zero(pin.unaligned_len);
@@ -1608,7 +1611,20 @@ ObjectDataHandler::read_ret ObjectDataHandler::read(
       assert(pin.unaligned_start_offset == 0);
       ret.append(std::move(aligned_bl));
     }
+    DEBUGT("got extent: {}", ctx.t, *maybe_indirect_extent.extent);
+    auto paddr = maybe_indirect_extent.extent->get_paddr();
+    all_cold &= ctx.tm.is_cold_device(paddr.get_device_id());
+
+    if (paddr.is_absolute()) {
+      ctx.tm.update_read_ratio(ctx.t, paddr.get_device_id());
+    }
+
   }
+  if (!all_cold) {
+    assert(ctx.tm.is_prefix_cached(prefix));
+    ctx.tm.update_logical_bucket_for_read(prefix);
+  }
+  ctx.tm.submit_read_ratio(ctx.t);
   co_return std::move(ret);
 }
 

src/crimson/os/seastore/random_block_manager.cc

@@ -5,17 +5,25 @@
 #include "crimson/os/seastore/random_block_manager.h"
 #include "crimson/os/seastore/random_block_manager/nvme_block_device.h"
 #include "crimson/os/seastore/random_block_manager/rbm_device.h"
+#include "crimson/os/seastore/random_block_manager/hdd_device.h"
 
 namespace crimson::os::seastore {
 
 seastar::future<random_block_device::RBMDeviceRef>
 get_rb_device(
-  const std::string &device)
+  const std::string &device, device_type_t dtype)
 {
-  return seastar::make_ready_future<random_block_device::RBMDeviceRef>(
-    std::make_unique<
-      random_block_device::nvme::NVMeBlockDevice
-    >(device + "/block"));
+  if (dtype == device_type_t::RANDOM_BLOCK_HDD) {
+    return seastar::make_ready_future<random_block_device::RBMDeviceRef>(
+      std::make_unique<
+        random_block_device::RotationalDevice
+      >(device + "/block"));
+  } else {
+    return seastar::make_ready_future<random_block_device::RBMDeviceRef>(
+      std::make_unique<
+        random_block_device::nvme::NVMeBlockDevice
+      >(device + "/block"));
+  }
 }
 
 }

src/crimson/os/seastore/random_block_manager.h

@@ -77,7 +77,7 @@ public:
 
   using allocate_ret_bare = std::list<alloc_paddr_result>;
   using allo_extents_ret = allocate_ertr::future<allocate_ret_bare>;
-  virtual allocate_ret_bare alloc_extents(size_t size) = 0;
+  virtual allocate_ret_bare alloc_extents(size_t size, paddr_t hint) = 0;
 
   virtual void mark_space_used(paddr_t paddr, size_t len) = 0;
   virtual void mark_space_free(paddr_t paddr, size_t len) = 0;
@@ -114,7 +114,7 @@ namespace random_block_device {
 }
 
 seastar::future<std::unique_ptr<random_block_device::RBMDevice>> 
-  get_rb_device(const std::string &device);
+  get_rb_device(const std::string &device, device_type_t dtype);
 
 std::ostream &operator<<(std::ostream &out, const rbm_extent_state_t &state);
 }

src/crimson/os/seastore/random_block_manager/avlallocator.cc

@@ -99,6 +99,15 @@ extent_len_t AvlAllocator::find_block(
   if (p != extent_size_tree.rend()) {
     max_size = p->end - p->start;
   }
+  const auto compare = extent_tree.key_comp();
+  auto rs = extent_tree.lower_bound(extent_range_t{start, size}, compare);
+  if (rs != extent_tree.end()) {
+    uint64_t offset = rs->start;
+    if (offset + size <= rs->end) {
+      start = offset;
+      return size;
+    }
+  }
 
   assert(max_size);
   if (max_size <= size) {
@@ -210,7 +219,7 @@ std::optional<interval_set<rbm_abs_addr>> AvlAllocator::alloc_extent(
 }
 
 std::optional<interval_set<rbm_abs_addr>> AvlAllocator::alloc_extents(
-  size_t size)
+  size_t size, rbm_abs_addr hint)
 {
   LOG_PREFIX(AvlAllocator::alloc_extents);
   if (available_size < size) {
@@ -224,10 +233,10 @@ std::optional<interval_set<rbm_abs_addr>> AvlAllocator::alloc_extents(
 
   interval_set<rbm_abs_addr> result;
 
-  auto try_to_alloc_block = [this, &result, FNAME] (uint64_t alloc_size)
+  auto try_to_alloc_block = [this, hint, &result, FNAME] (uint64_t alloc_size)
   {
+    rbm_abs_addr start = hint;
     while (alloc_size) {
-      rbm_abs_addr start = 0;
       extent_len_t len = find_block(std::min(max_alloc_size, alloc_size), start);
       ceph_assert(len);
       _remove_from_tree(start, len);

src/crimson/os/seastore/random_block_manager/avlallocator.h

@@ -65,7 +65,7 @@ public:
   std::optional<interval_set<rbm_abs_addr>> alloc_extent(
     size_t size) final;
   std::optional<interval_set<rbm_abs_addr>> alloc_extents(
-    size_t size) final;
+    size_t size, rbm_abs_addr hint) final;
 
   void free_extent(rbm_abs_addr addr, size_t size) final;
   void mark_extent_used(rbm_abs_addr addr, size_t size) final;

src/crimson/os/seastore/random_block_manager/block_rb_manager.cc

@@ -31,7 +31,7 @@ device_config_t get_rbm_ephemeral_device_config(
          ++secondary_index) {
       device_id_t secondary_id = static_cast<device_id_t>(secondary_index);
       secondary_devices.insert({
-        secondary_index, device_spec_t{magic, type, secondary_id}
+        secondary_index, device_spec_t{magic, type, backend_type_t::RANDOM_BLOCK, secondary_id}
       });
     }
   } else { // index > 0
@@ -41,7 +41,7 @@ device_config_t get_rbm_ephemeral_device_config(
   device_id_t id = static_cast<device_id_t>(DEVICE_ID_RANDOM_BLOCK_MIN + index);
   seastore_meta_t meta = {};
   return {is_major_device,
-          device_spec_t{magic, type, id},
+          device_spec_t{magic, type, backend_type_t::RANDOM_BLOCK, id},
           meta,
           secondary_devices};
 }
@@ -66,11 +66,13 @@ paddr_t BlockRBManager::alloc_extent(size_t size)
 }
 
 BlockRBManager::allocate_ret_bare
-BlockRBManager::alloc_extents(size_t size)
+BlockRBManager::alloc_extents(size_t size, paddr_t hint)
 {
   LOG_PREFIX(BlockRBManager::alloc_extents);
   assert(allocator);
-  auto alloc = allocator->alloc_extents(size);
+  rbm_abs_addr rbm_hint =
+    (hint == P_ADDR_NULL ? 0 : convert_paddr_to_abs_addr(hint));
+  auto alloc = allocator->alloc_extents(size, rbm_hint);
   if (!alloc) {
     return {};
   }

src/crimson/os/seastore/random_block_manager/block_rb_manager.h

@@ -58,7 +58,8 @@ public:
    */
   paddr_t alloc_extent(size_t size) override; // allocator, return blocks
 
-  allocate_ret_bare alloc_extents(size_t size) override; // allocator, return blocks
+  allocate_ret_bare alloc_extents(
+    size_t size, paddr_t hint) override; // allocator, return blocks
 
   void complete_allocation(paddr_t addr, size_t size) override;
 

src/crimson/os/seastore/random_block_manager/extent_allocator.h

@@ -35,7 +35,7 @@ public:
    *
    */
   virtual std::optional<interval_set<rbm_abs_addr>> alloc_extents(
-    size_t size) = 0;
+    size_t size, rbm_abs_addr hint) = 0;
 
   /**
    * free_extent

src/crimson/os/seastore/random_block_manager/hdd_device.cc

@@ -0,0 +1,156 @@
+// -*- mode:C++; tab-width:8; c-basic-offset:2; indent-tabs-mode:t -*-
+// vim: ts=8 sw=2 smarttab
+
+#include "crimson/common/errorator-utils.h"
+#include "crimson/os/seastore/logging.h"
+#include "crimson/os/seastore/random_block_manager/hdd_device.h"
+
+SET_SUBSYS(seastore_device);
+
+namespace crimson::os::seastore::random_block_device {
+
+seastar::future<> RotationalDevice::start(uint32_t shard_nums) {
+  device_shard_nums = shard_nums;
+  auto num_shard_services =
+    (device_shard_nums + seastar::this_smp_shard_count() - 1 ) /
+      seastar::this_smp_shard_count();
+  LOG_PREFIX(NVMeBlockDevice::start);
+  DEBUG("device_shard_nums={} seastar::smp={}, num_shard_services={}",
+    device_shard_nums, seastar::this_smp_shard_count(), num_shard_services);
+  return shard_devices.start(num_shard_services, device_path);
+}
+
+RotationalDevice::mkfs_ret RotationalDevice::mkfs(device_config_t config) {
+  LOG_PREFIX(RotationalDevice::mkfs);
+  INFO("{}", config);
+  return shard_devices.local().mshard_devices[0]->do_primary_mkfs(
+    config, seastar::this_smp_shard_count(), 0);
+}
+
+RotationalDevice::mount_ret RotationalDevice::mount() {
+  LOG_PREFIX(RotationalDevice::mount);
+  DEBUG("mount");
+  return shard_devices.invoke_on_all([](auto &local_device) {
+    return seastar::do_for_each(
+      local_device.mshard_devices,
+      [](auto &mshard_device) {
+      return mshard_device->do_shard_mount(
+      ).handle_error(
+	crimson::ct_error::assert_all(
+	  "Invalid error in RBMDevice::do_mount"
+	)
+      );
+    });
+  });
+}
+
+read_ertr::future<> RotationalDevice::read(
+  uint64_t offset,
+  bufferptr &bptr)
+{
+  auto length = bptr.length();
+  return device.dma_read(offset, bptr.c_str(), length
+  ).handle_exception([](auto e) -> read_ertr::future<size_t> {
+    return crimson::ct_error::input_output_error::make();
+  }).then([length](auto result) -> read_ertr::future<> {
+    if (result != length) {
+      return crimson::ct_error::input_output_error::make();
+    }
+    return read_ertr::now();
+  });
+}
+
+read_ertr::future<> RotationalDevice::_readv(
+  uint64_t offset,
+  std::vector<bufferptr> ptrs) {
+  LOG_PREFIX(NVMeBlockDevice::_readv);
+  DEBUG("block: read offset {}, {} buffers", offset, ptrs.size());
+  if (ptrs.size() == 0) {
+    return read_ertr::now();
+  }
+
+  std::vector<iovec> iov;
+  size_t length = 0;
+  for (auto &ptr : ptrs) {
+    length += ptr.length();
+    assert((ptr.length() % super.block_size) == 0);
+    iov.emplace_back(ptr.c_str(), ptr.length());
+  }
+  return device.dma_read(offset, std::move(iov)
+  ).handle_exception(
+    [FNAME](auto e) -> read_ertr::future<size_t> {
+    ERROR("read: dma_read got error{}", e);
+    return crimson::ct_error::input_output_error::make();
+  }).then([length, FNAME](auto result) -> read_ertr::future<> {
+    if (result != length) {
+      ERROR("read: dma_read got error with not proper length");
+      return crimson::ct_error::input_output_error::make();
+    }
+    return read_ertr::now();
+  });
+}
+
+write_ertr::future<> RotationalDevice::write(
+  uint64_t offset,
+  bufferptr bptr,
+  uint16_t stream)
+{
+  auto length = bptr.length();
+  return seastar::do_with(std::move(bptr), [this, offset, length](auto &bptr) {
+    return device.dma_write(offset, bptr.c_str(), length
+    ).handle_exception([](auto e) -> write_ertr::future<size_t> {
+      return crimson::ct_error::input_output_error::make();
+    }).then([length](auto result) -> write_ertr::future<> {
+      if (result != length) {
+        return crimson::ct_error::input_output_error::make();
+      }
+      return write_ertr::now();
+    });
+  });
+}
+
+open_ertr::future<> RotationalDevice::open(
+  const std::string &path,
+  seastar::open_flags mode)
+{
+  return seastar::open_file_dma(path, mode).then([this](auto file) {
+    device = std::move(file);
+  }).handle_exception([](auto e) -> open_ertr::future<> {
+    return crimson::ct_error::input_output_error::make();
+  });
+}
+
+write_ertr::future<> RotationalDevice::writev(
+  uint64_t offset,
+  ceph::bufferlist bl,
+  uint16_t stream) {
+  bl.rebuild_aligned(super.block_size);
+
+  return seastar::do_with(
+    bl.prepare_iovs(),
+    std::move(bl),
+    [this, offset](auto& iovs, auto& bl)
+  {
+    return write_ertr::parallel_for_each(
+      iovs,
+      [this, offset](auto& p) mutable
+    {
+      auto off = offset + p.offset;
+      auto len = p.length;
+      auto& iov = p.iov;
+      return device.dma_write(off, std::move(iov)
+      ).handle_exception(
+        [](auto e) -> write_ertr::future<size_t>
+      {
+        return crimson::ct_error::input_output_error::make();
+      }).then([len](size_t written) -> write_ertr::future<> {
+        if (written != len) {
+          return crimson::ct_error::input_output_error::make();
+        }
+        return write_ertr::now();
+      });
+    });
+  });
+}
+
+}

src/crimson/os/seastore/random_block_manager/hdd_device.h

@@ -0,0 +1,100 @@
+// -*- mode:C++; tab-width:8; c-basic-offset:2; indent-tabs-mode:t -*-
+// vim: ts=8 sw=2 smarttab
+
+#pragma once
+
+#include "crimson/os/seastore/random_block_manager/rbm_device.h"
+
+namespace crimson::os::seastore::random_block_device {
+class RotationalDevice : public RBMDevice {
+public:
+  RotationalDevice(
+    std::string device_path,
+    store_index_t store_index = 0)
+    : RBMDevice(store_index),
+      device_path(device_path)
+  {}
+  ~RotationalDevice() = default;
+
+  /// Device interface
+
+  seastar::future<> start(uint32_t shard_nums) final;
+
+  seastar::future<> stop() final {
+    return shard_devices.stop();
+  }
+
+  Device& get_sharded_device(store_index_t store_index) final {
+    assert(store_index < shard_devices.local().mshard_devices.size());
+    return *shard_devices.local().mshard_devices[store_index];
+  }
+
+  mkfs_ret mkfs(device_config_t config) final;
+
+  mount_ret mount() final;
+
+  device_type_t get_device_type() const final {
+    return device_type_t::RANDOM_BLOCK_HDD;
+  }
+
+  close_ertr::future<> close() final {
+    return device.close();
+  }
+
+  /// RBMDevice interface
+
+  read_ertr::future<> read(
+    uint64_t offset,
+    bufferptr &bptr) final;
+  read_ertr::future<> _readv(
+    uint64_t offset,
+    std::vector<bufferptr> ptrs) final;
+
+  write_ertr::future<> write(
+    uint64_t offset,
+    bufferptr bptr,
+    uint16_t stream = 0) final;
+
+  open_ertr::future<> open(
+    const std::string &path,
+    seastar::open_flags mode) final;
+
+  write_ertr::future<> writev(
+    uint64_t offset,
+    ceph::bufferlist bl,
+    uint16_t stream = 0) final;
+
+  stat_device_ret stat_device() final {
+    return seastar::file_stat(device_path, seastar::follow_symlink::yes
+    ).then([this](auto stat) {
+      return seastar::open_file_dma(
+        device_path,
+	seastar::open_flags::rw | seastar::open_flags::dsync
+      ).then([stat](auto file) mutable {
+	return seastar::do_with(std::move(file), [stat](auto &file) mutable {
+	  return file.size().then([stat](auto size) mutable {
+	    stat.size = size;
+	    return stat;
+	  }).then([file](auto stat) mutable {
+	    return file.close().then([stat] {
+	      return stat;
+	    });
+	  });
+	});
+      });
+    }).handle_exception([](auto e) -> stat_device_ret {
+      return crimson::ct_error::input_output_error::make();
+    });
+  }
+
+  std::string get_device_path() const final {
+    return device_path;
+  }
+
+private:
+  std::string device_path;
+  seastar::file device;
+  seastar::sharded<MultiShardDevices<RotationalDevice>> shard_devices;
+};
+
+}

src/crimson/os/seastore/random_block_manager/nvme_block_device.h

@@ -374,25 +374,7 @@ private:
   int namespace_id; // TODO: multi namespaces
   std::string device_path;
 
-  class MultiShardDevices {
-    public:
-      std::vector<std::unique_ptr<NVMeBlockDevice>> mshard_devices;
-
-    public:
-    MultiShardDevices(size_t count,
-                      const std::string path)
-    : mshard_devices() {
-      mshard_devices.reserve(count);
-      for (size_t store_index = 0; store_index < count; ++store_index) {
-        mshard_devices.emplace_back(std::make_unique<NVMeBlockDevice>(
-          path, store_index));
-      }
-    }
-    ~MultiShardDevices() {
-     mshard_devices.clear();
-    }
-  };
-  seastar::sharded<MultiShardDevices> shard_devices;
+  seastar::sharded<MultiShardDevices<NVMeBlockDevice>> shard_devices;
 };
 
 }

src/crimson/os/seastore/random_block_manager/rbm_device.cc

@@ -29,7 +29,6 @@ RBMDevice::mkfs_ret RBMDevice::do_primary_mkfs(device_config_t config,
     maybe_create = check_create_device(get_device_path(), size);
   }
 
-
   co_await std::move(maybe_create);
   auto st = co_await stat_device(
   ).safe_then([] (auto st) mutable {
@@ -44,9 +43,11 @@ RBMDevice::mkfs_ret RBMDevice::do_primary_mkfs(device_config_t config,
     );
   }
 
+  config.spec.id |= 0x80;
   const size_t cur_block_size = (*st).block_size;
   const size_t cur_total_size = (*st).size;
-  ceph_assert_always(journal_size > 0);
+  ceph_assert_always(journal_size > 0 ||
+                     config.spec.dtype == device_type_t::RANDOM_BLOCK_HDD);
   ceph_assert_always(cur_total_size >= journal_size);
   ceph_assert_always(shard_num > 0);
 

src/crimson/os/seastore/random_block_manager/rbm_device.h

@@ -111,7 +111,7 @@ public:
     return super.config.spec.magic;
   }
 
-  device_type_t get_device_type() const final {
+  virtual device_type_t get_device_type() const {
     return device_type_t::RANDOM_BLOCK_SSD;
   }
 
@@ -278,4 +278,24 @@ EphemeralRBMDeviceRef create_test_ephemeral(
   uint64_t journal_size = DEFAULT_TEST_CBJOURNAL_SIZE,
   uint64_t data_size = DEFAULT_TEST_CBJOURNAL_SIZE);
 
+template <typename T>
+class MultiShardDevices {
+  public:
+    std::vector<std::unique_ptr<T>> mshard_devices;
+
+  public:
+  MultiShardDevices(size_t count,
+                    const std::string path)
+  : mshard_devices() {
+    mshard_devices.reserve(count);
+    for (size_t store_index = 0; store_index < count; ++store_index) {
+      mshard_devices.emplace_back(std::make_unique<T>(
+        path, store_index));
+    }
+  }
+  ~MultiShardDevices() {
+    mshard_devices.clear();
+  }
+};
+
 }

src/crimson/os/seastore/seastore.cc

@@ -388,13 +388,16 @@ seastar::future<uint32_t> SeaStore::start()
   bool is_test = false;
 #endif
   using crimson::common::get_conf;
-  std::string type = get_conf<std::string>("seastore_main_device_type");
+  std::string type = get_conf<std::string>("seastore_hot_device_type");
   device_type_t d_type = string_to_device_type(type);
   assert(d_type == device_type_t::SSD ||
          d_type == device_type_t::RANDOM_BLOCK_SSD);
 
+  type = get_conf<std::string>("seastore_hot_backend_type");
+  auto b_type = string_to_backend_type(type);
+  INFO("main device type: {}, main backend type: {}", d_type, b_type);
   ceph_assert(root != "");
-  DeviceRef device_obj = co_await Device::make_device(root, d_type);
+  DeviceRef device_obj = co_await Device::make_device(root, d_type, b_type);
   device = std::move(device_obj);
   co_await get_shard_nums();
   co_await device->start(store_shard_nums);
@@ -467,8 +470,11 @@ Device::access_ertr::future<> SeaStore::_mount()
     device_id_t id = device_entry.first;
     [[maybe_unused]] magic_t magic = device_entry.second.magic;
     device_type_t dtype = device_entry.second.dtype;
-    std::string path = fmt::format("{}/block.{}.{}", root, dtype, std::to_string(id));
-    DeviceRef sec_dev = co_await Device::make_device(path, dtype);
+    backend_type_t btype = device_entry.second.btype;
+    auto btype_conf_str = get_conf<std::string>("seastore_cold_backend_type");
+    ceph_assert(string_to_backend_type(btype_conf_str) == btype);
+    std::string path = fmt::format("{}/block.{}", root, std::to_string(id));
+    DeviceRef sec_dev = co_await Device::make_device(path, dtype, btype);
     co_await sec_dev->start(store_shard_nums);
     co_await sec_dev->mount();
     auto sec_block_size = sec_dev->get_sharded_device(0).get_block_size();
@@ -662,6 +668,19 @@ seastar::future<> SeaStore::prepare_meta(uuid_d new_osd_fsid)
   co_await write_meta("mkfs_done", "yes");
 }
 
+std::optional<device_id_t> parse_device_id(const seastar::sstring &name) {
+  auto prefix_len = sizeof("block.") - 1;
+  if (name.starts_with("block.") && name.length() > prefix_len) {
+    int id = 0;
+    std::string id_str = name.substr(prefix_len);
+    std::istringstream iss(id_str);
+    iss >> id;
+    assert(id < std::numeric_limits<uint8_t>::max());
+    return std::make_optional<device_id_t>(id);
+  }
+  return std::nullopt;
+}
+
 Device::access_ertr::future<> SeaStore::_mkfs(uuid_d new_osd_fsid)
 {
   LOG_PREFIX(SeaStore::_mkfs);
@@ -674,6 +693,12 @@ Device::access_ertr::future<> SeaStore::_mkfs(uuid_d new_osd_fsid)
     co_return;
   }
   DEBUG("mkfs_done does not exist, starting mkfs");
+  auto dtype_str = get_conf<std::string>("seastore_cold_device_type");
+  auto dtype = string_to_device_type(dtype_str);
+  auto btype_str = get_conf<std::string>("seastore_cold_backend_type");
+  auto btype = string_to_backend_type(btype_str);
+  ceph_assert(!root.empty());
+  INFO("secondary device type: {}, secondary backend type: {}", dtype, btype);
   secondary_device_set_t sds;
   if (!root.empty()) {
     seastar::file rdir = co_await seastar::open_directory(root);
@@ -682,43 +707,57 @@ Device::access_ertr::future<> SeaStore::_mkfs(uuid_d new_osd_fsid)
     while (auto de = co_await lister()) {
       auto& entry = *de;
       DEBUG("found file: {}", entry.name);
-      if (entry.name.find("block.") == 0 && entry.name.length() > 6 ) {
-      // 6 for "block."
-        std::string entry_name = entry.name;
-        auto dtype_end = entry_name.find_first_of('.', 6);
-        device_type_t dtype =
-          string_to_device_type(
-            entry_name.substr(6, dtype_end - 6));
-        if (dtype == device_type_t::NONE) {
-          // invalid device type
-          co_return;
-        }
-        auto id = std::stoi(entry_name.substr(dtype_end + 1));
-        std::string path = fmt::format("{}/{}", root, entry_name);
-        DeviceRef sec_dev = co_await Device::make_device(path, dtype);
-        auto p_sec_dev = sec_dev.get();
-        secondaries.emplace_back(std::move(sec_dev));
-        co_await p_sec_dev->start(store_shard_nums);
-        magic_t magic = (magic_t)std::rand();
-        sds.emplace((device_id_t)id, device_spec_t{magic, dtype, (device_id_t)id});
-        co_await p_sec_dev->mkfs(
-          device_config_t::create_secondary(new_osd_fsid, id, dtype, magic)
-          ).handle_error(crimson::ct_error::assert_all("not possible"));
-        co_await set_secondaries();
+      auto p = parse_device_id(entry.name);
+      if (!p) {
+        continue;
       }
+      std::string path = fmt::format("{}/{}", root, entry.name);
+      DeviceRef sec_dev = co_await Device::make_device(path, dtype, btype);
+      auto p_sec_dev = sec_dev.get();
+      secondaries.emplace_back(std::move(sec_dev));
+      co_await p_sec_dev->start(store_shard_nums);
+      magic_t magic = (magic_t)std::rand();
+      auto id = *p;
+      sds.emplace((device_id_t)id,
+                  device_spec_t{magic, dtype, btype, (device_id_t)id});
+      co_await p_sec_dev->mkfs(
+        device_config_t::create_secondary(new_osd_fsid, id, dtype, btype, magic)
+      ).handle_error(crimson::ct_error::assert_all("not possible"));
+      co_await set_secondaries();
     }
     co_await rdir.close();
   }
 
+  if (sds.empty() && crimson::common::get_conf<bool>(
+        "seastore_logical_bucket_cache_test_stress")) {
+    // lbc test workload enabled while no secondary devices indicated, create one
+    std::string path = fmt::format("{}/block.1", root);
+    co_await seastar::make_directory(path);
+    DeviceRef sec_dev = co_await Device::make_device(path, dtype, btype);
+    auto p_sec_dev = sec_dev.get();
+    secondaries.emplace_back(std::move(sec_dev));
+    co_await p_sec_dev->start(store_shard_nums);
+    magic_t magic = (magic_t)std::rand();
+    device_id_t id = 0x1;
+    sds.emplace(id, device_spec_t{magic, dtype, btype, id});
+    co_await p_sec_dev->mkfs(
+      device_config_t::create_secondary(new_osd_fsid, id, dtype, btype, magic)
+    ).handle_error(crimson::ct_error::assert_all("not possible"));
+    co_await set_secondaries();
+  }
   device_id_t id = 0;
   device_type_t d_type = device->get_device_type();
+  backend_type_t b_type = device->get_backend_type();
   assert(d_type == device_type_t::SSD ||
       d_type == device_type_t::RANDOM_BLOCK_SSD);
+  assert(b_type != backend_type_t::NONE);
   if (d_type == device_type_t::RANDOM_BLOCK_SSD) {
       id = static_cast<device_id_t>(DEVICE_ID_RANDOM_BLOCK_MIN);
   }
   DEBUG("creating primary device");
-  co_await device->mkfs(device_config_t::create_primary(new_osd_fsid, id, d_type, sds));
+  co_await device->mkfs(
+    device_config_t::create_primary(
+      new_osd_fsid, id, d_type, b_type, sds));
   DEBUG("mounting {} secondaries", secondaries.size());
   for (auto& sec_dev : secondaries) {
       co_await sec_dev->mount();
@@ -3007,7 +3046,7 @@ SeaStore::read_meta(const std::string& key)
 seastar::future<std::string> SeaStore::get_default_device_class()
 {
   using crimson::common::get_conf;
-  std::string type = get_conf<std::string>("seastore_main_device_type");
+  std::string type = get_conf<std::string>("seastore_hot_device_type");
   return seastar::make_ready_future<std::string>(type);
 }
 

src/crimson/os/seastore/seastore.h

@@ -207,6 +207,10 @@ public:
       return 256;
     }
 
+    bool is_storage_full() const override final {
+      return transaction_manager ? transaction_manager->is_storage_full() : false;
+    }
+
     omap_root_t select_log_omap_root(Onode& onode) const;
 
   // only exposed to SeaStore

src/crimson/os/seastore/seastore_types.cc

@@ -816,6 +816,10 @@ std::ostream &operator<<(std::ostream &os, transaction_type_t type)
     return os << "CLEANER_MAIN";
   case transaction_type_t::CLEANER_COLD:
     return os << "CLEANER_COLD";
+  case transaction_type_t::PROMOTE:
+    return os << "PROMOTE";
+  case transaction_type_t::DEMOTE:
+    return os << "DEMOTE";
   case transaction_type_t::MAX:
     return os << "TRANS_TYPE_NULL";
   default:
@@ -1220,6 +1224,16 @@ std::ostream& operator<<(std::ostream& out, placement_hint_t h)
   }
 }
 
+std::ostream& operator<<(std::ostream& out, write_policy_t w)
+{
+  switch(w) {
+  case write_policy_t::WRITE_BACK:
+    return out << "WRITE_BACK";
+  case write_policy_t::WRITE_THROUGH:
+    return out << "WRITE_THROUGH";
+  }
+}
+
 bool can_delay_allocation(device_type_t type) {
   // Some types of device may not support delayed allocation, for example PMEM.
   // All types of device currently support delayed allocation.
@@ -1239,6 +1253,9 @@ device_type_t string_to_device_type(std::string type) {
   if (type == "RANDOM_BLOCK_SSD") {
     return device_type_t::RANDOM_BLOCK_SSD;
   }
+  if (type == "RANDOM_BLOCK_HDD") {
+    return device_type_t::RANDOM_BLOCK_HDD;
+  }
   return device_type_t::NONE;
 }
 
@@ -1261,15 +1278,31 @@ std::ostream& operator<<(std::ostream& out, device_type_t t)
     return out << "RANDOM_BLOCK_SSD";
   case device_type_t::RANDOM_BLOCK_EPHEMERAL:
     return out << "RANDOM_BLOCK_EPHEMERAL";
+  case device_type_t::RANDOM_BLOCK_HDD:
+    return out << "RANDOM_BLOCK_HDD";
   default:
     return out << "INVALID_DEVICE_TYPE!";
   }
 }
 
-std::ostream& operator<<(std::ostream& out, backend_type_t btype) {
-  if (btype == backend_type_t::SEGMENTED) {
-    return out << "SEGMENTED";
+backend_type_t string_to_backend_type(const std::string &str) {
+  if (str == "SEGMENTED") {
+    return backend_type_t::SEGMENTED;
+  } else if (str == "RANDOM_BLOCK") {
+    return backend_type_t::RANDOM_BLOCK;
   } else {
+    ceph_abort("backend str not valid");
+    return backend_type_t::SEGMENTED;
+  }
+}
+
+std::ostream& operator<<(std::ostream& out, backend_type_t btype) {
+  switch (btype) {
+  case backend_type_t::NONE:
+    return out << "NONE";
+  case backend_type_t::SEGMENTED:
+    return out << "SEGMENTED";
+  case backend_type_t::RANDOM_BLOCK:
     return out << "RANDOM_BLOCK";
   }
 }

src/crimson/os/seastore/seastore_types.h

@@ -966,6 +966,7 @@ enum class device_type_t : uint8_t {
   EPHEMERAL_MAIN,
   RANDOM_BLOCK_SSD,
   RANDOM_BLOCK_EPHEMERAL,
+  RANDOM_BLOCK_HDD,
   NUM_TYPES
 };
 
@@ -974,7 +975,8 @@ std::ostream& operator<<(std::ostream& out, device_type_t t);
 bool can_delay_allocation(device_type_t type);
 device_type_t string_to_device_type(std::string type);
 
-enum class backend_type_t {
+enum class backend_type_t : uint8_t {
+  NONE,
   SEGMENTED,    // SegmentManager: SSD, ZBD, HDD
   RANDOM_BLOCK  // RBMDevice:      RANDOM_BLOCK_SSD
 };
@@ -983,7 +985,7 @@ std::ostream& operator<<(std::ostream& out, backend_type_t);
 
 constexpr backend_type_t get_default_backend_of_device(device_type_t dtype) {
   assert(dtype != device_type_t::NONE &&
-	 dtype != device_type_t::NUM_TYPES);
+        dtype != device_type_t::NUM_TYPES);
   if (dtype >= device_type_t::HDD &&
       dtype <= device_type_t::EPHEMERAL_MAIN) {
     return backend_type_t::SEGMENTED;
@@ -992,6 +994,8 @@ constexpr backend_type_t get_default_backend_of_device(device_type_t dtype) {
   }
 }
 
+backend_type_t string_to_backend_type(const std::string &str);
+
 /**
  * Monotonically increasing identifier for the location of a
  * journal_record.
@@ -2066,6 +2070,13 @@ constexpr bool is_real_type(extent_types_t type) {
 
 std::ostream &operator<<(std::ostream &out, extent_types_t t);
 
+enum class write_policy_t {
+  WRITE_BACK,
+  WRITE_THROUGH
+};
+
+std::ostream& operator<<(std::ostream& out, write_policy_t w);
+
 /**
  * rewrite_gen_t
  *
@@ -2718,6 +2729,8 @@ enum class transaction_type_t : uint8_t {
   TRIM_ALLOC,
   CLEANER_MAIN,
   CLEANER_COLD,
+  PROMOTE,
+  DEMOTE,
   MAX
 };
 
@@ -2744,7 +2757,9 @@ constexpr bool is_background_transaction(transaction_type_t type) {
 constexpr bool is_rewrite_transaction(transaction_type_t type) {
   return type == transaction_type_t::TRIM_DIRTY ||
     type == transaction_type_t::CLEANER_MAIN ||
-    type == transaction_type_t::CLEANER_COLD;
+    type == transaction_type_t::CLEANER_COLD ||
+    type == transaction_type_t::DEMOTE ||
+    type == transaction_type_t::PROMOTE;
 }
 
 constexpr bool is_trim_transaction(transaction_type_t type) {
@@ -3648,6 +3663,7 @@ template <> struct fmt::formatter<crimson::os::seastore::transaction_type_t> : f
 template <> struct fmt::formatter<crimson::os::seastore::write_result_t> : fmt::ostream_formatter {};
 template <> struct fmt::formatter<crimson::os::seastore::omap_type_t> : fmt::ostream_formatter {};
 template <> struct fmt::formatter<ceph::buffer::list> : fmt::ostream_formatter {};
+template <> struct fmt::formatter<crimson::os::seastore::write_policy_t> : fmt::ostream_formatter {};
 #endif
 
 namespace fmt {

src/crimson/os/seastore/segment_manager/ephemeral.cc

@@ -59,6 +59,7 @@ device_config_t get_ephemeral_device_config(
 	device_spec_t{
 	  magic,
 	  get_sec_dtype(secondary_index),
+	  backend_type_t::SEGMENTED,
 	  secondary_id
 	}
       });
@@ -73,6 +74,7 @@ device_config_t get_ephemeral_device_config(
           device_spec_t{
             magic,
 	    get_sec_dtype(index),
+	    backend_type_t::SEGMENTED,
             id
           },
           meta,

src/crimson/os/seastore/transaction.h

@@ -162,7 +162,7 @@ public:
       ref->set_invalid(*this);
       write_set.erase(*ref);
       assert(ref->prior_instance);
-      retired_set.emplace(ref->prior_instance, trans_id);
+      retired_set.emplace(ref->prior_instance, *this);
       assert(read_set.count(ref->prior_instance->get_paddr(), extent_cmp_t{}));
       ref->reset_prior_instance();
     } else {
@@ -171,7 +171,7 @@ public:
       // XXX: prevent double retire -- retired_set.count(ref->get_paddr()) == 0
       // If it's already in the set, insert here will be a noop,
       // which is what we want.
-      retired_set.emplace(ref, trans_id);
+      retired_set.emplace(ref, *this);
     }
   }
 
@@ -364,6 +364,21 @@ public:
     }
   }
 
+  bool remove_from_retired_set(CachedExtent &ext) {
+    auto it = retired_set.find(ext.get_paddr());
+    if (it == retired_set.end()) {
+      return false;
+    }
+    auto &extent = it->extent;
+    if (extent->get_paddr() != ext.get_paddr()) {
+      return false;
+    } else {
+      assert(ext.get_length() == extent->get_length());
+      retired_set.erase(it);
+      return true;
+    }
+  }
+
   std::pair<bool, bool> pre_stable_extent_paddr_mod(
     read_set_item_t<Transaction> &item)
   {
@@ -393,7 +408,7 @@ public:
     bool retired) {
     read_set.insert(item);
     if (retired) {
-      retired_set.emplace(item.ref, trans_id);
+      retired_set.emplace(item.ref, *this);
     }
   }
   void maybe_update_pending_paddr(
@@ -416,15 +431,18 @@ public:
       auto &mextent = *i;
       write_set.erase(mextent);
       extent_len_t off = 0;
-      if (new_paddr.is_absolute_segmented()) {
-        assert(mextent.get_paddr().as_seg_paddr().get_segment_id()
-          == old_paddr.as_seg_paddr().get_segment_id());
+      if (old_paddr.is_absolute_segmented()) {
+        assert(mextent.get_paddr().as_seg_paddr().get_segment_id() ==
+          old_paddr.as_seg_paddr().get_segment_id());
+        assert(mextent.get_paddr().as_seg_paddr().get_segment_off() >=
+          old_paddr.as_seg_paddr().get_segment_off());
         assert(mextent.get_paddr().as_seg_paddr().get_segment_off()
-          >= old_paddr.as_seg_paddr().get_segment_off());
-        off = mextent.get_paddr().as_seg_paddr().get_segment_off()
-          - old_paddr.as_seg_paddr().get_segment_off();
+                + mextent.get_length() <=
+          old_paddr.as_seg_paddr().get_segment_off() + len);
+        off = mextent.get_paddr().as_seg_paddr().get_segment_off() -
+          old_paddr.as_seg_paddr().get_segment_off();
       } else {
-        assert(new_paddr.is_absolute_random_block());
+        assert(mextent.get_paddr().is_absolute_random_block());
         off = mextent.get_paddr().as_blk_paddr().get_device_off() -
           old_paddr.as_blk_paddr().get_device_off();
       }
@@ -432,6 +450,24 @@ public:
       write_set.insert(mextent);
     }
   }
+  void remove_shadow_from_write_set(
+    const paddr_t &shadow_paddr, extent_len_t len) {
+    std::vector<CachedExtent*> exts;
+    for (auto [bottom, top] = write_set.get_overlap(shadow_paddr, len);
+         bottom != top;
+         bottom++) {
+      auto &mextent = *bottom;
+      if (mextent.is_initial_pending()) {
+        assert(!mextent.is_shadow_extent());
+        continue;
+      }
+      assert(mextent.is_shadow_extent() && mextent.is_exist_clean());
+      exts.emplace_back(&mextent);
+    }
+    for (auto i :exts) {
+      write_set.erase(*i);
+    }
+  }
 
   template <typename F>
   auto for_each_finalized_fresh_block(F &&f) const {
@@ -540,7 +576,16 @@ public:
   friend class crimson::os::seastore::SeaStore;
   friend class TransactionConflictCondition;
 
+  uint64_t write_hit_hot = 0;
+  uint64_t write_hit_cold = 0;
+  uint64_t read_hit_hot = 0;
+  uint64_t read_hit_cold = 0;
+
   void reset_preserve_handle() {
+    write_hit_hot = 0;
+    write_hit_cold = 0;
+    read_hit_hot = 0;
+    read_hit_cold = 0;
     root.reset();
     offset = 0;
     delayed_temp_offset = 0;
@@ -576,6 +621,7 @@ public:
     views.clear();
     copied_lba_keys.clear();
     update_copied_lba_key = nullptr;
+    touched_prefix.clear();
   }
 
   bool did_reset() const {
@@ -692,22 +738,37 @@ public:
     return cache_hint;
   }
 
+  void touch_laddr_prefix(laddr_t laddr) {
+    touched_prefix.insert(laddr.get_object_prefix());
+  }
+
+  std::unordered_set<laddr_t> &get_touched_laddr_prefix() {
+    return touched_prefix;
+  }
+
   btree_cursor_stats_t cursor_stats;
 
   bool force_rewrite_conflict = false;
 
+  struct lmapping_t {
+    laddr_t dest = L_ADDR_NULL;
+    extent_len_t len = 0;
+  };
   using update_copied_lba_key_func_t =
-    std::function<void (laddr_t, paddr_t)>;
+    std::function<void (laddr_t, paddr_t,
+                  extent_len_t, std::optional<paddr_t>)>;
   void new_lba_key_copied(
     laddr_t src,
     laddr_t dest,
+    extent_len_t len,
     update_copied_lba_key_func_t &&func) {
-    copied_lba_keys.emplace(src, dest);
+    copied_lba_keys.emplace(src, lmapping_t{dest, len});
     if (!update_copied_lba_key) {
       update_copied_lba_key = std::move(func);
     }
   }
-  void maybe_sync_copied_lba_key(laddr_t laddr, paddr_t paddr) {
+  void maybe_sync_copied_lba_key(
+    laddr_t laddr, paddr_t paddr, std::optional<paddr_t> shadow) {
     if (likely(copied_lba_keys.empty())) {
       return;
     }
@@ -716,8 +777,9 @@ public:
     if (it == copied_lba_keys.end()) {
       return;
     }
-    laddr_t key = it->second;
-    update_copied_lba_key(key, paddr);
+    laddr_t key = it->second.dest;
+    extent_len_t len = it->second.len;
+    update_copied_lba_key(key, paddr, len, shadow);
   }
   RootBlockRef peek_root() {
     return root;
@@ -918,6 +980,8 @@ private:
    */
   retired_extent_set_t retired_set;
 
+  std::unordered_set<laddr_t> touched_prefix;
+
   /// stats to collect when commit or invalidate
   tree_stats_t onode_tree_stats;
   tree_stats_t omap_tree_stats; // exclude omap tree depth
@@ -948,7 +1012,7 @@ private:
 
   cache_hint_t cache_hint = CACHE_HINT_TOUCH;
 
-  std::map<laddr_t, laddr_t> copied_lba_keys;
+  std::map<laddr_t, lmapping_t> copied_lba_keys;
   update_copied_lba_key_func_t update_copied_lba_key;
 };
 using TransactionRef = Transaction::Ref;

src/crimson/os/seastore/transaction_manager.cc

@@ -39,6 +39,7 @@ TransactionManager::TransactionManager(
     journal(std::move(_journal)),
     epm(std::move(_epm)),
     backref_manager(std::move(_backref_manager)),
+    logical_bucket(nullptr),
     full_extent_integrity_check(
       crimson::common::get_conf<bool>(
         "seastore_full_integrity_check")),
@@ -46,6 +47,10 @@ TransactionManager::TransactionManager(
 {
   epm->set_extent_callback(this);
   journal->set_write_pipeline(&write_pipeline);
+  if (epm->has_cold_tier()) {
+    logical_bucket = epm->get_logical_bucket();
+    assert(support_logical_bucket());
+  }
 }
 
 TransactionManager::mkfs_ertr::future<> TransactionManager::mkfs()
@@ -157,6 +162,10 @@ TransactionManager::mount()
             assert(paddr.is_absolute());
             cache->update_tree_extents_num(type, 1);
             epm->mark_space_used(paddr, len);
+            if (support_logical_bucket() &&
+                !epm->is_cold_device(paddr.get_device_id())) {
+              logical_bucket->move_to_top(laddr.get_object_prefix());
+            }
           });
         } else {
           return backref_manager->scan_mapped_space(
@@ -183,6 +192,10 @@ TransactionManager::mount()
               assert(backref_key == P_ADDR_NULL);
               cache->update_tree_extents_num(type, 1);
               epm->mark_space_used(paddr, len);
+              if (support_logical_bucket() &&
+                !epm->is_cold_device(paddr.get_device_id())) {
+                logical_bucket->move_to_top(laddr.get_object_prefix());
+              }
             }
           });
         }
@@ -229,12 +242,34 @@ TransactionManager::ref_ret TransactionManager::remove(
   extent_ref_count_t refcount = cursor->get_refcount();
   auto laddr = cursor->get_laddr();
   auto length = cursor->get_length();
+  paddr_t shadow_addr = P_ADDR_NULL;
+  if (cursor->has_shadow_paddr()) {
+    shadow_addr = cursor->get_shadow_paddr();
+  }
   assert(refcount > 0);
   --refcount;
   co_await lba_manager->update_mapping_refcount(
     t, std::move(cursor), -1);
   if (refcount == 0) {
     cache->retire_extent(t, ref);
+    if (shadow_addr != P_ADDR_NULL) {
+      if (auto shadow = ref->get_shadow(); shadow) {
+        cache->retire_extent(t, shadow);
+      } else {
+        auto laddr = ref->get_laddr();
+        cache->retire_absent_extent_addr_by_type(
+          t, laddr, shadow_addr, length, ref->get_type(),
+          [ref, laddr](auto &extent) {
+            auto lextent = extent.template cast<LogicalChildNode>();
+            assert(extent.is_logical());
+            assert(!lextent->has_laddr());
+            assert(!extent.has_been_invalidated());
+            lextent->set_laddr(laddr);
+            extent.set_shadow_extent(true);
+            ref->set_shadow(lextent);
+          });
+      }
+    }
   }
   DEBUGT("removed {}~0x{:x} refcount={} -- {}",
 	 t, laddr, length,
@@ -298,11 +333,32 @@ TransactionManager::_remove(
 	LogicalChildNode
 	>();
       ceph_assert(extent);
-      cache->retire_extent(t, std::move(extent));
+      cache->retire_extent(t, extent);
+      if (mapping.has_shadow_val()) {
+        if (auto shadow = extent->get_shadow(); shadow) {
+          cache->retire_extent(t, shadow);
+        } else {
+          auto laddr = mapping.get_intermediate_base();
+          std::ignore = cache->retire_absent_extent_addr_by_type(
+            t, laddr,
+            mapping.get_shadow_val(),
+            mapping.get_intermediate_length(),
+            mapping.get_extent_type(),
+            [extent, laddr](auto &ext) {
+              auto lextent = ext.template cast<LogicalChildNode>();
+              assert(ext.is_logical());
+              assert(!lextent->has_laddr());
+              assert(!ext.has_been_invalidated());
+              lextent->set_laddr(laddr);
+              ext.set_shadow_extent(true);
+              extent->set_shadow(lextent);
+            });
+        }
+      }
     } else {
       auto &child_pos = maybe_mapped_extent.get_child_pos();
       auto laddr = mapping.get_intermediate_base();
-      std::ignore = cache->retire_absent_extent_addr_by_type(
+      auto ext = cache->retire_absent_extent_addr_by_type(
 	t, laddr,
 	mapping.get_val(),
 	mapping.get_intermediate_length(),
@@ -315,7 +371,23 @@ TransactionManager::_remove(
           child_pos.link_child(lextent.get());
           lextent->set_laddr(laddr);
         }
-      );
+      )->template cast<LogicalChildNode>();
+      if (mapping.has_shadow_val()) {
+        std::ignore = cache->retire_absent_extent_addr_by_type(
+          t, mapping.get_intermediate_base(),
+          mapping.get_shadow_val(),
+          mapping.get_intermediate_length(),
+          mapping.get_extent_type(),
+          [laddr, ext](auto &extent) {
+            auto lextent = extent.template cast<LogicalChildNode>();
+            assert(extent.is_logical());
+            assert(!lextent->has_laddr());
+            assert(!extent.has_been_invalidated());
+            lextent->set_laddr(laddr);
+            extent.set_shadow_extent(true);
+            ext->set_shadow(lextent);
+          });
+      }
     }
   }
 
@@ -425,7 +497,7 @@ TransactionManager::refs_ret TransactionManager::remove(
 
 base_iertr::future<LogicalChildNodeRef>
 TransactionManager::relocate_logical_extent(
-  Transaction &t, LBAMapping mapping)
+  Transaction &t, LBAMapping mapping, laddr_t new_laddr)
 {
   LOG_PREFIX(TransactionManager::relocate_logical_extent);
   SUBDEBUGT(seastore_tm, "relocate {}", t, mapping);
@@ -436,7 +508,7 @@ TransactionManager::relocate_logical_extent(
   if (!v.has_child()) {
     auto &child_pos = v.get_child_pos();
     auto laddr = mapping.get_key();
-    std::ignore = cache->retire_absent_extent_addr_by_type(
+    auto extent = cache->retire_absent_extent_addr_by_type(
       t,
       laddr,
       mapping.get_val(),
@@ -450,29 +522,119 @@ TransactionManager::relocate_logical_extent(
         child_pos.link_child(lextent.get());
         lextent->set_laddr(laddr);
       }
-    );
-    co_return cache->alloc_remapped_extent_by_type(
-      t, mapping.get_extent_type(), mapping.get_key(),
-      mapping.get_val(), 0, mapping.get_length(), std::nullopt
-    )->cast<LogicalChildNode>();
-  }
-
-  auto extent = co_await v.get_child_fut().si_then([](auto ext) {
-    return ext;
-  });
-
-  if (extent->is_stable()) {
-    cache->retire_extent(t, extent);
-    co_return cache->alloc_remapped_extent_by_type(
-      t, mapping.get_extent_type(), mapping.get_key(),
-      mapping.get_val(), 0, mapping.get_length(), std::nullopt
     )->cast<LogicalChildNode>();
+    if (mapping.has_shadow_val()) {
+      std::ignore = cache->retire_absent_extent_addr_by_type(
+        t, laddr,
+        mapping.get_shadow_val(),
+        mapping.get_intermediate_length(),
+        mapping.get_extent_type(),
+        [extent, laddr](auto &ext) {
+          auto lextent = ext.template cast<LogicalChildNode>();
+          assert(ext.is_logical());
+          assert(!lextent->has_laddr());
+          assert(!ext.has_been_invalidated());
+          lextent->set_laddr(laddr);
+          ext.set_shadow_extent(true);
+          extent->set_shadow(lextent);
+        });
+    }
   } else {
-    //TODO: relocating logical extents doesn't support
-    //      mutation pending extents yet.
-    assert(extent->is_initial_pending() || extent->is_exist_clean());
-    co_return extent;
+    auto extent = co_await v.get_child_fut_as<LogicalChildNode>();
+
+    if (extent->is_stable()) {
+      cache->retire_extent(t, extent);
+    } else {
+      //TODO: relocating logical extents doesn't support
+      //      mutation pending extents yet.
+      assert(extent->is_initial_pending() || extent->is_exist_clean());
+      extent->set_laddr(new_laddr);
+      if (mapping.has_shadow_val()) {
+        assert(extent->get_shadow());
+        extent->get_shadow()->set_laddr(new_laddr);
+      }
+      co_return extent;
+    }
   }
+  auto remapped_extent = cache->alloc_remapped_extent_by_type(
+    t, mapping.get_extent_type(), new_laddr,
+    mapping.get_val(), 0, mapping.get_length(), std::nullopt
+  )->cast<LogicalChildNode>();
+  if (mapping.has_shadow_val()) {
+    auto remapped_shadow = cache->alloc_remapped_extent_by_type(
+      t, mapping.get_extent_type(), new_laddr,
+      mapping.get_shadow_val(), 0, mapping.get_length(), std::nullopt
+    )->cast<LogicalChildNode>();
+    remapped_shadow->set_shadow_extent(true);
+    remapped_extent->set_shadow(remapped_shadow);
+  }
+  co_return remapped_extent;
+}
+
+base_iertr::future<LogicalChildNodeRef>
+TransactionManager::relocate_shadow_extent(
+  Transaction &t, LBAMapping mapping)
+{
+  LOG_PREFIX(TransactionManager::relocate_shadow_extent);
+  SUBDEBUGT(seastore_tm, "relocate {}", t, mapping);
+  assert(mapping.has_shadow_val());
+  assert(!mapping.is_zero_reserved());
+  assert(mapping.is_viewable());
+  assert(!mapping.is_indirect());
+  assert(t.get_src() == transaction_type_t::DEMOTE);
+  auto v = get_extent_if_linked(t, *mapping.direct_cursor);
+  LogicalChildNodeRef extent;
+  auto laddr = mapping.get_intermediate_base();
+  if (!v.has_child()) {
+    auto &child_pos = v.get_child_pos();
+    extent = cache->retire_absent_extent_addr_by_type(
+      t,
+      laddr,
+      mapping.get_val(),
+      mapping.get_length(),
+      mapping.get_extent_type(),
+      [laddr, &child_pos](auto &extent) {
+        auto lextent = extent.template cast<LogicalChildNode>();
+        assert(extent.is_logical());
+        assert(!lextent->has_laddr());
+        assert(!extent.has_been_invalidated());
+        child_pos.link_child(lextent.get());
+        lextent->set_laddr(laddr);
+      }
+    )->template cast<LogicalChildNode>();
+  } else {
+    extent = co_await std::move(v.get_child_fut());
+    if (extent->is_stable_dirty()) {
+      // the extent is dirty, skip it.
+      DEBUGT("skipping dirty extent: {}", t, *extent);
+      co_return LogicalChildNodeRef();
+    }
+    cache->retire_extent(t, extent);
+  }
+  if (auto shadow = extent->get_shadow(); shadow) {
+    cache->retire_extent(t, shadow);
+  } else {
+    auto shadow_paddr = mapping.get_shadow_val();
+    std::ignore = cache->retire_absent_extent_addr_by_type(
+      t, laddr, shadow_paddr, mapping.get_length(), mapping.get_extent_type(),
+      [laddr, extent](auto &ext) {
+        auto lextent = ext.template cast<LogicalChildNode>();
+        assert(ext.is_logical());
+        assert(!lextent->has_laddr());
+        assert(!ext.has_been_invalidated());
+        lextent->set_laddr(laddr);
+        ext.set_shadow_extent(true);
+        extent->set_shadow(lextent);
+      }
+    );
+  }
+  auto nextent =  cache->alloc_remapped_extent_by_type(
+    t, mapping.get_extent_type(), laddr,
+    mapping.get_shadow_val(), 0, mapping.get_length(), std::nullopt
+  )->cast<LogicalChildNode>();
+  nextent->set_prior_instance(extent);
+  nextent->set_last_committed_crc(extent->get_last_committed_crc());
+  co_return nextent;
 }
 
 TransactionManager::submit_transaction_iertr::future<>
@@ -726,6 +888,11 @@ TransactionManager::do_submit_transaction(
     journal->get_trimmer().update_journal_tails(
       cache->get_oldest_dirty_from().value_or(start_seq),
       cache->get_oldest_backref_dirty_from().value_or(start_seq));
+    if (support_logical_bucket()) {
+      for (auto &prefix : tref.get_touched_laddr_prefix()) {
+	logical_bucket->move_to_top(prefix.get_object_prefix());
+      }
+    }
     }).handle_error(
       submit_transaction_iertr::pass_further{},
       crimson::ct_error::assert_all("Hit error submitting to journal")
@@ -765,10 +932,12 @@ TransactionManager::get_next_dirty_extents(
   return cache->get_next_dirty_extents(t, seq, max_bytes);
 }
 
-TransactionManager::rewrite_extent_ret
+TransactionManager::rewrite_extent_iertr::future<
+  std::vector<CachedExtentRef>>
 TransactionManager::rewrite_logical_extent(
   Transaction& t,
-  LogicalChildNodeRef extent)
+  LogicalChildNodeRef extent,
+  paddr_t paddr_hint)
 {
   LOG_PREFIX(TransactionManager::rewrite_logical_extent);
   if (extent->has_been_invalidated()) {
@@ -776,16 +945,36 @@ TransactionManager::rewrite_logical_extent(
     ceph_abort();
   }
 
+  bool is_tracked =
+    support_logical_bucket() &&
+    // lextent is from hot tier
+    !epm->is_cold_device(extent->get_paddr().get_device_id()) &&
+    // lextent is cached by non volatile cache
+    logical_bucket->is_cached(
+      extent->get_laddr().get_object_prefix());
+
   if (get_extent_category(extent->get_type()) == data_category_t::METADATA) {
     assert(extent->is_fully_loaded());
     cache->retire_extent(t, extent);
-    auto nextent = cache->alloc_new_non_data_extent_by_type(
-      t,
-      extent->get_type(),
-      extent->get_length(),
-      extent->get_user_hint(),
-      // get target rewrite generation
-      extent->get_rewrite_generation())->cast<LogicalChildNode>();
+    LogicalChildNodeRef nextent;
+    while (!nextent) {
+      try {
+        nextent = cache->alloc_new_non_data_extent_by_type(
+          t,
+          extent->get_type(),
+          extent->get_length(),
+          extent->get_user_hint(),
+          // get target rewrite generation
+          extent->get_rewrite_generation(),
+          paddr_hint,
+          is_tracked)->cast<LogicalChildNode>();
+      } catch (crimson::ct_error::eagain&) {}
+      if (!nextent) {
+        epm->maybe_wake_background();
+        co_await trans_intr::make_interruptible(epm->wait_background());
+      }
+    }
+    assert(nextent->get_write_policy() != write_policy_t::WRITE_THROUGH);
     nextent->rewrite(t, *extent, 0);
 
     DEBUGT("rewriting meta -- {} to {}", t, *extent, *nextent);
@@ -815,6 +1004,7 @@ TransactionManager::rewrite_logical_extent(
       extent->get_paddr(),
       *nextent
     );
+    co_return std::vector<CachedExtentRef>{nextent};
   } else {
     assert(get_extent_category(extent->get_type()) == data_category_t::DATA);
 
@@ -823,13 +1013,29 @@ TransactionManager::rewrite_logical_extent(
       t, std::move(extent), 0, length);
     assert(extent->is_fully_loaded());
     cache->retire_extent(t, extent);
-    auto extents = cache->alloc_new_data_extents_by_type(
-      t,
-      extent->get_type(),
-      extent->get_length(),
-      extent->get_user_hint(),
-      // get target rewrite generation
-      extent->get_rewrite_generation());
+    std::vector<CachedExtentRef> extents;
+    while (extents.empty()) {
+      try {
+        extents = cache->alloc_new_data_extents_by_type(
+          t,
+          extent->get_type(),
+          extent->get_length(),
+          {
+            extent->get_user_hint(),
+            // get target rewrite generation
+            extent->get_rewrite_generation(),
+            is_tracked,
+            paddr_hint,
+            // WRITH_THROUGH is only effective for client io, so
+            // always set the write policy to WRITE_BACK here
+            write_policy_t::WRITE_BACK
+          });
+      } catch (crimson::ct_error::eagain&) {}
+      if (extents.empty()) {
+        epm->maybe_wake_background();
+        co_await trans_intr::make_interruptible(epm->wait_background());
+      }
+    }
     extent_len_t off = 0;
     auto left = extent->get_length();
     extent_ref_count_t refcount = 0;
@@ -841,6 +1047,7 @@ TransactionManager::rewrite_logical_extent(
     t.force_rewrite_conflict = (extents.size() > 1);
     for (auto &_nextent : extents) {
       auto nextent = _nextent->template cast<LogicalChildNode>();
+      assert(nextent->get_write_policy() != write_policy_t::WRITE_THROUGH);
       bool first_extent = (off == 0);
       ceph_assert(left >= nextent->get_length());
       nextent->rewrite(t, *extent, off);
@@ -883,6 +1090,7 @@ TransactionManager::rewrite_logical_extent(
       off += nextent->get_length();
       left -= nextent->get_length();
     }
+    co_return std::move(extents);
   }
 }
 
@@ -909,7 +1117,6 @@ TransactionManager::rewrite_extent_ret TransactionManager::rewrite_extent(
            rewrite_gen_printer_t{target_generation},
            sea_time_point_printer_t{modify_time},
            *extent);
-    ceph_assert(!extent->is_pending_io());
   }
 
   assert(extent->is_valid() && !extent->is_initial_pending());
@@ -949,7 +1156,9 @@ TransactionManager::rewrite_extent_ret TransactionManager::rewrite_extent(
   auto fut = rewrite_extent_iertr::now();
   if (extent->is_logical()) {
     assert(is_logical_type(extent->get_type()));
-    fut = rewrite_logical_extent(t, extent->cast<LogicalChildNode>());
+    fut = rewrite_logical_extent(
+      t, extent->cast<LogicalChildNode>(), P_ADDR_NULL
+    ).discard_result();
   } else if (is_backref_node(extent->get_type())) {
     fut = backref_manager->rewrite_extent(t, extent);
   } else {
@@ -1061,9 +1270,9 @@ TransactionManager::move_region(
         dst = co_await dst.refresh();
       }
     } else if (!src.is_zero_reserved()) {
-      auto extent = co_await relocate_logical_extent(t, src);
       auto laddr = calc_dst_key();
-      extent->set_laddr(laddr);
+      auto extent = co_await relocate_logical_extent(t, src, laddr);
+      assert(extent->get_laddr() == laddr);
       auto ret = co_await lba_manager->move_direct_mapping(
         t, src.get_effective_cursor_ref(),
         laddr, dst.get_effective_cursor_ref(), *extent);
@@ -1094,6 +1303,346 @@ TransactionManager::move_region(
   co_return;
 }
 
+TransactionManager::promote_extent_ret
+TransactionManager::promote_extent(
+  Transaction &t,
+  CachedExtentRef extent)
+{
+  LOG_PREFIX(TransactionManager::promote_extent);
+  assert(epm->is_cold_device(extent->get_paddr().get_device_id()));
+  DEBUGT("promote extent: {}", t, *extent);
+  ceph_assert(extent->is_logical());
+
+  std::vector<LogicalChildNodeRef> promoted_extents;
+  auto orig_ext = extent->cast<LogicalChildNode>();
+  // fill extent if it's not fully loaded
+  if (!extent->is_fully_loaded()) {
+    ceph_assert(extent->get_type() == extent_types_t::OBJECT_DATA_BLOCK);
+    extent = co_await cache->read_extent_maybe_partial(
+      t, extent->cast<ObjectDataBlock>(), 0, extent->get_length());
+  }
+
+  cache->retire_extent(t, extent);
+
+  if (get_extent_category(extent->get_type()) == data_category_t::DATA) {
+    std::vector<CachedExtentRef> promoted_raw_extents;
+    while (promoted_raw_extents.empty()) {
+      try {
+        promoted_raw_extents = cache->alloc_new_data_extents_by_type(
+          t,
+          orig_ext->get_type(),
+          orig_ext->get_length(),
+          {
+            placement_hint_t::HOT,
+            INIT_GENERATION,
+            true,
+            P_ADDR_NULL,
+            write_policy_t::WRITE_BACK
+          });
+      } catch (crimson::ct_error::eagain&) {}
+      if (promoted_raw_extents.empty()) {
+        epm->maybe_wake_background();
+        co_await trans_intr::make_interruptible(epm->wait_background());
+      }
+    }
+    t.touch_laddr_prefix(orig_ext->get_laddr().get_object_prefix());
+
+    promoted_extents.reserve(promoted_raw_extents.size());
+
+    extent_len_t offset = 0;
+    auto orig_laddr = orig_ext->get_laddr();
+    auto orig_paddr = orig_ext->get_paddr();
+    auto orig_length = orig_ext->get_length();
+    t.force_rewrite_conflict |= (promoted_raw_extents.size() > 1);
+    for (auto &extent : promoted_raw_extents) {
+      auto slice_laddr = (orig_laddr + offset).checked_to_laddr();
+      auto slice_length = extent->get_length();
+      extent->rewrite(t, *orig_ext, offset);
+      if (extent->get_paddr().is_absolute()) {
+        assert(!cache->is_on_cold_tier(extent->get_paddr()));
+      }
+
+      auto lext = extent->cast<LogicalChildNode>();
+      lext->set_laddr(slice_laddr);
+      //TODO: this memory copy should be saved
+      orig_ext->get_bptr().copy_out(
+        offset, slice_length, lext->get_bptr().c_str());
+      lext->set_last_committed_crc(lext->calc_crc32c());
+
+      promoted_extents.push_back(lext);
+
+      auto remapped_cold_extent = cache->alloc_remapped_extent_by_type(
+        t,
+        orig_ext->get_type(),
+        slice_laddr,
+        orig_paddr.add_offset(offset),
+        offset,
+        slice_length,
+        std::nullopt);
+      remapped_cold_extent->set_shadow_extent(true);
+      auto lremapped = remapped_cold_extent->template cast<LogicalChildNode>();
+      lext->set_shadow(lremapped);
+
+      offset += slice_length;
+    }
+    ceph_assert(offset == orig_length);
+  } else {
+    CachedExtentRef promoted_extent;
+    while (!promoted_extent) {
+      try {
+        promoted_extent = cache->alloc_new_non_data_extent_by_type(
+          t,
+          orig_ext->get_type(),
+          orig_ext->get_length(),
+          placement_hint_t::HOT,
+          INIT_GENERATION,
+          P_ADDR_NULL,
+          true);
+      } catch (crimson::ct_error::eagain&) {}
+      if (!promoted_extent) {
+        epm->maybe_wake_background();
+        co_await trans_intr::make_interruptible(epm->wait_background());
+      }
+    }
+    auto lext = promoted_extent->cast<LogicalChildNode>();
+    lext->set_laddr(orig_ext->get_laddr());
+    lext->rewrite(t, *orig_ext, 0);
+    if (lext->get_paddr().is_absolute()) {
+      assert(!cache->is_on_cold_tier(lext->get_paddr()));
+    }
+    t.touch_laddr_prefix(orig_ext->get_laddr().get_object_prefix());
+    //TODO: this memory copy should be saved
+    orig_ext->get_bptr().copy_out(
+      0,
+      orig_ext->get_length(),
+      lext->get_bptr().c_str());
+    promoted_extents.push_back(lext);
+    auto remapped_cold_extent = cache->alloc_remapped_extent_by_type(
+      t,
+      orig_ext->get_type(),
+      orig_ext->get_laddr(),
+      orig_ext->get_paddr(),
+      0,
+      orig_ext->get_length(),
+      std::nullopt);
+    remapped_cold_extent->set_shadow_extent(true);
+    auto lremapped = remapped_cold_extent->template cast<LogicalChildNode>();
+    lext->set_shadow(lremapped);
+  }
+
+  auto cursor = co_await lba_manager->get_cursor(
+    t, *orig_ext
+  ).handle_error_interruptible(
+    promote_extent_iertr::pass_further(),
+    crimson::ct_error::assert_all("invalid error"));
+  auto mapping = co_await resolve_cursor_to_mapping(t, std::move(cursor));
+  assert(!mapping.is_indirect());
+  co_return co_await lba_manager->promote_extent(
+    t, *mapping.direct_cursor, std::move(promoted_extents));
+}
+
+TransactionManager::promote_extent_ret
+TransactionManager::promote_extents_from_disk(
+  Transaction &t,
+  paddr_t paddr)
+{
+  using scan_device_func_t = BackrefManager::scan_device_func_t;
+  std::size_t size = 0;
+  scan_device_func_t func = [this, &t, &size](
+    paddr_t paddr, extent_len_t length, extent_types_t type, laddr_t laddr)
+      -> base_iertr::future<seastar::stop_iteration> {
+    if (type != extent_types_t::OBJECT_DATA_BLOCK) {
+      co_return seastar::stop_iteration::no;
+    }
+    auto cursor = co_await lba_manager->get_cursor(t, laddr
+      ).handle_error_interruptible(
+        crimson::ct_error::enoent::handle([](auto e) {
+          // Another no_conflict transaction should have removed
+          // the mapping between the backref retrieval and the
+          // lba search, ignore it.
+          return seastar::make_ready_future<LBACursorRef>();
+        }),
+        crimson::ct_error::pass_further_all{}
+      );
+    if (!cursor || cursor->is_end() ||
+        !cursor->get_paddr().is_absolute() ||
+        !cache->is_on_cold_tier(cursor->get_paddr())) {
+      // the mapping has been modified and the extent is
+      // either removed or already on the hot tier, skip it.
+      co_return seastar::stop_iteration::no;
+    }
+    assert(cursor->is_direct());
+    assert(!cursor->has_shadow_paddr());
+    auto extent = co_await read_cursor_by_type(t, std::move(cursor), type);
+    if (extent->is_stable_dirty()) {
+      // dirty extents shouldn't be promoted as is in
+      // the real world
+      co_return seastar::stop_iteration::no;
+    }
+    auto &pinboard = *cache->get_extent_pinboard();
+    pinboard.remove(*extent);
+    extent->set_pin_state(extent_pin_state_t::Promoting);
+    co_await promote_extent(t, extent);
+    size += length;
+    if (size >= crimson::common::get_conf<
+        Option::size_t>("seastore_cache_promotion_size")) {
+      co_return seastar::stop_iteration::yes;
+    } else {
+      co_return seastar::stop_iteration::no;
+    }
+  };
+  co_await backref_manager->scan_device(t, paddr, func);
+}
+
+TransactionManager::rewrite_extents_ret TransactionManager::rewrite_extents(
+  Transaction &t,
+  std::vector<CachedExtentRef> &extents,
+  rewrite_gen_t target_generation,
+  placement_hint_t hint,
+  sea_time_point modify_time)
+{
+  LOG_PREFIX(TransactionManager::rewrite_extents);
+  return seastar::do_with(
+    P_ADDR_NULL,
+    L_ADDR_NULL,
+    [this, &t, target_generation, modify_time, &extents, FNAME, hint]
+    (auto &paddr_hint, auto &next_laddr) {
+    return trans_intr::do_for_each(
+      extents,
+      [this, &t, target_generation, modify_time, FNAME,
+      &paddr_hint, &next_laddr, hint](auto &extent) {
+      {
+        auto updated = cache->update_extent_from_transaction(t, extent);
+        if (!updated) {
+          DEBUGT("extent is already retired, skipping -- {}", t, *extent);
+          return rewrite_extent_iertr::now();
+        }
+        extent = updated;
+        ceph_assert(!extent->is_pending_io());
+      }
+
+      assert(extent->is_valid() && !extent->is_initial_pending());
+      if (extent->is_stable_dirty()) {
+        if (epm->can_inplace_rewrite(t, extent)) {
+          DEBUGT("delta overwriting extent -- {}", t, *extent);
+          t.add_inplace_rewrite_extent(extent);
+          extent->set_inplace_rewrite_generation();
+          return rewrite_extent_iertr::now();
+        }
+	if (extent->get_version() == 1 && extent->has_mutation()) {
+	  t.get_rewrite_stats().account_n_dirty();
+	} else {
+	  // extent->get_version() > 1 or DIRTY
+	  t.get_rewrite_stats().account_dirty(extent->get_version());
+	}
+        extent->set_target_rewrite_generation(INIT_GENERATION);
+      } else {
+        extent->set_target_rewrite_generation(target_generation, hint);
+        ceph_assert(modify_time != NULL_TIME);
+        extent->set_modify_time(modify_time);
+      }
+
+      if (is_backref_node(extent->get_type())) {
+        DEBUGT("rewriting backref extent -- {}", t, *extent);
+        return backref_manager->rewrite_extent(t, extent);
+      }
+
+      if (extent->get_type() == extent_types_t::ROOT) {
+        DEBUGT("rewriting root extent -- {}", t, *extent);
+        cache->duplicate_for_write(t, extent);
+        return rewrite_extent_iertr::now();
+      }
+
+      if (extent->is_logical()) {
+        auto ext = extent->template cast<LogicalChildNode>();
+        if (next_laddr != ext->get_laddr()) {
+          paddr_hint = P_ADDR_NULL;
+        }
+        next_laddr = (ext->get_laddr() + ext->get_length()).checked_to_laddr();
+        return rewrite_logical_extent(t, ext, paddr_hint
+        ).si_then([&paddr_hint](auto nlextents) {
+          for (auto &nlextent : nlextents) {
+            paddr_hint = nlextent->get_paddr() + nlextent->get_length();
+          }
+        });
+      } else {
+        DEBUGT("rewriting physical extent -- {}", t, *extent);
+        return lba_manager->rewrite_extent(t, extent);
+      }
+    });
+  });
+}
+TransactionManager::demote_region_ret
+TransactionManager::demote_region(
+  Transaction &t,
+  laddr_t start,
+  loffset_t max_proceed_size)
+{
+  LOG_PREFIX(TransactionManager::demote_region);
+  auto prefix = start.get_object_prefix();
+  DEBUGT("start demote {}", t, prefix);
+  auto cursor = co_await lba_manager->upper_bound_right(
+    t, start
+  ).handle_error_interruptible(
+    crimson::ct_error::enoent::handle([](auto) {
+      // It's possible that there has been no lba mappings left
+      // when demoting a region, for example, a temp recovering
+      // object may have been renamed, which makes the lba mappings
+      // in its own region moved.
+      return seastar::make_ready_future<LBACursorRef>();
+    }),
+    demote_region_iertr::pass_further{},
+    crimson::ct_error::assert_all("unexpected enoent"));
+  demote_region_res_t ret{0, 0, false};
+  if (!cursor) {
+    ret.complete = true;
+    co_return ret;
+  }
+  auto it = co_await resolve_cursor_to_mapping(t, std::move(cursor));
+  std::vector<CachedExtentRef> extents;
+  while ((ret.demoted_size + ret.evicted_size) < max_proceed_size) {
+    if (it.is_end() || it.get_key().get_object_prefix() != prefix) {
+      ret.complete = true;
+      break;
+    }
+    if (it.is_indirect()) {
+      it = co_await it.next();
+      continue;
+    }
+    if (it.has_shadow_val()) {
+      DEBUGT("demote shadow {}", t, it);
+      auto extent = co_await relocate_shadow_extent(t, it);
+      if (!extent) {
+        DEBUGT("{} can't be demoted", t, it);
+        continue;
+      }
+      ret.demoted_size += extent->get_length();
+      auto cursor = co_await lba_manager->demote_extent(
+        t, *it.direct_cursor, *extent);
+      auto nit = co_await resolve_cursor_to_mapping(t, std::move(cursor));
+      it = co_await nit.next();
+    } else if (!it.is_indirect() && !it.is_zero_reserved() &&
+      !epm->is_cold_device(it.get_val().get_device_id())) {
+      DEBUGT("demote hot {}", t, it);
+      auto extent = co_await read_cursor_by_type(
+        t, it.direct_cursor, it.get_extent_type());
+      ret.evicted_size += extent->get_length();
+      extents.push_back(extent);
+      it = co_await it.next();
+    } else {
+      DEBUGT("skip {}", t, it);
+      it = co_await it.next();
+    }
+  }
+
+  co_await rewrite_extents(
+    t, extents, epm->get_max_hot_gen() + 1,
+    placement_hint_t::COLD,
+    seastar::lowres_system_clock::now());
+
+  co_return ret;
+}
+
 TransactionManager::get_extents_if_live_ret
 TransactionManager::get_extents_if_live(
   Transaction &t,
@@ -1199,9 +1748,11 @@ TransactionManagerRef make_transaction_manager(
   auto rbs = std::make_unique<RBMDeviceGroup>();
   auto backref_manager = create_backref_manager(*cache);
   SegmentManagerGroupRef cold_sms = nullptr;
+  RBMDeviceGroupRef cold_rbs = nullptr;
   std::vector<SegmentProvider*> segment_providers_by_id{DEVICE_ID_MAX, nullptr};
 
   auto p_backend_type = primary_device->get_backend_type();
+  INFO("primary backend: {}", p_backend_type);
 
   if (p_backend_type == backend_type_t::SEGMENTED) {
     auto dtype = primary_device->get_device_type();
@@ -1209,6 +1760,7 @@ TransactionManagerRef make_transaction_manager(
 		dtype != device_type_t::EPHEMERAL_COLD);
     sms->add_segment_manager(static_cast<SegmentManager*>(primary_device));
   } else {
+    assert(p_backend_type != backend_type_t::NONE);
     auto rbm = std::make_unique<BlockRBManager>(
       static_cast<RBMDevice*>(primary_device), "", is_test);
     rbs->add_rb_manager(std::move(rbm));
@@ -1217,17 +1769,29 @@ TransactionManagerRef make_transaction_manager(
   for (auto &p_dev : secondary_devices) {
     if (p_dev->get_backend_type() == backend_type_t::SEGMENTED) {
       if (p_dev->get_device_type() == primary_device->get_device_type()) {
+	INFO("add {} to main segment backend", device_id_printer_t{p_dev->get_device_id()});
         sms->add_segment_manager(static_cast<SegmentManager*>(p_dev));
       } else {
         if (!cold_sms) {
           cold_sms = std::make_unique<SegmentManagerGroup>();
         }
+	INFO("add {} to cold segment backend", device_id_printer_t{p_dev->get_device_id()});
         cold_sms->add_segment_manager(static_cast<SegmentManager*>(p_dev));
       }
     } else {
+      assert(p_backend_type != backend_type_t::NONE);
       auto rbm = std::make_unique<BlockRBManager>(
 	static_cast<RBMDevice*>(p_dev), "", is_test);
-      rbs->add_rb_manager(std::move(rbm));
+      if (p_dev->get_device_type() == primary_device->get_device_type()) {
+	INFO("add {} to rbm backend", device_id_printer_t{p_dev->get_device_id()});
+	rbs->add_rb_manager(std::move(rbm));
+      } else {
+	if (!cold_rbs) {
+	  cold_rbs = std::make_unique<RBMDeviceGroup>();
+	}
+	INFO("add {} to cold rbm backend", device_id_printer_t{p_dev->get_device_id()});
+	cold_rbs->add_rb_manager(std::move(rbm));
+      }
     }
   }
 
@@ -1271,15 +1835,20 @@ TransactionManagerRef make_transaction_manager(
       store_index,
       *backref_manager, trimmer_config,
       backend_type, roll_start, roll_size,
-      !pure_rbm_backend);
+      !pure_rbm_backend
+        || crimson::common::get_conf<bool>(
+            "seastore_logical_bucket_cache_test_stress")
+    );
 
   AsyncCleanerRef cleaner;
   JournalRef journal;
 
-  SegmentCleanerRef cold_segment_cleaner = nullptr;
+  AsyncCleanerRef cold_cleaner = nullptr;
+  bool scan_alloc_on_boot = false;
 
   if (cold_sms) {
-    cold_segment_cleaner = SegmentCleaner::create(
+    assert(!cold_rbs);
+    auto segment_cleaner = SegmentCleaner::create(
       store_index,
       cleaner_config,
       std::move(cold_sms),
@@ -1289,11 +1858,21 @@ TransactionManagerRef make_transaction_manager(
       cleaner_is_detailed,
       /* is_cold = */ true);
     if (backend_type == backend_type_t::SEGMENTED) {
-      for (auto id : cold_segment_cleaner->get_device_ids()) {
+      for (auto id : segment_cleaner->get_device_ids()) {
         segment_providers_by_id[id] =
-          static_cast<SegmentProvider*>(cold_segment_cleaner.get());
+          static_cast<SegmentProvider*>(segment_cleaner.get());
       }
     }
+    cold_cleaner = std::move(segment_cleaner);
+  } else if (cold_rbs) {
+    scan_alloc_on_boot = true;
+    cold_cleaner = RBMCleaner::create(
+      store_index,
+      std::move(cold_rbs),
+      *backref_manager,
+      *lba_manager,
+      cleaner_is_detailed,
+      true);
   }
 
   if (backend_type == backend_type_t::SEGMENTED) {
@@ -1314,14 +1893,16 @@ TransactionManagerRef make_transaction_manager(
     journal = journal::make_segmented(
       store_index,
       *segment_cleaner,
-      *journal_trimmer);
+      *journal_trimmer,
+      scan_alloc_on_boot);
   } else {
     cleaner = RBMCleaner::create(
       store_index,
       std::move(rbs),
       *backref_manager,
       *lba_manager,
-      cleaner_is_detailed);
+      cleaner_is_detailed,
+      false);
     journal = journal::make_circularbounded(
       store_index,
       *journal_trimmer,
@@ -1333,7 +1914,8 @@ TransactionManagerRef make_transaction_manager(
 
   epm->init(std::move(journal_trimmer),
 	    std::move(cleaner),
-	    std::move(cold_segment_cleaner));
+	    std::move(cold_cleaner),
+	    cache->get_extent_pinboard());
   epm->set_primary_device(primary_device);
 
   INFO("main backend type: {}, cold tier: {}",

src/crimson/os/seastore/transaction_manager.h

@@ -26,6 +26,7 @@
 #include "crimson/os/seastore/logging.h"
 #include "crimson/os/seastore/seastore_types.h"
 #include "crimson/os/seastore/cache.h"
+#include "crimson/os/seastore/logical_bucket.h"
 #include "crimson/os/seastore/root_meta.h"
 #include "crimson/os/seastore/lba_manager.h"
 #include "crimson/os/seastore/backref_manager.h"
@@ -111,6 +112,11 @@ public:
    * responsible to update the corresponding lba mapping.
    */
   base_iertr::future<LogicalChildNodeRef> relocate_logical_extent(
+    Transaction &t,
+    LBAMapping mapping,
+    laddr_t new_laddr);
+
+  base_iertr::future<LogicalChildNodeRef> relocate_shadow_extent(
     Transaction &t,
     LBAMapping mapping);
 
@@ -240,6 +246,28 @@ public:
     }
   };
 
+  void update_hit_ratio(Transaction& t, device_id_t id) {
+    if (epm->is_cold_device(id)) {
+      t.write_hit_cold++;
+    } else {
+      t.write_hit_hot++;
+    }
+  }
+
+  void update_read_ratio(Transaction& t, device_id_t id) {
+    if (epm->is_cold_device(id)) {
+      t.read_hit_cold++;
+    } else {
+      t.read_hit_hot++;
+    }
+  }
+
+  void submit_read_ratio(Transaction& t) {
+    if (cache) {
+      cache->update_read_ratio(t);
+    }
+  }
+
   template <typename T>
   using lextent_init_func_t = std::function<void (T&)>;
   /**
@@ -547,24 +575,27 @@ public:
     LOG_PREFIX(TransactionManager::alloc_non_data_extent);
     SUBDEBUGT(seastore_tm, "{} hint {}~0x{:x} phint={} ...",
               t, T::TYPE, laddr_hint, len, placement_hint);
-    auto ext = cache->alloc_new_non_data_extent<T>(
-      t,
-      len,
-      placement_hint,
-      INIT_GENERATION);
+    TCachedExtentRef<T> ext;
+    while (!ext) {
+      try {
+        ext = cache->alloc_new_non_data_extent<T>(
+          t, len, {placement_hint, INIT_GENERATION});
+      } catch(crimson::ct_error::eagain&) {}
+      if (!ext) {
+        epm->maybe_wake_background();
+        co_await trans_intr::make_interruptible(epm->wait_background());
+      }
+    }
     // user must initialize the logical extent themselves.
     assert(is_user_transaction(t.get_src()));
     ext->set_seen_by_users();
-    return lba_manager->alloc_extent(
+    co_await lba_manager->alloc_extent(
       t,
       laddr_hint,
       *ext,
-      EXTENT_DEFAULT_REF_COUNT
-    ).si_then([ext=std::move(ext), &t, FNAME](auto &&) mutable {
-      SUBDEBUGT(seastore_tm, "allocated {}", t, *ext);
-      return alloc_extent_iertr::make_ready_future<TCachedExtentRef<T>>(
-	std::move(ext));
-    });
+      EXTENT_DEFAULT_REF_COUNT);
+    SUBDEBUGT(seastore_tm, "allocated {}", t, *ext);
+    co_return ext;
   }
 
   /**
@@ -588,11 +619,21 @@ public:
     LOG_PREFIX(TransactionManager::alloc_data_extents);
     SUBDEBUGT(seastore_tm, "{} hint {}~0x{:x} phint={} ...",
               t, T::TYPE, laddr_hint, len, placement_hint);
-    auto exts = cache->alloc_new_data_extents<T>(
-      t,
-      len,
-      placement_hint,
-      INIT_GENERATION);
+    std::vector<TCachedExtentRef<T>> exts;
+    while (exts.empty()) {
+      try {
+        exts = cache->alloc_new_data_extents<T>(
+          t, len,
+          {
+            placement_hint, INIT_GENERATION, false, P_ADDR_NULL,
+            epm->get_write_policy(T::TYPE, len)
+          });
+      } catch (crimson::ct_error::eagain&) {}
+      if (exts.empty()) {
+        epm->maybe_wake_background();
+        co_await trans_intr::make_interruptible(epm->wait_background());
+      }
+    }
     // user must initialize the logical extent themselves
     assert(is_user_transaction(t.get_src()));
     for (auto& ext : exts) {
@@ -622,6 +663,17 @@ public:
 	  exts.begin(), exts.end()),
 	EXTENT_DEFAULT_REF_COUNT);
     }
+    auto &front = exts.front();
+    if (front->get_write_policy() != write_policy_t::WRITE_THROUGH &&
+        front->get_rewrite_generation() <= epm->get_max_hot_gen()) {
+      auto prefix = front->get_laddr().get_object_prefix();
+      t.touch_laddr_prefix(prefix);
+      if (auto pool = prefix.get_pool();
+          unlikely(hobject_t::is_temp_pool(pool))) {
+        prefix.set_pool(hobject_t::get_logical_pool(pool));
+        t.touch_laddr_prefix(prefix);
+      }
+    }
     for (auto &ext : exts) {
       SUBDEBUGT(seastore_tm, "allocated {}", t, *ext);
     }
@@ -848,6 +900,24 @@ public:
      });
   }
 
+  // non-trivial method, should only be used in DEBUG build
+  bool is_prefix_cached(laddr_t prefix) {
+    if (!logical_bucket) {
+      return true;
+    }
+    return logical_bucket->is_cached(prefix);
+  }
+
+  void update_logical_bucket_for_read(laddr_t prefix) {
+    if (logical_bucket) {
+      logical_bucket->move_to_top(prefix.get_object_prefix());
+    }
+  }
+
+  bool is_cold_device(device_id_t id) const {
+    return epm->is_cold_device(id);
+  }
+
   /**
    * submit_transaction
    *
@@ -900,6 +970,66 @@ public:
     rewrite_gen_t target_generation,
     sea_time_point modify_time) final;
 
+  using ExtentCallbackInterface::rewrite_extents_ret;
+  rewrite_extents_ret rewrite_extents(
+    Transaction &t,
+    std::vector<CachedExtentRef> &extents,
+    rewrite_gen_t target_generation,
+    placement_hint_t hint,
+    sea_time_point modify_time) final;
+
+  using ExtentCallbackInterface::promote_extent_ret;
+  promote_extent_ret promote_extent(
+    Transaction &t,
+    CachedExtentRef extent) final;
+
+  promote_extent_ret promote_extents_from_disk(
+    Transaction &t,
+    paddr_t paddr) final;
+
+  using ExtentCallbackInterface::demote_region_res_t;
+  using ExtentCallbackInterface::demote_region_ret;
+  demote_region_ret demote_region(
+    Transaction &t,
+    laddr_t start,
+    std::size_t max_proceed_size) final;
+
+  maybe_remove_shadow_ret maybe_remove_shadow(
+    Transaction &t,
+    CachedExtent &e) {
+    if (!e.is_logical()) {
+      co_return;
+    }
+    auto &extent = static_cast<LogicalChildNode&>(e);
+    auto cursor = co_await lba_manager->get_cursor(t, extent
+    ).handle_error_interruptible(
+      crimson::ct_error::input_output_error::pass_further{},
+      crimson::ct_error::enoent::assert_failure{}
+    );
+    if (cursor->has_shadow_paddr()) {
+      if (auto shadow = extent.get_shadow(); shadow) {
+        cache->retire_extent(t, shadow);
+      } else {
+        auto laddr = cursor->get_laddr();
+        std::ignore = cache->retire_absent_extent_addr_by_type(
+          t, laddr,
+          cursor->get_shadow_paddr(),
+          cursor->get_length(),
+          cursor->get_extent_type(),
+          [&extent, laddr](auto &ext) {
+            auto lextent = ext.template cast<LogicalChildNode>();
+            assert(ext.is_logical());
+            assert(!lextent->has_laddr());
+            assert(!ext.has_been_invalidated());
+            lextent->set_laddr(laddr);
+            ext.set_shadow_extent(true);
+            extent.set_shadow(lextent);
+          }
+        );
+      }
+    }
+  }
+
   using ExtentCallbackInterface::get_extents_if_live_ret;
   get_extents_if_live_ret get_extents_if_live(
     Transaction &t,
@@ -1073,6 +1203,10 @@ public:
     return epm->get_stat();
   }
 
+  bool is_storage_full() const {
+    return epm->is_storage_full();
+  }
+
   ExtentTransViewRetriever& get_etvr() {
     return *cache;
   }
@@ -1091,6 +1225,7 @@ public:
     if (!mapping.is_indirect() && mapping.is_zero_reserved()) {
       SUBDEBUGT(seastore_tm, "zero reserved, mapping {}, {} remaps",
 		t, mapping, remaps);
+      ceph_assert(!mapping.has_shadow_val());
       //TODO: drop this assert
       assert(mapping.get_extent_type() == extent_types_t::OBJECT_DATA_BLOCK);
       auto type = mapping.get_extent_type();
@@ -1178,6 +1313,9 @@ public:
       return cut_mapping<T>(
 	t, (laddr + aligned_len).checked_to_laddr(), std::move(mapping), false);
     } else {
+      if (mapping.is_linked_direct() && mapping.get_val().is_absolute()) {
+	update_hit_ratio(t, mapping.get_val().get_device_id());
+      }
       return remove(t, std::move(mapping)
       ).handle_error_interruptible(
 	punch_mappings_iertr::pass_further{},
@@ -1282,6 +1420,10 @@ public:
     co_return mapping;
   }
 
+  bool support_logical_bucket() const {
+    return epm->has_cold_tier() && logical_bucket != nullptr;
+  }
+
   ~TransactionManager();
 
 private:
@@ -1294,6 +1436,7 @@ private:
   BackrefManagerRef backref_manager;
 
   WritePipeline write_pipeline;
+  LogicalBucket *logical_bucket;
 
   bool full_extent_integrity_check = true;
 
@@ -1382,8 +1525,19 @@ private:
     assert(cursor.get_pos() != BTREENODE_POS_NULL);
     ceph_assert(t.get_trans_id() == cursor.ctx.trans.get_trans_id());
     auto p = cursor.parent->cast<LBALeafNode>();
-    return p->template get_child<LogicalChildNode>(
+    auto v = p->template get_child<LogicalChildNode>(
       t, cursor.ctx.cache, cursor.get_pos(), cursor.key);
+    if (v.has_child()) {
+      return v.get_child_fut_as<LogicalChildNode>(
+      ).si_then([&t](auto lext) {
+        if (auto shadow = lext->get_shadow();\
+            shadow && shadow->is_stable()) {
+          std::ignore = t.maybe_add_to_read_set(shadow);
+        }
+        return lext;
+      });
+    }
+    return v;
   }
 
   base_iertr::future<LogicalChildNodeRef> read_cursor_by_type(
@@ -1491,7 +1645,7 @@ private:
           SUBTRACET(seastore_tm, "retire extent place holder...", t);
           auto &child_pos = ret.get_child_pos();
           auto laddr = pin.get_key();
-          std::ignore = cache->retire_absent_extent_addr_by_type(
+          auto ext = cache->retire_absent_extent_addr_by_type(
             t, laddr, pin.get_val(), original_len, pin.get_extent_type(),
             [&child_pos, laddr](auto &extent) mutable {
               auto lextent = extent.template cast<LogicalChildNode>();
@@ -1501,7 +1655,23 @@ private:
               child_pos.link_child(lextent.get());
               lextent->set_laddr(laddr);
             }
-          );
+          )->template cast<LogicalChildNode>();
+          if (pin.has_shadow_val()) {
+            cache->retire_absent_extent_addr_by_type(
+              t, pin.get_key(), pin.get_shadow_val(),
+              original_len, pin.get_extent_type(),
+              [laddr, ext](auto &extent) {
+                auto lextent = extent.template cast<LogicalChildNode>();
+                assert(extent.is_logical());
+                assert(!lextent->has_laddr());
+                assert(!extent.has_been_invalidated());
+                lextent->set_laddr(laddr);
+                extent.set_shadow_extent(true);
+                ext->set_shadow(lextent);
+              }
+            );
+          }
+
         }
       }
 
@@ -1525,12 +1695,37 @@ private:
         SUBTRACET(seastore_tm, "retire extent...", t);
         assert(extent->is_seen_by_users());
         cache->retire_extent(t, extent);
+        if (pin.has_shadow_val()) {
+          if (auto shadow = extent->get_shadow()) {
+            cache->retire_extent(t, shadow);
+          } else {
+            auto laddr = pin.get_key();
+            cache->retire_absent_extent_addr_by_type(
+              t, laddr, pin.get_shadow_val(),
+              original_len, pin.get_extent_type(),
+              [laddr, extent](auto &ext) {
+                auto lextent = ext.template cast<LogicalChildNode>();
+                assert(ext.is_logical());
+                assert(!lextent->has_laddr());
+                assert(!ext.has_been_invalidated());
+                lextent->set_laddr(laddr);
+                ext.set_shadow_extent(true);
+                extent->set_shadow(lextent);
+              }
+            );
+          }
+        }
       }
       for (auto &remap : remaps) {
         auto remap_offset = remap.offset;
         auto remap_len = remap.len;
         auto remap_laddr = (original_laddr + remap_offset).checked_to_laddr();
         auto remap_paddr = pin.get_val().add_offset(remap_offset);
+        auto shadow_paddr = P_ADDR_NULL;
+        if (pin.has_shadow_val()) {
+          assert(pin.get_shadow_val() != P_ADDR_NULL);
+          shadow_paddr = pin.get_shadow_val().add_offset(remap_offset);
+        }
         SUBDEBUGT(seastore_tm, "remap direct pin into {}~0x{:x} {} ...",
                   t, remap_laddr, remap_len, remap_paddr);
         ceph_assert(remap_len < original_len);
@@ -1545,6 +1740,21 @@ private:
           remap_offset,
           remap_len,
           original_bptr);
+        if (shadow_paddr != P_ADDR_NULL) {
+          SUBTRACET(seastore_tm, "remap shadow {}", t, shadow_paddr);
+          auto cold_ext = cache->alloc_remapped_extent<T>(
+            t,
+            remap_laddr,
+            shadow_paddr,
+            remap_offset,
+            remap_len,
+            std::nullopt);
+          cold_ext->set_shadow_extent(true);
+          auto &lremapped = static_cast<
+            LogicalChildNode&>(*remapped_extent);
+          auto lcold_ext = cold_ext->template cast<LogicalChildNode>();
+          lremapped.set_shadow(lcold_ext);
+        }
         // user must initialize the logical extent themselves.
         remapped_extent->set_seen_by_users();
         remap.extent = remapped_extent.get();
@@ -1604,9 +1814,11 @@ private:
     Transaction &t,
     LBAMapping mapping);
 
-  rewrite_extent_ret rewrite_logical_extent(
+  rewrite_extent_iertr::future<std::vector<CachedExtentRef>>
+  rewrite_logical_extent(
     Transaction& t,
-    LogicalChildNodeRef extent);
+    LogicalChildNodeRef extent,
+    paddr_t hint);
 
   submit_transaction_direct_ret do_submit_transaction(
     Transaction &t,

src/crimson/osd/main.cc

@@ -264,6 +264,15 @@ int main(int argc, const char* argv[])
               // use a random osd uuid if not specified
               osd_uuid.generate_random();
             }
+            if (auto c = local_conf().get_val<uint64_t>("seastore_cold_devices_count");
+                c != 0) {
+              auto root = local_conf().get_val<std::string>("osd_data");
+              for (size_t i = 1; i <= c; i++) {
+                auto path = fmt::format("{}/block.{}", root, i);
+                seastar::touch_directory(path).get();
+              }
+              seastar::sync_directory(root).get();
+            }
             osd.mkfs(
 	      *store,
 	      whoami,

src/crimson/osd/ops_executer.cc

@@ -609,6 +609,28 @@ OpsExecuter::do_execute_op(OSDOp& osd_op)
     "handling op {} on object {}",
     ceph_osd_op_name(osd_op.op.op),
     get_target());
+  // Best-effort capacity gate (failsafe): drop data-allocating ops before a
+  // transaction is created if the local store is full. This is Crimson's
+  // analog to classic's osd_failsafe_full_ratio check (PrimaryLogPG.cc:2162),
+  // preventing allocator exhaustion assert aborts when the local store fills
+  // before the monitor's pool FLAG_FULL has propagated. We reply -EAGAIN so
+  // the client resends, matching both classic's silent drop and Crimson's own
+  // mon-driven full path (PG::run_executer_fut, pg.cc), rather than returning
+  // a result that depends on whether full flags have reached the client yet.
+  switch (osd_op.op.op) {
+  case CEPH_OSD_OP_CREATE:
+  case CEPH_OSD_OP_WRITE:
+  case CEPH_OSD_OP_WRITEFULL:
+  case CEPH_OSD_OP_WRITESAME:
+  case CEPH_OSD_OP_APPEND:
+  case CEPH_OSD_OP_COPY_FROM2:
+    if (pg->is_local_store_full() && !get_message().has_flag(CEPH_OSD_FLAG_FULL_TRY)) {
+      return crimson::ct_error::eagain::make();
+    }
+    break;
+  default:
+    break;
+  }
   switch (const ceph_osd_op& op = osd_op.op; op.op) {
   case CEPH_OSD_OP_SYNC_READ:
     [[fallthrough]];

src/crimson/osd/pg.h

@@ -240,6 +240,10 @@ public:
     return false;
   }
 
+  bool is_local_store_full() const {
+    return shard_services.is_local_storage_full(get_store_index());
+  }
+
   epoch_t get_last_peering_reset_epoch() const final {
     return get_last_peering_reset();
   }
@@ -269,7 +273,7 @@ public:
     return pgid;
   }
 
-  const unsigned int get_store_index() {
+  unsigned get_store_index() const {
     return store_index;
   }
   PGBackend& get_backend() {

src/crimson/osd/shard_services.h

@@ -533,6 +533,12 @@ public:
     return store;
   }
 
+  bool is_local_storage_full(store_index_t store_index) const {
+    const auto &shard_store =
+        local_state.b_store.f_store.get_sharded_store(store_index);
+    return shard_store.is_storage_full();
+  }
+
   struct shard_stats_t {
     double reactor_utilization;
   };

src/crimson/tools/store_nbd/tm_driver.cc

@@ -168,7 +168,10 @@ seastar::future<> TMDriver::mkfs()
 {
   assert(config.path);
   logger().debug("mkfs");
-  return Device::make_device(*config.path, device_type_t::SSD
+  return Device::make_device(
+    *config.path,
+    device_type_t::SSD,
+    backend_type_t::SEGMENTED
   ).then([this](DeviceRef dev) {
     device = std::move(dev);
     seastore_meta_t meta;
@@ -176,9 +179,12 @@ seastar::future<> TMDriver::mkfs()
     return device->mkfs(
       device_config_t{
         true,
-        (magic_t)std::rand(),
-        device_type_t::SSD,
-        0,
+        device_spec_t{
+	  (magic_t)std::rand(),
+	  device_type_t::SSD,
+	  backend_type_t::SEGMENTED,
+	  0
+	},
         meta,
         secondary_device_set_t()});
   }).safe_then([this] {
@@ -210,7 +216,10 @@ seastar::future<> TMDriver::mount()
 {
   return (config.mkfs ? mkfs() : seastar::now()
   ).then([this] {
-    return Device::make_device(*config.path, device_type_t::SSD);
+    return Device::make_device(
+      *config.path,
+      device_type_t::SSD,
+      backend_type_t::SEGMENTED);
   }).then([this](DeviceRef dev) {
     device = std::move(dev);
     return device->mount();

src/librbd/api/Group.cc

@@ -822,7 +822,10 @@ int Group<I>::image_list(librados::IoCtx& group_ioctx,
 
   std::vector<cls::rbd::GroupImageStatus> image_ids;
 
-  group_image_list(group_ioctx, group_name, &image_ids);
+  int r = group_image_list(group_ioctx, group_name, &image_ids);
+  if (r < 0) {
+    return r;
+  }
 
   for (auto image_id : image_ids) {
     IoCtx ioctx;

src/librbd/librbd.cc

@@ -5700,10 +5700,6 @@ extern "C" int rbd_snap_list(rbd_image_t image, rbd_snap_info_t *snaps,
   memset(snaps, 0, sizeof(*snaps) * *max_snaps);
 
   int r = librbd::api::Snapshot<>::list(ictx, cpp_snaps);
-  if (r == -ENOENT) {
-    tracepoint(librbd, snap_list_exit, 0, *max_snaps);
-    return 0;
-  }
   if (r < 0) {
     tracepoint(librbd, snap_list_exit, r, *max_snaps);
     return r;
@@ -7266,12 +7262,6 @@ extern "C" int rbd_group_image_list(rados_ioctx_t group_p,
   int r = librbd::api::Group<>::image_list(group_ioctx, group_name,
                                            &cpp_images);
 
-  if (r == -ENOENT) {
-    tracepoint(librbd, group_image_list_exit, 0);
-    *image_size = 0;
-    return 0;
-  }
-
   if (r < 0) {
     tracepoint(librbd, group_image_list_exit, r);
     return r;
@@ -7447,12 +7437,6 @@ extern "C" int rbd_group_snap_list(rados_ioctx_t group_p,
   int r = librbd::api::Group<>::snap_list(group_ioctx, group_name, true, false,
                                           &cpp_snaps);
 
-  if (r == -ENOENT) {
-    *snaps_size = 0;
-    tracepoint(librbd, group_snap_list_exit, 0);
-    return 0;
-  }
-
   if (r < 0) {
     tracepoint(librbd, group_snap_list_exit, r);
     return r;

src/pybind/mgr/dashboard/frontend/cypress/e2e/pools/pools.po.ts

@@ -64,6 +64,32 @@ export class PoolPageHelper extends PageHelper {
       .should('have.value', bpsLimit);
   }
 
+  private ensurePoolDeletionEnabled(name: string) {
+    this.clickRowActionButton(name, 'delete');
+    cy.get('cds-modal').then(($modal) => {
+      if ($modal.text().includes("Can't delete")) {
+        cy.get('cds-modal button').contains('Enable').click({ force: true });
+        cy.get('cds-modal').should('not.exist');
+      } else {
+        cy.get('cds-modal .cds--modal-close-button button').click({ force: true });
+        cy.get('cds-modal').should('not.exist');
+      }
+    });
+  }
+
+  delete(
+    name: string,
+    columnIndex?: number,
+    section?: string,
+    cdsModal = true,
+    isMultiselect = false,
+    shouldReload = false,
+    confirmInput = true
+  ) {
+    this.ensurePoolDeletionEnabled(name);
+    super.delete(name, columnIndex, section, cdsModal, isMultiselect, shouldReload, confirmInput);
+  }
+
   private setApplications(apps: string[]) {
     if (!apps || apps.length === 0) {
       return;

src/pybind/mgr/dashboard/frontend/cypress/e2e/rgw/users.po.ts

@@ -9,13 +9,37 @@ export class UsersPageHelper extends PageHelper {
   pages = pages;
 
   @PageHelper.restrictTo(pages.create.url)
-  create(tenant: string, user_id: string, fullname: string, email: string, maxbuckets: string) {
+  create(
+    tenant: string,
+    user_id: string,
+    fullname: string,
+    email: string,
+    maxbuckets: string,
+    account_id?: string,
+    selectPolicy: boolean = false
+  ) {
     // Enter in user_id
     cy.get('#user_id').type(user_id);
     // Show Tenant
     cy.get('#show_tenant_input').click({ force: true });
     // Enter in tenant
     cy.get('#tenant').type(tenant);
+
+    if (account_id) {
+      cy.get('select#link_account').select(account_id);
+      if (selectPolicy) {
+        // Wait for the policies combo box to appear (it only shows when account is linked and not root user)
+        cy.get('cds-combo-box#account_policies').scrollIntoView().should('be.visible');
+        cy.get('cds-combo-box#account_policies input[type="text"]').scrollIntoView().click();
+        cy.get('cds-combo-box#account_policies cds-dropdown-list ul[role="listbox"]')
+          .should('be.visible')
+          .find('li[role="option"]')
+          .first()
+          .find('input[type="checkbox"]')
+          .check({ force: true });
+      }
+    }
+
     // Enter in full name
     cy.get('#display_name').click().type(fullname);
 
@@ -179,6 +203,16 @@ export class UsersPageHelper extends PageHelper {
       'contain.text',
       `${account_name} - ${tenant}`
     );
+
+    cy.get('cds-combo-box#account_policies').scrollIntoView().should('be.visible');
+    cy.get('cds-combo-box#account_policies input[type="text"]').scrollIntoView().click();
+    cy.get('cds-combo-box#account_policies cds-dropdown-list ul[role="listbox"]')
+      .should('be.visible')
+      .find('li[role="option"]')
+      .first()
+      .find('input[type="checkbox"]')
+      .check({ force: true });
+
     cy.contains('button', 'Edit User').click();
 
     this.getTableRow(tenant + '$' + user_id).as('AccountUser');

src/pybind/mgr/dashboard/frontend/src/app/ceph/block/block.module.ts

@@ -109,7 +109,7 @@ import { NvmeSubsystemViewBreadcrumbResolver } from './nvme-subsystem-view/nvme-
 import { NvmeSubsystemViewComponent } from './nvme-subsystem-view/nvme-subsystem-view.component';
 import { NvmeofSubsystemPerformanceComponent } from './nvmeof-subsystem-performance/nvmeof-subsystem-performance.component';
 import { NvmeofTabsComponent } from './nvmeof-tabs/nvmeof-tabs.component';
-import { NvmeofGatewayGroupDeleteGuardModalComponent } from './nvmeof-gateway-group/nvmeof-gateway-group-delete-guard-modal.component';
+
 import { NvmeofSetupCardsComponent } from './nvmeof-setup-cards/nvmeof-setup-cards.component';
 import { NvmeofGatewayGroupFilterComponent } from './nvmeof-gateway-group-filter/nvmeof-gateway-group-filter.component';
 import { NvmeofEditAuthenticationComponent } from './nvmeof-edit-authentication/nvmeof-edit-authentication.component';
@@ -207,7 +207,6 @@ import { NvmeofEditAuthenticationComponent } from './nvmeof-edit-authentication/
     NvmeofSubsystemOverviewComponent,
     NvmeofSubsystemPerformanceComponent,
     NvmeofTabsComponent,
-    NvmeofGatewayGroupDeleteGuardModalComponent,
     NvmeofEditAuthenticationComponent
   ],
 

src/pybind/mgr/dashboard/frontend/src/app/ceph/block/iscsi-target-list/iscsi-target-list.component.html

@@ -3,8 +3,8 @@
 <cd-alert-panel
   type="info"
   *ngIf="available === false"
-  title="iSCSI Targets not available"
-  i18n-title
+  alertTitle="iSCSI Targets not available"
+  i18n-alertTitle
 >
   <ng-container i18n
     >Please consult the&nbsp;<cd-doc section="iscsi"></cd-doc>&nbsp;on how to configure and enable

src/pybind/mgr/dashboard/frontend/src/app/ceph/block/nvmeof-edit-authentication/nvmeof-edit-authentication.component.html

@@ -13,8 +13,8 @@
     @if (showAuthAlert) {
       <cd-alert-panel
         type="warning"
-        title="Subsystem DHCHAP Key will be deleted permanently"
-        i18n-title
+        alertTitle="Subsystem DHCHAP Key will be deleted permanently"
+        i18n-alertTitle
         class="cd-nvmeof-edit-auth-downgrade-alert"
       >
         <span i18n>

src/pybind/mgr/dashboard/frontend/src/app/ceph/block/nvmeof-gateway-group/nvmeof-gateway-group-delete-guard-modal.component.html

@@ -1,51 +0,0 @@
-<cds-modal
-  size="sm"
-  [open]="open"
-  (overlaySelected)="closeModal()"
->
-  <cds-modal-header (closeSelect)="closeModal()">
-    <h2
-      cdsModalHeaderLabel
-      modal-primary-focus
-      i18n
-    >
-      Manage {{ gatewayName }}
-    </h2>
-    <span
-      class="cds--type-heading-03"
-      cdsModalHeaderHeading
-      i18n
-      >Can't delete {{ gatewayName }}</span
-    >
-  </cds-modal-header>
-  <section cdsModalContent>
-    <p
-      class="cds--type-body-01 cds--mb-05"
-      i18n
-    >
-      This resource has connected items that must be deleted first. Delete the connected items, and
-      try again.
-    </p>
-    <p
-      class="cds--type-label-01 cds--mb-04"
-      i18n
-    >
-      View connected items:
-    </p>
-    <div>
-      @for (sub of connectedSubsystems; track sub.nqn) {
-        <div class="cds--mb-04">
-          <a
-            class="cds--link cds--type-body-01"
-            tabindex="0"
-            (click)="navigateToSubsystem(sub.nqn)"
-            (keydown.enter)="navigateToSubsystem(sub.nqn)"
-          >
-            <span class="cds--mr-03">{{ sub.nqn }}</span>
-            <cd-icon type="launch"></cd-icon>
-          </a>
-        </div>
-      }
-    </div>
-  </section>
-</cds-modal>