gian/ceph-mirror
1
0
Fork 0
mirror of https://github.com/ceph/ceph synced 2026-08-02 07:03:18 +00:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
163,484 Commits 389 Branches 411 Tags 2.4 GiB
e89e2104a8
Commit Graph

163484 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Xuehan Xu
e89e2104a8 crimson/os/seastore/epm: reserve the space in the cold tier before 
demoting

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:20 +08:00
Xuehan Xu
528bba37b9 crimson/os/seastore/epm: add try_reserve_main and abort_main_usage 
As the counterpart of try_reserve_cold and abort_cold_usage

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:20 +08:00
Xuehan Xu
953710c5e6 crimson/os/seastore/transaction_manager: add the shadow to the trans' 
read_set when getting extents

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:20 +08:00
Xuehan Xu
ba0655657c crimson/os/seastore/transaction: mutation_pending extents' paddrs should 
be on the same device as the stable prior's old paddr

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:20 +08:00
Xuehan Xu
64263d609d crimson/os/seastore/journal/segmented_journal: scan the journal for the 
allocation map on boot if the cold tier is an RBM one

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:20 +08:00
Xuehan Xu
acdb292361 crimson/os/seastore/journal: refactor SegmentedJournal::replay() to 
adapt to Journal::scan_valid_record_delta()

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:20 +08:00
Xuehan Xu
f56b2a5f42 crimson/os/seastore/journal: coroutinize SegmentedJournal::replay() 
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:20 +08:00
Xuehan Xu
aad3524f7a crimson/os/seastore/journal: lift the alloc_map scanning from CircularBoundedJournal to Journal 
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Xuehan Xu
92a569aa4a crimson/os/seastore/btree: define fmt::formatter for lba_val_map_t in 
btree_types.h

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Xuehan Xu
aba5a77872 crimson/os/seastore/lba: make sure all mappings in the copied range is 
examined when updating paddrs for the copied mappings

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Xuehan Xu
b899ade928 crimson/os/seastore/lba: add more rigorous assertions 
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Xuehan Xu
f18fe72baa crimson/os/seastore/transaction_manager: indicate the extents as COLD 
when demoting them

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Xuehan Xu
43f048c81e crimson/os/seastore/epm: avoid adjust generation for eviction if the 
main device is not SEGMENTED

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Xuehan Xu
0a0a7e3da0 crimson/os/seastore/cache: allow LogNode to be remapped 
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Xuehan Xu
a4f338e77f crimson/os/seastore: also update lba mappings' shadow fields when 
updating paddr synchronously

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Xuehan Xu
6db2d97c74 crimson/os/seastore/lba: merge the whole mapping into the delta 
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Xuehan Xu
e141ac3f38 crimson/os/seastore/extent_pinboard: don't add extents under promotion 
to cache

This is to avoid an under-promotion extent from being added back to and
evicted from the cache again before the promotion ends.

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Xuehan Xu
a5693801a6 crimson/os/seastore/lba,TM: remove shadows when trimming dirty non-data 
extents

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Xuehan Xu
a9220d9470 crimson/os/seastore/cached_extent: let pending cached extents hold pointers to the transactions 
This is necessary for the promote transaction to find the transactions
that are retiring the extents promoted

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Xuehan Xu
e3c1baf8c8 crimson/os/seastore: demote/promote background processes are also 
rewrite transactions

So we need to merge the extents promoted/demoted with their counterparts
in client transactions, just like trim/cleaner transaction do.

The difference between demote/promote transactions and trim/cleaner
transactions is that the former also involves remapping extents while the
latter only involves rewriting and mutating extents.

With the logical bucket cache in place, all rewrite transactions need to
handle shadow extents carafully, too.

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Xuehan Xu
eeba1411a2 crimson/os/seastore/transaction_manager: wait for the demotion when the 
rbm main device is not enough

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Xuehan Xu
e4979f98ed crimson/os/seastore: drop backref when all backends are RBMs 
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Xuehan Xu
c8769a0f34 qa/config/crimson_qa_overrides.yaml: add test workload for LBC 
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Xuehan Xu
660c7d86f4 crimson/os/seastore/cache: count promote/demote related metrics 
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Xuehan Xu
c48848ba9f test/crimson/seastore/test_transaction_manager: the scatter_allocation case shouldn't submit the trans 
In scatter_allocation, an extent is split into several
small ones, and the last allocation is supposed to fail,
in which case, the small extents that belong to the same
allocation wouldn't be attached to the parent lba leaf
node. Sumitting the transaction would trigger assertions,
and is not consistent with the behavior of nornal crimson
OSDs.

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Zhang Song
5314e38d39 crimson/os/seastore: add test workload to promote/evict aggressively 
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Xuehan Xu
d86fb5fd80 crimson/os/seastore/random_block_manager: try to allocate consecutive rbm space when rewriting extents 
Add the address of the last allocation as the hint to the current round
of allocation in rbm

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Zhang Song
eb9e9b9d06 crimson/os/seastore/cache: add hit ratio metric 
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Zhang Song
2a5257a04c crimson/os/seastore: make cold rbm cleaner use different metric prefix 
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Zhang Song
a8b0baca91 crimson/os/seastore: allow RBM backend as hot tier 
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Zhang Song
c969a8b9b5 crimson/os/seastore: update logical bucket cache when processing 
read/write transactions

All client accessed extents are to be tracked by logical bucket cache
before eviction

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Zhang Song
afde4c6dcb crimson/os/seastore: add write through policy 
Write extents larger than some threshold to the cold tier directly

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Zhang Song
43d0b3316a crimson/osd: create cold devices when mkfs 
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-31 09:40:19 +08:00
Zhang Song
98f1892fb6 crimson/os/seastore: throttle bandwidth to secondary devices 
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-30 10:12:27 +08:00
Zhang Song
223dde6bab crimson/os/seastore: introduce RotationalDevice 
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-30 10:12:27 +08:00
Zhang Song
eca32e1fa9 crimson/os/seastore: set backend type explicitly 
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-30 10:12:27 +08:00
Zhang Song
051db088b3 crimson/os/seastore/EPM/BackgroundProcess: add promote and demote process 
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-30 10:12:27 +08:00
Zhang Song
91eb5d2a9c crimson/os/seastore/EPM: initialize logical bucket and pinboard 
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-30 10:12:27 +08:00
Zhang Song
c7b3fd1350 crimson/os/seastore: don't add shadow extent to cache 
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-30 10:12:26 +08:00
Zhang Song
3f2b958478 crimson/os/seastore: avoid evicting promoted extent to the cold tier 
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-30 10:12:26 +08:00
Zhang Song
d088cfc46d crimson/os/seastore: consolidate parameters on extent allocation path 
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-30 10:12:26 +08:00
Zhang Song
1af4a97792 crimson/os/seastore: implement demote region 
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-30 10:12:26 +08:00
Zhang Song
c3e971a84c crimson/os/seastore: implement promote extent 
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-30 10:12:26 +08:00
Zhang Song
ad7469ca59 crimson/os/seastore/lba: introduce shadow paddr 
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-30 10:12:26 +08:00
Zhang Song
671b1b4ffc crimson/os/seastore: introduce logical bucket and demote process 
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-30 10:12:26 +08:00
Zhang Song
2d629af107 crimson/os/seastore: support promote extents purged from pinboard 
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-30 10:12:26 +08:00
Zhang Song
6c526c811b crimson/os/seastore: rename extent_2q_state_t to extent_pin_state_t 
This is because, even for LRU caches, extents still have pin states,
although it's only Fresh. What's more, in the logical bucket cache
world, extents in both LRU and 2Q caches may be in the
Fresh/PendingPromote/Promoting state.

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-30 10:12:26 +08:00
Zhang Song
ece1a68497 crimson/os/seastore: introduce PROMOTE and DEMOTE transaction type 
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-30 10:12:26 +08:00
Xuehan Xu
f17f12b4a1 doc/dev/crimson: add design the document for LogicalBucketCache 
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
 2026-07-30 10:12:26 +08:00
mheler
e82ccd0873
Merge pull request #70051 from mheler/wip-copyobject-replace 2026-07-29 13:30:53 -05:00