Compare commits

...

1648 Commits

Author SHA1 Message Date
Shai Fultheim
a8eaf2e9d2 crimson/seastore/rbm: fix leakage on conflicted OOL writes
Problem:
In-flight out-of-line (OOL) writes did not clear t.pending_ool on successful
completion. If the transaction was subsequently marked conflicted during
commit/prepare, it erroneously marked the completed write as conflicted.
This caused:
1. conflict_pending_free_bytes to leak upward.
2. Physical extents to remain allocated but untracked, leaking space.
Eventually, this led to allocator exhaustion and OSD deadlock.

Solution:
Clear t.pending_ool on OOL write completion and transaction reset. Implement
a backpressure condition variable for pending deferred frees so that write
operations wait when accumulated conflicted frees exceed a 5% threshold.

Change:
* Added clear_pending_ool() to Transaction and called it from the
  do_write() finally continuation in RandomBlockOolWriter::
  alloc_write_ool_extents, so it runs on every exit path (success,
  error, or conflict-driven unwind) and the conflicted check plus the
  clear are atomic with respect to reactor task interleaving.
* Cleared pending_ool in Transaction::reset() to prevent leaks across
  transaction reuse/retry.
* Tracked conflicted pending free bytes in RBMCleaner.
* Added wait_for_conflict_drain() to pause new OOL writes when deferred
  frees exceed 5% of data capacity.

The pending deferred frees are also a release source for the reservation
gate introduced by the preceding commit: account them in
try_reserve_projected_usage()'s release-source check, and wake blocked IO
when they drain so the gate is re-evaluated.

Signed-off-by: Shai Fultheim <shai.fultheim@gmail.com>
2026-08-01 02:52:48 +03:00
Shai Fultheim
3804ef0dd1 crimson/seastore/rbm: gate writes and space reserve on allocator state
Problem:
Using the stats.used_bytes counter for RBM capacity check is unreliable.
Unmetered data paths (such as alloc_paddr or overwrites) cause it to drift
and diverge from the physical block allocator state. This leads to
premature ENOSPC rejections or capacity overcommits that crash the OSD.

Solution:
Implement is_storage_full() and try_reserve_projected_usage() using the
ground truth of physical allocator state (AVL tree's available size).
This replaces the interim statfs-based is_storage_full() failsafe check
introduced by the preceding OSDOp gating commit with the allocator's
authoritative view, as anticipated there.

Change:
* Updated BlockRBManager::get_free_blocks() to return the actual count
  of free blocks tracked by the AVL allocator.
* Implemented get_allocator_used_bytes() in RBMCleaner to compute the
  total allocated size across all RBM devices.
* Changed try_reserve_projected_usage() to query the allocator state
  instead of stats.used_bytes, and reduced headroom to 1%.
* Overrode the virtual is_storage_full() hook in RBMCleaner to trigger
  boundary gating using the same allocator state.

* Guarded is_storage_full() to return false until the background process
  is ready (mirroring the is_ready assertion in try_reserve_projected_usage),
  so capacity cannot be misreported as full while the allocator is still
  being populated during mount.

* Gate only while a release source exists: RBM has no cleaner, so a
  failed reservation can only be woken by an in-flight transaction
  releasing its projected usage. If none exists, admit the IO instead of
  blocking (which would deadlock, e.g. a sole writer on a full device)
  and let the block allocator be the final authority: a genuinely full
  device fails the allocation with ENOSPC through the existing
  alloc-failure path. Zero-usage reservations are always admitted: they
  cannot overcommit, and deletes -- the space-releasing transactions --
  are metadata-only.

* Report statfs (get_stat) and the rbm_cleaner metrics from the same
  allocator view via a get_used_bytes() helper, so operator-visible
  capacity matches the admission gating; both fall back to
  stats.used_bytes until the allocator is populated during mount.

Signed-off-by: Shai Fultheim <shai.fultheim@gmail.com>
2026-08-01 02:50:37 +03:00
Shai Fultheim
0c4cba1010 test/crimson/seastore: exercise the failsafe-full threshold
Add storage_full_failsafe_threshold to the RBM-backed transaction
manager tests: a fresh store reports not-full at the default ratio,
real extent allocations flip is_storage_full() once the used fraction
crosses osd_failsafe_full_ratio, and restoring the default ratio
clears the condition, verifying the threshold tracks the config knob.

Signed-off-by: Shai Fultheim <shai.fultheim@gmail.com>
2026-08-01 01:17:14 +03:00
Shai Fultheim
f7bff3598d crimson/os/seastore: derive failsafe-full threshold from osd_failsafe_full_ratio
RBMCleaner::is_storage_full() hardcoded a 3% free limit. Read the same
config classic uses for its failsafe check (osd_failsafe_full_ratio,
default 0.97) instead, treating the store as full once the free fraction
drops below (1 - ratio), so the threshold tracks the knob rather than a
magic constant.

Signed-off-by: Shai Fultheim <shai.fultheim@gmail.com>
2026-08-01 01:17:14 +03:00
Shai Fultheim
365f478dae crimson/osd: gate writes at OSDOp boundary when local store is full
When SeaStore runs out of space, allocations in object_data_handler.cc hit
enospc::assert_failure and abort the OSD. Gating on monitor OSDMap pool FULL
flags alone is insufficient: flag propagation lags by seconds-to-minutes and
cannot protect the local allocator from sudden exhaustion.

Add a local failsafe gate at the OpsExecuter boundary, Crimson's analog of
classic's osd_failsafe_full_ratio check (PrimaryLogPG.cc:2162). Data-allocating
ops (CREATE/WRITE/WRITEFULL/WRITESAME/APPEND/COPY_FROM2) are dropped with
-EAGAIN so the client resends, matching Crimson's existing mon-driven full path
(the "drop request" case in pg.cc) and classic's silent drop. This keeps
behavior uniform regardless of whether pool full flags have reached the client
yet. CEPH_OSD_FLAG_FULL_TRY is exempt; space-reclaiming ops
(ZERO/TRUNCATE/ROLLBACK) are deliberately not gated. The mon-driven pool
FLAG_FULL path in run_executer is untouched and remains authoritative.

The threshold uses the store's statfs accounting (stats.used_bytes). The
follow-on (#69352) moves RBM capacity checks to the allocator's authoritative
view and wires mon-side NEARFULL/FULL via statfs, so clients get a clean stop
rather than resend-until-resolved.

Signed-off-by: Shai Fultheim <shai.fultheim@gmail.com>
2026-08-01 01:17:14 +03:00
Shilpa Jagannath
8482cb11bc
Merge pull request #70510 from smanjara/wip-shilpa-test_zonegroup_rename
qa/tests: fix flaky multisite tests
2026-07-31 10:20:58 -07:00
John Mulligan
ac9a88d3ed
Merge pull request #70679 from phlogistonjohn/jjm-smb-etc-altpaths
smb: support altpaths for updating non-AD users/groups live while a cluster is running

Reviewed-by: Anoop C S <anoopcs@cryptolab.net>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
2026-07-31 10:09:31 -04:00
Jaya Prakash
3434180fc5
Merge pull request #70492 from Jayaprakash-ibm/wip-jaya-spl-clr-docs-ext
doc/rados/bluestore: Documentation for Interpreting spillover cleaner stats

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>
2026-07-31 18:59:17 +05:30
Ilya Dryomov
2caabd7a28
Merge pull request #70729 from ivancich/wip-fix-ceph-spec-in
rgw: revert debian portion of update build config to streamline builds

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2026-07-31 14:01:47 +02:00
Jaya Prakash
2c452108d3 doc/rados/bluestore: Documentation for Interpreting spillover cleaner stats
Signed-off-by: Jaya Prakash <jayaprakash@ibm.com>
2026-07-31 16:59:09 +05:30
Nizamudeen A
0152782bfe
Merge pull request #70723 from rhcs-dashboard/rename-tab
mgr/dashboard: Title says "Gateways" but it displays gateway group names in NVME/TCP

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-31 14:46:44 +05:30
Ilya Dryomov
b1c466c2eb
Merge pull request #70086 from VinayBhaskar-V/wip-group-image-list
librbd: propagate ENOENT for non-existent groups and images

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2026-07-31 11:14:08 +02:00
J. Eric Ivancich
cdbd4dabe9 rgw: revert debian portion of update build config to streamline builds
The debian portion broke the build. Reverting a portion of
c3c4a139ad.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2026-07-30 14:31:54 -04:00
pujashahu
0323ff2141 mgr/dashboard: Title says "Gateways" but it displays gateway group names in NVME/TCP
Fixes: https://tracker.ceph.com/issues/75443
Signed-off-by: pujaoshahu <pshahu@redhat.com>
2026-07-30 21:35:18 +05:30
Gil Bregman
3d9aa9f329
Merge pull request #70705 from gbregman/main
mgr/dashboard: Add --keep-connections parameter to NVMEoF CLI
2026-07-30 16:32:14 +03:00
Patrick Donnelly
218983d0cf
Merge PR #70689 into main
* refs/pull/70689/head:
	.github/workflows: switch to pull_request_target trigger

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2026-07-30 09:29:44 -04:00
Patrick Donnelly
31519526f7
Merge PR #70690 into main
* refs/pull/70690/head:
	.github/workflows: switch to pull_request_target trigger

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2026-07-30 09:29:20 -04:00
Patrick Donnelly
c2afa96e8e
Merge PR #70695 into main
* refs/pull/70695/head:
	.githubmap: add myself

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-30 09:15:08 -04:00
Patrick Donnelly
beccebe2e8
Merge PR #70683 into main
* refs/pull/70683/head:
	script/ptl-tool: add 'crimson', 'build/ops', 'common', 'tests' to SUPPORTED_QA_TAGS

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-30 08:30:37 -04:00
Gil Bregman
fd39161c7f mgr/dashboard: Add --keep-connections parameter to NVMEoF CLI
Fixes: https://tracker.ceph.com/issues/78833

Signed-off-by: Gil Bregman <gbregman@il.ibm.com>
2026-07-30 14:56:12 +03:00
Venky Shankar
6beaf71d96 Merge PR #69835 into main
* refs/pull/69835/head:
	mds/Server: return after responding on error paths

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2026-07-30 14:01:37 +05:30
Alexander Indenbaum
660ba02205 .githubmap: add myself
Signed-off-by: Alexander Indenbaum <aindenba@redhat.com>
2026-07-30 10:41:40 +03:00
bluikko
4e380d304b
Merge pull request #70663 from bluikko/wip-doc-install-ref-links
doc/install: use ref for intra-docs links
2026-07-30 12:00:47 +07:00
Kotresh HR
10292cc86f Merge PR #70212 into main
* refs/pull/70212/head:
	qa/cephfs-mirror: Fix tests for ISO sync_time_stamp
	doc/cephfs-mirroring: document ISO-8601 sync timestamps
	cephfs_mirror: format sync timestamps as ISO-8601 local time
	tools/cephfs-mirror: report epoch time in last snap sync time

Reviewed-by: Karthik U S <karthik.u.s1@ibm.com>
2026-07-30 09:28:12 +05:30
Xuehan Xu
ab4a964a1c
Merge pull request #56650 from zhscn/dev-lbc
crimson/os/seastore: introduce logical bucket cache

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Kefu Chai <k.chai@proxmox.com>
Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2026-07-31 11:28:31 +08:00
Xuehan Xu
b8f9beb9c1 crimson/os/seastore/transaction_manager: don't demote dirty extents
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:20 +08:00
Xuehan Xu
6b6c8d4da3 crimson/os/seastore/transaction_manager: allow non-existing lba regions
when demoting them

It is possible that, when we do demote_region, the target region has
already been removed from the lba tree, for example, a temp recovering
object might have already been renamed to the real one, which means all
lba mappings within its original region have been moved away.

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:20 +08:00
Xuehan Xu
e89e2104a8 crimson/os/seastore/epm: reserve the space in the cold tier before
demoting

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:20 +08:00
Xuehan Xu
528bba37b9 crimson/os/seastore/epm: add try_reserve_main and abort_main_usage
As the counterpart of try_reserve_cold and abort_cold_usage

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:20 +08:00
Xuehan Xu
953710c5e6 crimson/os/seastore/transaction_manager: add the shadow to the trans'
read_set when getting extents

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:20 +08:00
Xuehan Xu
ba0655657c crimson/os/seastore/transaction: mutation_pending extents' paddrs should
be on the same device as the stable prior's old paddr

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:20 +08:00
Xuehan Xu
64263d609d crimson/os/seastore/journal/segmented_journal: scan the journal for the
allocation map on boot if the cold tier is an RBM one

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:20 +08:00
Xuehan Xu
acdb292361 crimson/os/seastore/journal: refactor SegmentedJournal::replay() to
adapt to Journal::scan_valid_record_delta()

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:20 +08:00
Xuehan Xu
f56b2a5f42 crimson/os/seastore/journal: coroutinize SegmentedJournal::replay()
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:20 +08:00
Xuehan Xu
aad3524f7a crimson/os/seastore/journal: lift the alloc_map scanning from CircularBoundedJournal to Journal
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Xuehan Xu
92a569aa4a crimson/os/seastore/btree: define fmt::formatter for lba_val_map_t in
btree_types.h

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Xuehan Xu
aba5a77872 crimson/os/seastore/lba: make sure all mappings in the copied range is
examined when updating paddrs for the copied mappings

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Xuehan Xu
b899ade928 crimson/os/seastore/lba: add more rigorous assertions
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Xuehan Xu
f18fe72baa crimson/os/seastore/transaction_manager: indicate the extents as COLD
when demoting them

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Xuehan Xu
43f048c81e crimson/os/seastore/epm: avoid adjust generation for eviction if the
main device is not SEGMENTED

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Xuehan Xu
0a0a7e3da0 crimson/os/seastore/cache: allow LogNode to be remapped
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Xuehan Xu
a4f338e77f crimson/os/seastore: also update lba mappings' shadow fields when
updating paddr synchronously

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Xuehan Xu
6db2d97c74 crimson/os/seastore/lba: merge the whole mapping into the delta
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Xuehan Xu
e141ac3f38 crimson/os/seastore/extent_pinboard: don't add extents under promotion
to cache

This is to avoid an under-promotion extent from being added back to and
evicted from the cache again before the promotion ends.

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Xuehan Xu
a5693801a6 crimson/os/seastore/lba,TM: remove shadows when trimming dirty non-data
extents

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Xuehan Xu
a9220d9470 crimson/os/seastore/cached_extent: let pending cached extents hold pointers to the transactions
This is necessary for the promote transaction to find the transactions
that are retiring the extents promoted

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Xuehan Xu
e3c1baf8c8 crimson/os/seastore: demote/promote background processes are also
rewrite transactions

So we need to merge the extents promoted/demoted with their counterparts
in client transactions, just like trim/cleaner transaction do.

The difference between demote/promote transactions and trim/cleaner
transactions is that the former also involves remapping extents while the
latter only involves rewriting and mutating extents.

With the logical bucket cache in place, all rewrite transactions need to
handle shadow extents carafully, too.

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Xuehan Xu
eeba1411a2 crimson/os/seastore/transaction_manager: wait for the demotion when the
rbm main device is not enough

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Xuehan Xu
e4979f98ed crimson/os/seastore: drop backref when all backends are RBMs
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Xuehan Xu
c8769a0f34 qa/config/crimson_qa_overrides.yaml: add test workload for LBC
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Xuehan Xu
660c7d86f4 crimson/os/seastore/cache: count promote/demote related metrics
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Xuehan Xu
c48848ba9f test/crimson/seastore/test_transaction_manager: the scatter_allocation case shouldn't submit the trans
In scatter_allocation, an extent is split into several
small ones, and the last allocation is supposed to fail,
in which case, the small extents that belong to the same
allocation wouldn't be attached to the parent lba leaf
node. Sumitting the transaction would trigger assertions,
and is not consistent with the behavior of nornal crimson
OSDs.

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Zhang Song
5314e38d39 crimson/os/seastore: add test workload to promote/evict aggressively
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Xuehan Xu
d86fb5fd80 crimson/os/seastore/random_block_manager: try to allocate consecutive rbm space when rewriting extents
Add the address of the last allocation as the hint to the current round
of allocation in rbm

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Zhang Song
eb9e9b9d06 crimson/os/seastore/cache: add hit ratio metric
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Zhang Song
2a5257a04c crimson/os/seastore: make cold rbm cleaner use different metric prefix
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Zhang Song
a8b0baca91 crimson/os/seastore: allow RBM backend as hot tier
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Zhang Song
c969a8b9b5 crimson/os/seastore: update logical bucket cache when processing
read/write transactions

All client accessed extents are to be tracked by logical bucket cache
before eviction

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Zhang Song
afde4c6dcb crimson/os/seastore: add write through policy
Write extents larger than some threshold to the cold tier directly

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Zhang Song
43d0b3316a crimson/osd: create cold devices when mkfs
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-31 09:40:19 +08:00
Afreen Misbah
3e1c41c193
Merge pull request #70621 from rhcs-dashboard/fix-IBMCEPH-16202
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-31 03:07:40 +05:30
Afreen Misbah
386cd71173
Merge pull request #70558 from rhcs-dashboard/rgw-user-form-fix
Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2026-07-31 03:07:16 +05:30
Afreen Misbah
de09de6004
Merge pull request #70655 from rhcs-dashboard/fix-78823
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-31 03:06:35 +05:30
Afreen Misbah
1066090245
Merge pull request #70643 from rhcs-dashboard/fix-pool-deletion
mgr/dashboard: Pool deletion enhancements

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Naman Munet <nmunet@redhat.com>
2026-07-31 01:10:41 +05:30
Patrick Donnelly
413f3e7590
Merge PR #70721 into main
* refs/pull/70721/head:
	doc/dev: update release checklist for github automations

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2026-07-30 15:24:28 -04:00
John Mulligan
2569c8bd8b
Merge pull request #70684 from phlogistonjohn/jjm-grpc-sock-len
python-common/smb: work around socket path length limits for grpc lib

Reviewed-by: Anoop C S <anoopcs@cryptolab.net>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
2026-07-30 14:19:33 -04:00
Patrick Donnelly
0b3995d9cc
Merge PR #70717 into main
* refs/pull/70717/head:
	.github/workflows/pr-checklist: switch to pull_request_target and scope permissions

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2026-07-30 13:37:05 -04:00
Afreen Misbah
d0add82594 mgr/dashboard: Pool deletion enhancements
Admin role:
- show confirmation modal that pool deletion is disabled
- allow enabling pool deletion
- fix pool deletion text

Pool mgr role
- show a modal that pool deletion is disabled

Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-30 21:48:01 +05:30
Patrick Donnelly
35c5193cd3
doc/dev: update release checklist for github automations
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-30 11:09:46 -04:00
Patrick Donnelly
35060603bc
.github/workflows/pr-checklist: switch to pull_request_target and scope permissions
To make `main` the authoritative workflow source.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-30 10:55:51 -04:00
J. Eric Ivancich
92cef49cde
Merge pull request #70483 from gardran/wip-gardran-fix-78491
rgw/rados/rgw_sync: fix unintended uint64_t->int conversion

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
2026-07-30 10:40:34 -04:00
Naman Munet
b9fccdffd2 mgr/dashboard: Add required field validation for managed policies in account user creation
fixes: https://tracker-origin.ceph.com/issues/78684

Signed-off-by: Naman Munet <naman.munet@ibm.com>
2026-07-30 16:09:47 +05:30
Sagar Gopale
5bab45c606 mgr/dashboard: hide number spinners for fixed priority values
Fixes: https://tracker.ceph.com/issues/78823
Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-07-30 14:07:55 +05:30
Sagar Gopale
4adc02dffb mgr/dashboard: Fix password form validation mismatch errors
Fixes: https://tracker.ceph.com/issues/78781

Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-07-30 12:07:13 +05:30
Zhang Song
98f1892fb6 crimson/os/seastore: throttle bandwidth to secondary devices
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-30 10:12:27 +08:00
Zhang Song
223dde6bab crimson/os/seastore: introduce RotationalDevice
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-30 10:12:27 +08:00
Zhang Song
eca32e1fa9 crimson/os/seastore: set backend type explicitly
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-30 10:12:27 +08:00
Zhang Song
051db088b3 crimson/os/seastore/EPM/BackgroundProcess: add promote and demote process
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-30 10:12:27 +08:00
Zhang Song
91eb5d2a9c crimson/os/seastore/EPM: initialize logical bucket and pinboard
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-30 10:12:27 +08:00
Zhang Song
c7b3fd1350 crimson/os/seastore: don't add shadow extent to cache
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-30 10:12:26 +08:00
Zhang Song
3f2b958478 crimson/os/seastore: avoid evicting promoted extent to the cold tier
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-30 10:12:26 +08:00
Zhang Song
d088cfc46d crimson/os/seastore: consolidate parameters on extent allocation path
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-30 10:12:26 +08:00
Zhang Song
1af4a97792 crimson/os/seastore: implement demote region
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-30 10:12:26 +08:00
Zhang Song
c3e971a84c crimson/os/seastore: implement promote extent
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-30 10:12:26 +08:00
Zhang Song
ad7469ca59 crimson/os/seastore/lba: introduce shadow paddr
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-30 10:12:26 +08:00
Zhang Song
671b1b4ffc crimson/os/seastore: introduce logical bucket and demote process
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-30 10:12:26 +08:00
Zhang Song
2d629af107 crimson/os/seastore: support promote extents purged from pinboard
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-30 10:12:26 +08:00
Zhang Song
6c526c811b crimson/os/seastore: rename extent_2q_state_t to extent_pin_state_t
This is because, even for LRU caches, extents still have pin states,
although it's only Fresh. What's more, in the logical bucket cache
world, extents in both LRU and 2Q caches may be in the
Fresh/PendingPromote/Promoting state.

Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-30 10:12:26 +08:00
Zhang Song
ece1a68497 crimson/os/seastore: introduce PROMOTE and DEMOTE transaction type
Signed-off-by: Zhang Song <zhangsong02@qianxin.com>
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-30 10:12:26 +08:00
Xuehan Xu
f17f12b4a1 doc/dev/crimson: add design the document for LogicalBucketCache
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-30 10:12:26 +08:00
Patrick Donnelly
9336dcba48
.github/workflows: switch to pull_request_target trigger
Switch the trigger from `pull_request` to `pull_request_target` so that
the workflow definition on `main` is evaluated for PRs targeting active
release branches (main, umbrella, tentacle, squid). This allows the check
to run cleanly across release branches without requiring workflow file
backports to each branch.

Additionally:
- Update `src/script/verify-qa` to accept an optional target directory argument ($1).
- Run the trusted `verify-qa` script from `main` against the checked-out PR directory (`./pull_request`).

The concern of exfiltrating secrets is important but not relevant here:
we're not using the secrets in the definition and we explicitly
downgrade the github token to `read` permission.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-29 21:05:05 -04:00
Patrick Donnelly
71ddab8e32
.github/workflows: switch to pull_request_target trigger
Switch the trigger from `pull_request` to `pull_request_target` so that
the workflow definition on `main` is evaluated for PRs targeting active
release branches (main, umbrella, tentacle, squid). This allows the check
to run cleanly across release branches without requiring workflow file
backports to each branch.

While this workflow passes `github.token`, the token permissions are
strictly restricted to read-only (`contents: read` and `pull-requests: read`).
Because no PR code is checked out or executed, using `pull_request_target`
presents zero security risk here.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-29 20:43:02 -04:00
John Mulligan
144c6e8f76 python-common/smb: work around socket path length limits for grpc lib
The grpc library (at least the version on centos/rocky 10) has a limit
of 107 characters. Even a fairly short host name is going to be close
to this limit due to the UUID and service name in the path.

Because the tool doesn't do much else and it is safe, the workaround is
to chdir to into the parent dir of the socket and only pass the relative
path name of socket.

Fixes: https://tracker.ceph.com/issues/78857
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2026-07-29 17:53:50 -04:00
Yuri Weinstein
4705abd056 script/ptl-tool: add 'crimson', 'build/ops', 'common', 'tests' to SUPPORTED_QA_TAGS
PRs carrying these GitHub labels were silently dropped from both the
QA tracker's PR "Labels" column and the Redmine ticket's tag_list,
since SUPPORTED_QA_TAGS is a fixed whitelist and these were never
added to it despite being real, actively-used component/category
labels (e.g. ceph/ceph#70661, #70658 for crimson).

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2026-07-29 14:21:21 -07:00
John Mulligan
0a6d0cb043 cephadm: update how smb configwatch sidecar is started
In order to update users (and groups) live without AD support we need
the configwatch sidecar to be configured for split (altfiles) nsswitch.
The capability to configure configwatch for nsswitch was added to
sambacc under a special mode of the run subcommand.
Switch to starting the confgwatch sidecar using the run subcommand and
configuring the appropriate user and nsswitch setup options.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2026-07-29 16:00:01 -04:00
John Mulligan
8026e6941c cephadm: update smb env to specify writable shared passwd/group files
Update environment for smb containers to specify arguments that sambacc
will use to configure the containers for a split configuration for
users (passwd) and group files. The writable copy will be store in
the samba state dir (/var/lib/samba in the container) and be shared
across the primary and sidecar containers so that configwatch can
pull new users and groups lists from ceph and apply them live.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2026-07-29 16:00:01 -04:00
mheler
e82ccd0873
Merge pull request #70051 from mheler/wip-copyobject-replace 2026-07-29 13:30:53 -05:00
bluikko
2dcf7a1267
Merge pull request #70659 from bluikko/wip-doc-cephadm-install-substitutions
doc/cephadm: enable substitutions on CLI command example
2026-07-29 23:05:19 +07:00
Casey Bodley
bd3a488a1f
Merge pull request #69533 from qiuxinyidian/main
rgw:adding bucket index Indexless check when bi list

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2026-07-29 11:44:29 -04:00
VinayBhaskar-V
c4c937d81c librbd: propagate ENOENT for non-existent groups and images
Calling `list_images()` on a non-existent group falsely succeeded and
returned an empty iterator.This happened because `Group<I>::image_list`
ignored the negative error code returned by `group_image_list`, allowing
execution to proceed blindly. Furthermore, the C API `rbd_group_image_list`
intercepted `-ENOENT` and forced a success (0) return value.

Similarly other public C APIs `rbd_snap_list` and `rbd_group_snap_list`
also intercepts -ENOENT and forces a success (0) return value which
is incorrect.

Fix these behavior by:
- Propagating errors properly inside `Group<I>::image_list`.
- Removing the incorrect `-ENOENT` masking in `rbd_group_image_list`,
`rbd_snap_list`, rbd_group_snap_list`

Now `list_images()` consistently raise an `ObjectNotFound` error when
invoked on a non-existent group and public C APIs propagate ENOENT
without masking it into 0.
Also Extended the `TestGroups` test fixture with `self.dne_group` to
validate the expected error paths across all relevant APIs.

Fixes: https://tracker.ceph.com/issues/78108
Signed-off-by: VinayBhaskar-V <vvarada@redhat.com>
2026-07-29 19:11:14 +05:30
Afreen Misbah
25187a4389
Merge pull request #70097 from tchaikov/wip-mgr-testcase-single-bounce
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-29 18:26:36 +05:30
Afreen Misbah
51131eb3d9
Merge pull request #70515 from rhcs-dashboard/fix-17067
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-29 18:26:28 +05:30
Pedro Gonzalez Gomez
0bb0d59a85
Merge pull request #70604 from rhcs-dashboard/service-form-text-label-fix
mgr/dashboard: fix cd-text-label-list string writeValue

Reviewed-by: Nizamudeen A <nia@redhat.com>
2026-07-29 14:11:46 +02:00
Ville Ojamo
3fe15c1f71 doc/install: use ref for intra-docs links
Use ref instead of external link definitions for intra-docs links.
Add labels if necessary in files within the install/ directory. Use
existing labels for links outside the install/ directory but do not add
new labels outside of the install/ directory.

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>
2026-07-29 17:24:56 +07:00
Ville Ojamo
872d3bd14c doc/cephadm: enable substitutions on CLI command example
Substitutions were not enabled on a prompt directive that tried to use
the stable-release substitution so the substitution string was rendered
verbatim.

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>
2026-07-29 16:40:54 +07:00
Afreen Misbah
66a1de3eff
Merge pull request #70625 from rhcs-dashboard/fix/dashboard-gzip-mime-type
mgr/dashboard: add text/javascript to gzip mime types

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-29 15:07:01 +05:30
Afreen Misbah
a5803a5ed5
Merge pull request #70606 from rhcs-dashboard/add-doc-url
mgr/dashboard: Add alerts doc url to active alert list


Reviewed-by: Aashish Sharma <aasharma@redhat.com>
2026-07-29 13:32:30 +05:30
Venky Shankar
c748ba8b79
Merge pull request #68519 from edwinzrodriguez/cephfs-bench
cephfs-tool: Improve reporting and config options

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2026-07-29 13:09:51 +05:30
Guillaume Abrioux
4d26ec1395
Merge pull request #70486 from guits/fix-nvme-reformat
ceph-volume: fix nvme preformat wiping VG on osds-per-device > 1
2026-07-29 09:34:24 +02:00
Pedro Gonzalez Gomez
851ca4c75c
Merge pull request #70603 from rhcs-dashboard/snap-edit-fix
mgr/dashboard: fix snap schedule startDate

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2026-07-29 08:54:00 +02:00
root
58ef0cd984 mgr/dashboard: fix cd-text-label-list string writeValue
When editing services such as oauth2-proxy, backend fields like scope
arrive as a single string. Spreading that string into the list turned
each character into its own row. Normalize string values to one item.

Fixes: https://tracker.ceph.com/issues/78808
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@ibm.com>
2026-07-29 08:42:58 +02:00
Kefu Chai
070e0ca298
Merge pull request #69680 from sunyuechi/wip-bufferlist-omap
test/crimson/omap: enlarge values under ASan to shrink depth-driven tests

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-29 11:47:24 +08:00
SrinivasaBharathKanta
bfebe2ea25
Merge pull request #66757 from senolcolak/wip-v2-mgr-inventory-crush-host
mgr: override OSD hostname with CRUSH physical host in all metadata paths
2026-07-29 08:58:21 +05:30
SrinivasaBharathKanta
9da521bd82
Merge pull request #69885 from sunyuechi/wip-rados-fix-std-fd-close
tools/rados: fix stdio/stream cleanup
2026-07-29 08:56:20 +05:30
SrinivasaBharathKanta
97cc3c8d93
Merge pull request #67023 from trociny/wip-74490
mon: allow getpoolstats for mgr modules
2026-07-29 08:55:52 +05:30
SrinivasaBharathKanta
44075f25c8
Merge pull request #69973 from ljflores/wip-tracker-77382
qa/suites/upgrade: ignore expected telemetry warning
2026-07-29 08:55:12 +05:30
Kefu Chai
86c46693c2
Merge pull request #70206 from sunyuechi/wip-dockerfile-sccache-dnf
Dockerfile.build: install sccache from distro packages when available

Reviewed-by: Zack Cerza <zack@redhat.com>
2026-07-29 11:22:01 +08:00
SrinivasaBharathKanta
a51414287f
Merge pull request #69325 from NitzanMordhai/wip-nitzan-mgr-memory-grow-debug-high-level
mgr/ClusterState: remove debug 30 memory spike
2026-07-29 05:24:49 +05:30
SrinivasaBharathKanta
33afffca75
Merge pull request #69099 from MaxKellermann/osd_SnapMapper__fmt_no_sprintf
osd/SnapMapper: use fmt::format() instead of fmt::sprintf()
2026-07-29 05:24:11 +05:30
SrinivasaBharathKanta
0718de9184
Merge pull request #69098 from MaxKellermann/common_options__long_desc__c_string
common/options: convert `long_desc` to C string
2026-07-29 05:23:55 +05:30
SrinivasaBharathKanta
33a2cbcd2d
Merge pull request #70231 from bill-scales/issue58417
osd: Fix race hazard causing duplicated message after OSD marked down
2026-07-29 05:13:48 +05:30
SrinivasaBharathKanta
15825ec7bf
Merge pull request #70233 from bill-scales/issue54932
osd: Fix PeeringState::proc_lease crash caused by race hazard
2026-07-29 05:13:18 +05:30
Radoslaw Zarzynski
6e2ec3160c
Merge pull request #70333 from bill-scales/issue76953
osd: Fix valgrind uninitialized variable issues in fast EC

Reviewed-by: Matty Williams <Matty.Williams@ibm.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2026-07-28 22:22:56 +02:00
J. Eric Ivancich
c3c4a139ad
Merge pull request #70505 from ivancich/wip-fix-ceph-spec-in
rgw: update rpm and debian builds

Reviewed-by: Thomas Serlin <tserlin@redhat.com>
2026-07-28 16:16:58 -04:00
Afreen Misbah
3a081d5cd0 mgr/dashboard: add text/javascript to gzip mime types
CherryPy's serve_file() uses Python's mimetypes module which returns
text/javascript for .js files (RFC 9239), but the gzip config only
listed application/javascript. This meant the main Angular bundle
was being served uncompressed. With Cheroot write timeouts,
the uncompressed transfer times out over slower networks, causing
ERR_CONTENT_LENGTH_MISMATCH in the browser.

Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-28 19:13:55 +05:30
Afreen Misbah
ff681c5884
Merge pull request #70544 from rhcs-dashboard/switcher
mgr/dashboard: rgw gateway switcher read-only for resource page

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-28 19:03:46 +05:30
Xuehan Xu
c2a7345ca7
Merge pull request #70547 from xxhdx1985126/wip-crimson-recover-delete-fix
crimson/osd/pg_recovery: call on_global_recover on replica-recover-deleted objects only when it's not missing on the primary too

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-28 21:19:26 +08:00
Patrick Donnelly
e677c76f12
Merge PR #70581 into main
* refs/pull/70581/head:
	doc/governance: update CEC term length

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2026-07-28 08:43:05 -04:00
Patrick Donnelly
af0017d0d0
Merge PR #70574 into main
* refs/pull/70574/head:
	.github/workflows: bump checkout action version
	.github/workflows/qa-symlink: sparse checkout verify-qa
	.github/workflows/qa-symlink: run only for active branches

Reviewed-by: Joseph Mundackal <jmundackal@bloomberg.net>
2026-07-28 08:40:32 -04:00
Patrick Donnelly
b44f03b373
Merge PR #70582 into main
* refs/pull/70582/head:
	doc/governance: add Redouane to CSC

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2026-07-28 08:40:04 -04:00
Patrick Donnelly
aff8483b3e
Merge PR #70586 into main
* refs/pull/70586/head:
	.githubmap: add BBoozmen

Reviewed-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-07-28 08:39:18 -04:00
Patrick Donnelly
e344300980
Merge PR #70598 into main
* refs/pull/70598/head:
	.githubmap: add mheler

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-28 08:36:50 -04:00
Vallari Agrawal
3c18e56570
Merge pull request #70614 from VallariAg/wip-nvmeof-upgrade-refresh
qa/suites/nvmeof/upgrade: use refresh True for wait_for_service
2026-07-28 16:03:38 +05:30
Alex Ainscow
7eaf1fb72f
Merge pull request #69662 from aainscow/truncate_and_write_tests
test/osd: Add truncate-then-write tests 

Reviewed-by: Bill Scales <bill_scales@uk.ibm.com>
2026-07-28 11:27:43 +01:00
Kefu Chai
a09c01ffb4
Merge pull request #69758 from tchaikov/wip-bumpup-seastar
crimson: bump seastar submodule and adapt to its new future API 

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2026-07-28 18:24:32 +08:00
Aashish Sharma
03df721ac7 mgr/dashboard: Add alerts doc url to active alert list
Add documentation link in Learn more column for every alert in the
active alerts list table

Fixes: https://tracker-origin.ceph.com/issues/78747

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2026-07-28 15:16:37 +05:30
Vallari Agrawal
5027b84eb3
qa/suites/nvmeof/upgrade: use refresh True for wait_for_service
Add refresh: True to upgrade tests, same as we added for
https://tracker.ceph.com/issues/73538

Fixes: https://tracker.ceph.com/issues/78770

Co-authored-by: barakda <barak.davidov@gmail.com>
Signed-off-by: Vallari Agrawal <vallari.agrawal@ibm.com>
2026-07-28 14:05:23 +05:30
Kefu Chai
31ed48ac8f crimson: migrate off deprecated seastar::smp::count and all_cpus()
The updated seastar deprecates the global smp::count and smp::all_cpus() in
favour of the per-instance smp::shard_count() and smp::all_shards(). Under the
CI -Werror build these deprecation warnings are fatal.

Replace the uses with the reactor-local free functions
seastar::this_smp_shard_count() and seastar::this_smp_all_shards(), which read
the current smp instance's shard count and shard-id range. crimson runs a
single smp instance and every migrated site executes on a reactor thread (the
alienstore worker threads do not touch these), so the behaviour is unchanged.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-28 16:07:19 +08:00
Kefu Chai
dcbe48ff0b crimson: use the seastar span sink API and build at API level 10
At SEASTAR_API_LEVEL >= 9 seastar dropped output_stream::write(net::packet) in
favour of write(std::span<temporary_buffer<char>>). Convert Socket's writes to
the span form, writing the packet's released fragments and keeping them alive
across the write with do_with, then raise Seastar_API_LEVEL from 6 to 10 to
match the updated seastar. The span write overloads are unconditional, so the
Socket change is valid at every API level.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-28 16:05:39 +08:00
Kefu Chai
df181842f1 crimson: bump seastar submodule and adapt to its new future API
Update src/seastar to ceph-umbrella-25.05.0-1074-gcced0236, which is
scylladb/seastar master (fd71a5b4c7) with the ceph patches reapplied,
and adapt crimson to the future API changes the bump brings in:

  - interruptible_future.h: seastar split its single ready_future_marker into
    set_ready_future_marker and set_from_tuple_ready_future_marker; route
    futurize::from_tuple through the matching markers.
  - thread_pool.h: future_state::set() is now templated and get_value() is
    ref-qualified; pass seastar::internal::monostate{} for the void case and
    move the state before reading its value.

These crimson changes need the new seastar, so they share one commit with the
submodule bump. The build stays at Seastar_API_LEVEL 6 here; the level is
raised in the following commit.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-28 16:05:39 +08:00
Matan Breizman
5efd84b2ef
Merge pull request #70180 from Matan-B/wip-matanb-seastore-coll-batch
crimson/seastore: batch same-collection transactions

Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-28 10:13:32 +03:00
bluikko
6d7963b016
Merge pull request #70467 from bluikko/wip-doc-nginx-capitalization
doc: standardize capitalization of NGINX following the vendor
2026-07-28 14:04:30 +07:00
Hezko
aac1f3c478
Merge pull request #70209 from Hezko/warn-on-deprecated-param
mgr/dashboard: add deprecation warning on ns add cli command
2026-07-28 09:25:41 +03:00
Afreen Misbah
e4fec86961
Merge pull request #70560 from rhcs-dashboard/fix-gateway-readonly
Reviewed-by: Naman Munet <nmunet@redhat.com>
2026-07-28 11:48:23 +05:30
Afreen Misbah
60c64a4977
Merge pull request #70566 from rhcs-dashboard/service-form-helper-fix
Reviewed-by: Nizamudeen A <nia@redhat.com>
2026-07-28 11:48:18 +05:30
Pedro Gonzalez Gomez
a11f0e3d4e mgr/dashboard: fix snap schedule startDate
Fixes date time picker that was wrongly setting the form date

Fixes: https://tracker-origin.ceph.com/issues/78742
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@ibm.com>
2026-07-28 08:05:53 +02:00
Venky Shankar
ed8939d940 Merge PR #70375 into main
* refs/pull/70375/head:
	src/pybind: fixing the test_invalid_client_id failure

Reviewed-by: Jos Collin <jcollin@redhat.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2026-07-28 11:25:58 +05:30
Kefu Chai
757d192b43
Merge pull request #69726 from tchaikov/wip-drop-bluestore-pmem
blk,build: drop BlueStore PMEM support

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
2026-07-28 13:27:17 +08:00
Tomer Haskalovitch
c47b6f4e8d mgr/dashboard: add deprecation warning on ns add nvme cli command
Fixes: https://tracker.ceph.com/issues/78228
Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>
2026-07-28 07:30:40 +03:00
Kefu Chai
6f4345554d
Merge pull request #68209 from sunyuechi/riscv-bench-crc
test/common: add RISC-V CRC32C performance benchmark to unittest_crc32c

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-28 09:40:42 +08:00
Gil Bregman
0b5dfd5a80
Merge pull request #70578 from gbregman/main
mgr/cephadm: Add CNC fields to NVMEoF spec file
2026-07-28 02:04:26 +03:00
Matthew N. Heler
0ba95d1a87 .githubmap: add mheler
Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
2026-07-27 17:24:32 -05:00
Syed Ali Ul Hasan
b1be0f8d8f mgr/dashboard: rgw gateway switcher read-only for resource page
- Fixes: https://tracker.ceph.com/issues/78513

Signed-off-by: Syed Ali Ul Hasan <syedaliulhasan19@gmail.com>
2026-07-28 02:05:43 +05:30
Afreen Misbah
e781e5a1a5
Merge pull request #69790 from syedali237/dashboard/user-management
mgr/dashboard : migrated User Table tabs to resource pages

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Abhishek Desai <abhishek.desai1@ibm.com>
2026-07-28 01:49:50 +05:30
Shilpa Jagannath
405b99064f qa/multisite: fix flaky test_bucket_sync_disable
rely on 'bucket sync status' instead of looking up sync-status object.
it reports that there are no sync sources once the bucket's disabled
flag has propagated. also add a meta checkpoint after disabling.

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2026-07-27 19:40:48 +00:00
Alex Ainscow
c95853db84 test/osd: cover truncate+write EC transaction behaviour
Add unit-tests to recreate the problems found by the IO Sequencer in the previous commit.

Some tests here rely on frameworks added in Umbrella and as such we are keeping them
independent (in their own PRs).

Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>
Signed-off-by: Matty Williams <Matty.Williams@ibm.com>
Assisted-by: IBM Bob:Claude/GPT
2026-07-27 18:47:37 +01:00
Gil Bregman
622feb2a72 mgr/cephadm: Add CNC fields to NVMEoF spec file
Fixes: https://tracker-origin.ceph.com/issues/78675

Signed-off-by: Gil Bregman <gbregman@il.ibm.com>
2026-07-27 20:40:04 +03:00
Patrick Donnelly
a51222647d
.githubmap: add BBoozmen
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-27 13:23:31 -04:00
J. Eric Ivancich
855a5edf70 rgw: update build config to streamline builds
Some binaries and man pages were not included in rpm builds or debian
builds.  This adds those artifacts to ceph.spec.in
debian/ceph-common.install and debian/radosgw.install.

Signed-off-by: J. Eric Ivancich <ivancich@redhat.com>
2026-07-27 13:09:48 -04:00
Patrick Donnelly
66524970a1
Merge PR #70576 into main
* refs/pull/70576/head:
	script/ptl-tool: add more refs for main branch

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2026-07-27 13:06:57 -04:00
Syed Ali Ul Hasan
78a2d82cb1 mgr/dashboard: migrated user table tabs to resource pages
- Fixes: https://tracker.ceph.com/issues/77475

Signed-off-by: Syed Ali Ul Hasan <syedaliulhasan19@gmail.com>
2026-07-27 21:35:31 +05:30
Patrick Donnelly
0c139f2b83
doc/governance: update CEC term length
Fixes: https://tracker.ceph.com/issues/77966
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-27 11:43:12 -04:00
Patrick Donnelly
1d3b2372e3
doc/governance: add Redouane to CSC
Fixes: https://tracker.ceph.com/issues/77965
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-27 11:13:56 -04:00
Patrick Donnelly
ebc60e401f
script/ptl-tool: add more refs for main branch
Start with "upstream", then "origin", then finally the local main branch.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-27 09:45:28 -04:00
Redouane Kachach
0a6e4207e6
Merge pull request #69844 from kginonredhat/issue-77835-fail-fs-mds-upgrade-wait
mgr/cephadm: wait for active MDS during fail_fs upgrade

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2026-07-27 15:39:40 +02:00
Patrick Donnelly
841baf903e
.github/workflows: bump checkout action version
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-27 09:36:03 -04:00
Patrick Donnelly
901aad681f
.github/workflows/qa-symlink: sparse checkout verify-qa
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-27 09:36:03 -04:00
Patrick Donnelly
59d8a46d4b
.github/workflows/qa-symlink: run only for active branches
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-27 09:35:56 -04:00
Patrick Donnelly
a8817bb61f
Merge PR #70259 into main
* refs/pull/70259/head:
	ptl-tool: keep .githubmap sorted when adding new contributors

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2026-07-27 09:34:04 -04:00
Oguzhan Ozmen
993f3f7c50
Merge pull request #70449 from BBoozmen/wip-oozmen-78572
qa/tasks/dnsmasq: set no-resolv on the systemd-resolved path
2026-07-27 09:22:26 -04:00
Afreen Misbah
c4a890a226
Merge pull request #70202 from rhcs-dashboard/fix-toast-length
Fix long toast notifications

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-27 18:36:50 +05:30
Afreen Misbah
2e51b71146
Merge pull request #69189 from syedali237/pools-table
mgr/dashboard: converted pools list table tab to resource pages

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-27 18:35:32 +05:30
Kefu Chai
a1582bdf64
Merge pull request #69862 from sunyuechi/wip-crimson-mon-fix-uaf-run-command
crimson/mon/MonClient: fix use-after-free in run_command

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-27 20:45:23 +08:00
Kefu Chai
cf2d42d48a
Merge pull request #70189 from sunyuechi/fix-sccache-dist-status-check
cmake: fix sccache job limits when dist is unavailable

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-27 20:37:59 +08:00
Kefu Chai
8131835af8
Merge pull request #69821 from sunyuechi/wip-rgw-inverted-checks
rgw: fix three inverted condition checks

Reviewed-by: Daniel Gryniewicz <dang1@ibm.com>
2026-07-27 20:27:46 +08:00
bluikko
6a2df53773
Merge pull request #70324 from bluikko/wip-doc-man-ceph-osd-pool-get
doc/man: sync osd pool get/set flags in 8/ceph.rst
2026-07-27 17:29:13 +07:00
Naman Munet
b4d5e0a7bb mgr/dashboard: Update certificate description field with proper help text
fixes: https://tracker-origin.ceph.com/issues/78698

Signed-off-by: Naman Munet <naman.munet@ibm.com>
2026-07-27 15:37:33 +05:30
pujashahu
0d29372b6b mgr/dashboard : Group name should be read only when user tries to edit existing nvmeof gateway
Fixes: https://tracker-origin.ceph.com/issues/78685

Signed-off-by: pujaoshahu <pshahu@redhat.com>
2026-07-27 14:44:48 +05:30
Avan
57074b5085
Merge pull request #70224 from avanthakkar/fix-smb-cluster-resource-resync
mgr/smb: only resync clusters that reference a changed resource


Reviewed-by: Anoop C S <anoopcs@cryptolab.net>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2026-07-27 14:35:46 +05:30
pujashahu
cbc41b07d8 mgr/dashboard : If user is selected auto-fetch option while creating subsystems then subnet-mask is mandatory param
Fixes: https://tracker.ceph.com/issues/78641

Signed-off-by: pujashahu <pshahu@redhat.com>
2026-07-27 14:01:06 +05:30
Radoslaw Zarzynski
c2709f56c0
Merge pull request #70151 from sunyuechi/fix-ec-slice-map-offset
osd/ECUtil: fix offset accumulation in slice_map

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
Reviewed-by: Alex Ainscow <aainscow@uk.ibm.com>
2026-07-27 10:19:58 +02:00
Afreen Misbah
24f44a95a0
Merge pull request #70338 from rhcs-dashboard/cephfs-access-denied-issue
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-27 12:35:11 +05:30
Afreen Misbah
90e7f65f41
Merge pull request #70341 from rhcs-dashboard/access-denied-issue-read-only-user
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-27 12:35:07 +05:30
Afreen Misbah
44b87d90d5
Merge pull request #70420 from rhcs-dashboard/fix-favicon-issue-upstream
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-27 12:33:10 +05:30
Afreen Misbah
1ee4fb8384
Merge pull request #69981 from rhcs-dashboard/fix-77976-main
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-27 12:31:02 +05:30
Afreen Misbah
b653950fab
Merge pull request #70369 from rhcs-dashboard/logs-component-filter-shifting
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-27 12:30:46 +05:30
Vallari Agrawal
a25d535bd1
Merge pull request #70399 from VallariAg/wip-submodule-192
mgr/dashboard: bump nvmeof submodule to 1.9.2
2026-07-27 12:23:12 +05:30
Afreen Misbah
8a1ded4890
Merge pull request #70251 from rhcs-dashboard/nfs-share-edit-issue
mgr/dashboard: Subvolume group and subvolume not listed in NFS Share form on edit.

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-27 12:05:44 +05:30
Aviv Caro
3a8215d69c
Merge pull request #70461 from leonidc/fix-nvme-gw-show-all
nvmeofgw:  fix command nvme-gw show/show-all
2026-07-27 09:24:46 +03:00
Afreen Misbah
55f797ccd3
Merge pull request #70409 from rhcs-dashboard/add-alert-button
mgr/dashboard: Add alert button in data resiliency tab

Reviewed-by: Nizamudeen A <nia@redhat.com>
2026-07-27 11:54:44 +05:30
Ville Ojamo
aee2067e14 doc: standardize capitalization of NGINX following the vendor
The vendor stylizes it as NGINX.
Previously docs capitalized it as either NGINX, Nginx or nginx. nginx
was the preferred capitalization before commercialization of the
software.

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>
2026-07-27 12:17:08 +07:00
Syed Ali Ul Hasan
1b4f9f10ca mgr/dashboard: migrated pool table tabs into resource pages
- Fixes:  https://tracker.ceph.com/issues/76713

Signed-off-by: Syed Ali Ul Hasan <syedaliulhasan19@gmail.com>
2026-07-26 18:37:32 +05:30
Xuehan Xu
6dab4b5034 crimson/osd/pg_recovery: call on_global_recover on replica-recover-deleted
objects only when it's not missing on the primary too

The issue was introduced by 60537bdd2f
which didn't fix the issue it was supposed to because of the same error
this commit tries to fix.

Fixes: https://tracker-origin.ceph.com/issues/78676
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-26 20:55:32 +08:00
Matan Breizman
cedad57aee crimson/os/seastore/omap_manager: count key length in the overwrite gap
ow_gap_from_last_entry() returns how much the node grows when the tail entry
is overwritten. However, it compared *value* lengths only.

overwriting pg-log key (31B) is longer than a _fastinfo key (9B) -
So the growth was under-counted by 22 bytes.

Also assert in _append()/_overwrite() that the write stays within capacity() to
avoid similar issues.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-07-26 07:36:42 +00:00
Anthony D'Atri
72b684aaea
Merge pull request #69038 from zdover23/wip-doc-2026-05-21-mgr-prometheus-ceph-exporter
doc/mgr: document ceph-exporter
2026-07-25 11:55:06 -04:00
Laura Flores
165d47959c
Merge pull request #69684 from ljflores/wip-tracker-76927
qa/tasks: change collectl source to download.ceph.com

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
Reviewed-by: Radosław Zarzyński <Radoslaw.Adam.Zarzynski@ibm.com>
2026-07-24 16:59:29 -05:00
Shilpa Jagannath
90098a7027 qa/multisite: fixes multisite test: test_zonegroup_rename
period commit only triggers async push (RGWPeriodPusher)
from the a zone's gateway to the other zones. it doesn't block
until they've applied it. a reconfigure delay alone might not
be sufficient. add checkpoint retries to allow for lag.

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2026-07-24 20:39:30 +00:00
Shraddha Agrawal
72f3242b18
Merge pull request #70481 from shraddhaag/wip-shraddhaag-crimson-isa
src/crimson: statically link EC plugin, ISA, in crimson-osd
2026-07-24 23:27:11 +05:30
Adam Emerson
f4a2c0e7f3
Merge pull request #67732 from adamemerson/wip-policy-ux
rgw: Policy evalaution logging

Reviewed-by: Matthew N. Heler <matthew.heler@hotmail.com>
Reviewed-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-07-24 12:42:31 -04:00
Afreen Misbah
9b679b2236
Merge pull request #70410 from syedali237/dashboard/images
mgr/dashboard: migrated images table tabs to resource pages

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Abhishek Desai <abhishek.desai1@ibm.com>
2026-07-24 21:29:14 +05:30
Patrick Donnelly
1085687ac0
ptl-tool: keep .githubmap sorted when adding new contributors
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-24 10:22:17 -04:00
Christopher Hoffman
0efbe10e14
Merge pull request #62342 from dparmar18/i62673
mgr/volumes: accept unit with subvolume resize

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Neeraj Pratap Singh <neesingh@redhat.com>
Reviewed-by: Christopher Hoffman <choffman@redhat.com>
2026-07-24 09:28:15 -04:00
Kefu Chai
032c1a1c37
Merge pull request #69939 from tchaikov/wip-mgr-heap-asok
mgr: add heap admin socket command

Reviewed-by: Matthew N. Heler <matthew.heler@hotmail.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2026-07-24 21:16:05 +08:00
Christopher Hoffman
f0633dc430
Merge pull request #69880 from sunyuechi/client-fscrypt-fixes
client: fix fscrypt error-path bugs

Reviewed-by: Christopher Hoffman <choffman@redhat.com>
2026-07-24 09:15:20 -04:00
Vallari Agrawal
16f59e2b6f
Merge pull request #70281 from VallariAg/wip-nvmeof-allow-service-dump
mgr/cephadm: set mgr cap for NVMeoF to allow cmd "service dump"
2026-07-24 17:42:54 +05:30
Syed Ali Ul Hasan
d2e64ba6b4 mgr/dashboard: migrated images table tabs to resource pages
- Fixes: https://tracker.ceph.com/issues/78479

Signed-off-by: Syed Ali Ul Hasan <syedaliulhasan19@gmail.com>
2026-07-24 15:35:41 +05:30
Matty Williams
26f32cbab3 test: Add TruncateWrite ops to IO Sequencer and Sequences using them
current io exerciser sequences can model standalone truncates and writes,
but cannot generate a single transaction that truncates an object and
then writes sparse regions. Add SingleTruncateWriteOp, DoubleTruncateWriteOp,
and TripleTruncateWriteOp, wire them through IoSequence, ObjectModel, and
RadosIo, and extend EC sequences/tests so truncate+write transactions can
be reproduced and validated.

Signed-off-by: Matty Williams <Matty.Williams@ibm.com>
Assisted-by: IBM Bob:Claude/GPT
2026-07-24 10:08:18 +01:00
Shraddha Agrawal
93300b45c0 src/crimson/osd: statically link the ISA plugin in crimson-osd
Problem:
Before this change, when the ISA plugin (classic build, dlopen'd),
would try to log, it'd segfault due to null g_ceph_context.

We need a way to make logging work for EC plugins in crimson.

Solution:

By building the ISA plugin WITH_CRIMSON, its logging is
routed to seastar's logger, solving the issue.

It is not build separately and dlopen'd like in classic because
doing so would require the plugin to have its own seastar logger
copy, an undesirable outcome.

It is a statically built library in the crimson-osd binary, and thus
it shares the single seastar logger. The library is linked with
--whole-archive so the self-registration constructor is not dropped.

Signed-off-by: Shraddha Agrawal <shraddha.agrawal000@gmail.com>
2026-07-24 13:51:36 +05:30
Shraddha Agrawal
1462041bd9 src/erasure-code: support statically-linked (built-in) plugins
This commit adds the functionality in EC plugin registry to support
built in (statically linked) plugins. This is done by 2 main changes:

1. introducing a new map, builtin_plugins, where plugins can register
a factory for itself at startup.
2. load() now first checks if the plugin is builtin and instantiates it,
otherwise it falls back to dynamically linking the plugin.

Signed-off-by: Shraddha Agrawal <shraddha.agrawal000@gmail.com>
2026-07-24 13:51:36 +05:30
Vallari Agrawal
e12106a501
Merge pull request #70248 from VallariAg/wip-nvmeof-teuthology-mtls-improve
qa: Add cephadm-signed mtls test in nvmeof/mtls_test.sh
2026-07-24 12:33:48 +05:30
Guillaume Abrioux
9020d1dccd
ceph-volume: fix nvme preformat wiping VG on osds-per-device > 1
prepare_data_device() was calling nvme_utils.preformat() unconditionally
for every OSD slot. With --osds-per-device N, this means the second osd
reformats the device after the first OSD had already created its vg
on it which makes vgcreate fail with "device has a signature".

this commit fixes this issue by checking api.get_device_vgs() before
preformatting: if a ceph VG already exists on the device, skip the format.
This mirrors the logic create_lv() already uses to reuse an existing VG.

Fixes: https://tracker.ceph.com/issues/78615

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2026-07-24 09:02:09 +02:00
Neeraj Pratap Singh
af19eff6ed src/pybind: fixing the test_invalid_client_id failure
The test_invalid_client_id was failing due to the
incorrect returns by handle_command and in get_perf_data()
in mgr/ststs/fs/perf_stats.py
Introduced-by: f8f972182c
Fixes:https://tracker.ceph.com/issues/76162
Signed-off-by: Neeraj Pratap Singh <Neeraj.Pratap.Singh1@ibm.com>
2026-07-24 09:20:06 +05:30
Yuri Weinstein
5f026f27c7
Merge pull request #70120 from ceph/wip-yuriw-78149-fix-main
qa/tests: fix POOL_FULL ignorelist pattern in upgrade tests

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
2026-07-23 15:31:27 -07:00
Yuri Weinstein
8b261cd505 qa/tests: drop parens from all health-code ignorelist entries
Monitor.cc logs a "Health check cleared: <CODE> (was: ...)" line
for any health check clearing mid-test, with the code bare (no
parens) -- while the "Health check failed"/"Health check failed
(unmute)" lines wrap the code in parens. Every \(CODE\) entry in
this file therefore only matches the raise/failed form and misses
the corresponding cleared form, the same gap just fixed for
POOL_FULL.

Drop the escaped parens from the remaining entries so each matches
both log forms, consistent with the bare entries already present
(OSD_ROOT_DOWN, MDS_INSUFFICIENT_STANDBY, POOL_FULL).

Non-code entries (reached quota, overall HEALTH_, slow request,
noscrub, nodeep-scrub, osds down, insufficient standby, etc.) are
unchanged.

Fixes: https://tracker.ceph.com/issues/78149
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2026-07-23 12:32:51 -07:00
Daniel Gryniewicz
77f6a3b47e
Merge pull request #68291 from dang/wip-dang-standalone
RGW Standalone build
2026-07-23 13:07:04 -04:00
SrinivasaBharathKanta
a3a7111966
Merge pull request #68753 from pablodelara/isal_crypto_update
crypto/isa-l: update isa-l_crypto submodule to v2.26.1
2026-07-23 21:38:27 +05:30
SrinivasaBharathKanta
857f36715e
Merge pull request #69429 from tchaikov/wip-cmddesc
mon,common: correct typo in command desc and tighten the command desc formatter
2026-07-23 21:37:50 +05:30
SrinivasaBharathKanta
635a92806d
Merge pull request #69181 from stzuraski898/wip-sz-74693-perf_schema_no_counters
mgr/ActivePyModules: use RAII for perf schema section management
2026-07-23 21:37:23 +05:30
Vallari Agrawal
ab00786785
mgr/dashboard: bump nvmeof submodule to 1.9.2
Update gateway submodule and protobuf files.

Fixes: https://tracker.ceph.com/issues/78501

Signed-off-by: Vallari Agrawal <vallari.agrawal@ibm.com>
2026-07-23 20:44:45 +05:30
Kautilya Tripathi
b385483049
Merge pull request #70418 from knrt10/fix-crimson-osd-abort-lba-tree
crimson/seastore: keep on-page checksum in sync after merge_content_to
2026-07-23 19:46:12 +05:30
Garry Drankovich
ad980c6b65 rgw/rados/rgw_sync: fix unintended uint64_t->int conversion
Fixes: https://tracker.ceph.com/issues/78491

Signed-off-by: Garry Drankovich <garry.drankovich@clyso.com>
2026-07-23 16:53:59 +03:00
Aashish Sharma
2f4a86c0a3 mgr/dashboard: carbonize Setup multi-site replication wizard
Fixes: https://tracker.ceph.com/issues/77976

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2026-07-23 18:30:32 +05:30
Matan Breizman
0836e417ba crimson/os/seastore/omap_manager: predict append (not overwrite) on initial_pending
An initial_pending LogNode always append_kv()s and never overwrites.

expect_overflow() was still called with can_ow=true whenever the tail entry is
_fastinfo. It then used only the overwrite gap while the append
itself consumed a full entryi - so the node rolled over far too late and
used_space() overran capacity().

This would result in `assert(capacity() >= used_space()`.

Gate can_ow on !is_initial_pending() so the append is predicted with the full
entry size and rolls over in time.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-07-23 12:18:32 +00:00
Matan Breizman
9e87510be9 crimson/os/seastore/omap_manager: calculate the overwrite growth (gap)
The previous implementation computed ow_gap_from_last_entry() but never used
the result.
Use it to check only the additional space required when the tail
entry grows, which matches overwrite logic.

Note, this is a cleanup - behavior remains similar.

See 825c8f149c

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-07-23 12:12:53 +00:00
Kotresh HR
78b7fa265b qa/cephfs-mirror: Fix tests for ISO sync_time_stamp
Fixes: https://tracker.ceph.com/issues/76506
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-07-23 17:16:42 +05:30
Kotresh HR
10e008aaca doc/cephfs-mirroring: document ISO-8601 sync timestamps
Fixes: https://tracker.ceph.com/issues/76506
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-07-23 17:14:52 +05:30
Kotresh HR
f6d85aad99 cephfs_mirror: format sync timestamps as ISO-8601 local time
Show sync_time_stamp and metrics_updated_at as human-readable
local timestamps in peer_status and mgr status. Keep epoch in
omap and counter dump.

Fixes: https://tracker.ceph.com/issues/76506
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-07-23 17:06:46 +05:30
Afreen Misbah
b891817d66
Merge pull request #70362 from rhcs-dashboard/Irrelevant-realmdescription
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-23 15:54:26 +05:30
Vallari Agrawal
672b422cc9
qa: Add cephadm-signed mtls test in nvmeof/mtls_test.sh
Expand nvmeof mtls test to include cephadm-signed cert
(ssl=true + enable_auth=true, no certs)

Also improve wait_for_service() logic to assert all
gateways are running.

Fixes: https://tracker.ceph.com/issues/78295

Signed-off-by: Vallari Agrawal <vallari.agrawal@ibm.com>
2026-07-23 15:44:21 +05:30
Hezko
267f0eb9d6
Merge pull request #70223 from Hezko/fix-list-hosts-empty
mgr/dashboard: fix list_hosts nvmeof cli commanad empty table bug
2026-07-23 12:55:14 +03:00
Afreen Misbah
4ecb4c8e70
Merge pull request #68845 from rhcs-dashboard/fix-14295
mgr/dashboard: [Dashboard] Incorrect action button label "Edit User" instead of "Save" in Edit User page

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-23 15:18:56 +05:30
Afreen Misbah
805bd0d741
Merge pull request #70274 from rhcs-dashboard/fix-namespace
mgr/dashboard: Add RADOS Namespace Support in NVMe-oF UI

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-07-23 15:18:16 +05:30
Afreen Misbah
528e0fde60
Merge pull request #70278 from rhcs-dashboard/configuration-form-fix
mgr/dashboard: Edit should support setting one or more rgw service co…

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-23 15:06:21 +05:30
Afreen Misbah
4083c6e78a
Merge pull request #70292 from rhcs-dashboard/fix-hw-dashboard-stale-metrics
mgr/dashboard: fix stale metrics in hardware dashboards

Reviewed-by: Aashish Sharma <aasharma@redhat.com>
2026-07-23 12:47:42 +05:30
Abhishek Desai
a3f425e4c0 mgr/dashboard : Favicon issue
fixes : https://tracker.ceph.com/issues/78530
Signed-off-by: Abhishek Desai <abhishek.desai1@ibm.com>
2026-07-23 12:20:23 +05:30
Leonid Chernin
b820cf5743 nvmeofgw: fix command nvme-gw show-all
add missing fields after merge show-all pr

fixes: https://tracker.ceph.com/issues/78593

Signed-off-by: Leonid Chernin <leonidc@il.ibm.com>
2026-07-23 07:53:23 +03:00
Aishwarya Mathuria
2b72c5bd0e
Merge pull request #70250 from amathuria/wip-amat-seastore-fix-pass-admin
crimson/os/seastore/rbm: fix pass_admin to pass admin command to ioctl
2026-07-23 10:19:50 +05:30
bluikko
d0f26bed0f
Merge pull request #69867 from bluikko/wip-fix-sts-rgw-yaml-in
common/options: fix invalid command for STS key generation
2026-07-23 11:21:55 +07:00
Kefu Chai
a4c43f6e50
Merge pull request #70146 from tchaikov/wip-mgr-logging-with-level
mgr: forward the root-logger fallback at the record's own level 

Reviewed-by: Matthew N. Heler <matthew.heler@hotmail.com>
2026-07-23 11:03:03 +08:00
Zac Dover
0295ffaed7
Merge pull request #70450 from zdover23/wip-doc-2026-07-23-releases-update
doc/releases: move reef from supported releases

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2026-07-23 08:46:53 +10:00
Afreen Misbah
8321ca25d7 mgr/dashboard: fix stale metrics in hardware dashboards
Switch all stat panels in the hardware and hardware compression
dashboards from lastNotNull to last reduceFunction. When a host is
unreachable or node-proxy stops, metric silence is ambiguous — it
could mean host down, agent crash, or scrape failure. lastNotNull
freezes the last known value (e.g. 4 NVMe drives), causing
inconsistency with the global Overview which reflects live count
reductions.

Use last so panels show N/A when no current data arrives.

Fixes: https://tracker.ceph.com/issues/78360

Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-23 01:43:38 +05:30
Afreen Misbah
59b6c5b829 mgr/dashboard: fix notification kebab menu e2e selector
Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-23 00:42:59 +05:30
Afreen Misbah
59cb51d1b8 mgr/dashboard: use descriptive name and type for alert filter param
Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-23 00:36:09 +05:30
Gabriel Benhanokh
3bfe0dfc62
Merge pull request #70210 from benhanokh/rgw-dedup-enable-threads-config
rgw: add rgw_enable_dedup_threads config to control per-RGW dedup par…
2026-07-22 19:51:01 +03:00
Oguzhan Ozmen
014487f584
Merge pull request #70299 from BBoozmen/wip-oozmen-78364
rgw/admin: guard bilog autotrim on non-exporting (archive) zones
2026-07-22 11:48:03 -04:00
Oguzhan Ozmen
9e160aa8f1
Merge pull request #70003 from BBoozmen/wip-oozmen-70858
rgw: fix bilog autotrim of stale bucket.instance metadata after peer deletion
2026-07-22 11:47:39 -04:00
Abhishek Desai
481d6e4699 mgr/dashboard : Access denied issue for settings icon
fixes : https://tracker.ceph.com/issues/78407

Signed-off-by: Abhishek Desai <abhishek.desai1@ibm.com>
2026-07-22 21:02:11 +05:30
Zac Dover
a3eea0c946 doc/releases: move reef from supported releases
Move the Reef release from the list of supported releases.

Signed-off-by: Zac Dover <zac.dover@clyso.com>
2026-07-23 01:26:22 +10:00
Kobi Ginon
474d8a70b2 mgr/cephadm: wait for active MDS during fail_fs upgrade
When fail_fs is enabled with max_mds > 1, _prepare_for_mds_upgrade()
issued fs fail and then continued unconditionally, skipping the
up:active wait. _complete_mds_upgrade() also set joinable=true without
waiting for in-rank MDS to recover.

Wait for all in-rank MDS to reach up:active before proceeding with the
upgrade and after re-joining the filesystem. Add fs.ready to the
mds_upgrade_sequence verify task so ceph-fuse teardown does not race
with MDS still in up:rejoin.

Fixes: https://tracker.ceph.com/issues/77835

Signed-off-by: Kobi Ginon <kginon@redhat.com>
2026-07-22 18:20:24 +03:00
Oguzhan Ozmen
2a48100588 qa/tasks/dnsmasq: set no-resolv on the systemd-resolved path
On the systemd-resolved (debian) path, dnsmasq has only address= records
and no upstream, so queries it can't answer locally are forwarded via
/etc/resolv.conf which points back at systemd-resolved, which routes
the cname domains back to dnsmasq. The resulting loop burns a 5s
resolver retry per round trip which manifests itself in rgw s3tests
on Ubuntu running too slow hitting the job timeout.

Set no_resolv on that path so dnsmasq answers authoritatively and never
forwards. The rpm/NetworkManager path is unchanged, as dnsmasq there is
the system resolver and must forward upstream.

Fixes: https://tracker.ceph.com/issues/78572
Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-07-22 15:03:47 +00:00
Nathan Hoad
0f3412d97e
Merge pull request #68574 from nhoad/gc-shard-id
rgw: add support for --shard-id in gc process and gc list commands
2026-07-22 10:39:47 -04:00
Yuval Lifshitz
4b5506816b
Merge pull request #69666 from ShreeJejurikar/wip-rgw-kafka-4x-support
rgw: support kafka 4.x brokers
2026-07-22 17:26:53 +03:00
Abhishek Desai
281e220dbb mgr/dashboard : Access denied issue for cephfs-mgr role
fixes : https://tracker.ceph.com/issues/78414
Signed-off-by: Abhishek Desai <abhishek.desai1@ibm.com>

 Conflicts:
	src/pybind/mgr/dashboard/tests/test_access_control.py
        - conflict due to other access denied PRs
2026-07-22 19:53:04 +05:30
Jesse Williamson
4fab556e12
Merge pull request #70266 from chardan/jfw-rgw-libfdb-fixup
fixups for (EXPERIMENTAL) libfdb FoundationDB support
2026-07-22 06:31:25 -07:00
Afreen Misbah
0dd6849ee3 mgr/dashboard: fix notification back button stacking history
Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-22 18:31:33 +05:30
Casey Bodley
5dac2a7669
Merge pull request #69642 from lumir-sliva/rgw-lc-expiration-header-current-version
rgw/lc: report x-amz-expiration only for the current version

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2026-07-22 08:53:36 -04:00
mohit84
95a007b9a0
Merge pull request #69824 from mohit84/backfill_stuck
crimson/osd: fix backfill deadlock in BackfillState::Waiting

BackfillState::Enqueuing unconditionally posts RequestWaiting{}
when budget available return false. Waiting can exit only via
ObjectPushed which requires an in-flight push. If the budget
is exhausted with no pushes in flight no objectPush will ever
arrive and the PG stalls permanently.

Solution: Introduce a dedicated BudgetBlocked state. When
budget_available() is false and tracked_objects_completed()==true
post RequestBudgetBlocked{} instead of RequestWaiting{}.
BudgetBlocked calls get_backfill_throttle() to suspend until a slots 
free up, then fires BudgetAvailable{} which transitions directly back
to Enqueuing to dispatch the next batch of pushes.

Fixes: https://tracker.ceph.com/issues/77804
2026-07-22 17:56:12 +05:30
Afreen Misbah
a1a70d06f4 mgr/dashboard: fix notification re-navigation when already on page
When a notification is clicked in the panel while the user is already
on the notifications page, the selected notification now updates
correctly by bypassing the preselect guard for query param changes.

Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-22 17:34:39 +05:30
Pedro Gonzalez Gomez
93703c3971
Merge pull request #69218 from rhcs-dashboard/77030-control-flow-fix-pool-creation-page
mgr/dashboard: Fixing control flow on submit button for pool creation page

Reviewed-by: Abhishek Desai <abhishek.desai1@ibm.com>
Reviewed-by: Puja Shahu <pshahu@redhat.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-22 12:57:22 +02:00
Afreen Misbah
8e12637e36
Merge pull request #70419 from rhcs-dashboard/replace-gherkin-lint
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-22 16:01:28 +05:30
Afreen Misbah
191bc2ac95
Merge pull request #69958 from rhcs-dashboard/fix-access-denied-issue
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-22 15:59:17 +05:30
ShreeJejurikar
efe54523a4 rgw: support kafka 4.x brokers
librdkafka submodule

Vendors librdkafka v2.12.1 as a submodule (default) and teaches the
notification teuthology task to set up Kafka 4.x KRaft brokers. The
distro-shipped librdkafka 1.6.1 cannot SCRAM-auth against Kafka 4.x
brokers (#75900); the bundled version fixes that. Build with
WITH_SYSTEM_RDKAFKA=ON to opt back into the distro library (floor 2.6.1).
follwinf changes needed with teh submodule:
* build kafka_stub only in case of system librdkafka
* use the same header location in submodule and system
* supress compilation warning to avoid -WError issues

2.12.1 matches Fedora(44)'s current librdkafka-devel, so the vendored copy
tracks a version that distro packagers have already vetted rather than
whichever release happens to be latest upstream.

The teuthology version matrix picks one of {3.9.2, 4.1, 4.2, 4.3} per run.
3.9.2 covers the last Zookeeper-based release for the migration cohort.
4.1/4.2/4.3 track Apache's current three supported minor lines and are
resolved to the latest patch at test time via dlcdn.apache.org/kafka/, so
new patches within a supported minor need no PR update. A new minor line
(4.4) requires renaming one YAML.
* new java version (17) is needed for kafka 4.2 and up (we wil use it on
  all version)
* kerberos task need to run before the kafka install task

After pulling this branch, fetch the new submodule:

    git submodule update --init src/librdkafka

Fixes: https://tracker.ceph.com/issues/77336

Signed-off-by: ShreeJejurikar <shreemj8@gmail.com>
Co-authored-by: Yuval Lifshitz <ylifshit@ibm.com>
2026-07-22 10:29:04 +00:00
Matan Breizman
6cae08dcc8 crimson/seastore: batch only txns with matching data_features
Transaction::append() asserts both transactions share data_features.
The per-collection batcher merged queued client txns unconditionally,
so a data_features mismatch would be asserted in build_next_batch().

Use diffrent data_features as a grouping boundary.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-07-22 09:15:57 +00:00
Matan Breizman
b9e06d8e49 crimson/seastore: run pg-log trim txns solo, off the batched path
Per-collection batching merges many client txns into one seastore txn.
Batched OP_OMAP_RMKEYS is not stable yet, possibly due to an exposed
internal bug that is seen with the new behavior.
Avoid batching OMAP_RMKEYS for now.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-07-22 09:15:57 +00:00
pujaoshahu
81feb0d1c9 mgr/dashboard: [Dashboard] Incorrect action button label "Edit User" instead of "Save" in Edit User page
Fixes: https://tracker.ceph.com/issues/76507

Signed-off-by: pujaoshahu <pshahu@redhat.com>

Summary: Updated submit button labels in edit forms to display "Save" instead of "Edit User", "Edit Pool", or "Edit Host" for better UX consistency, while keeping form titles unchanged.

Modified Files
role-form.component.ts: Added submitAction property to control submit button label by mode; edit mode uses Save changes, create mode uses Create Role.

role-form.component.html: Submit button now binds to submitAction instead of action/resource text.

role-form.component.spec.ts: Added tests for submitAction behavior in create and edit modes.

user-form.component.ts: Uses submitAction property; edit mode label updated to Save changes.

user-form.component.html: Submit button uses submitAction binding.

user-form.component.spec.ts: Added/updated tests covering submitAction behavior.

pool-form.component.ts: Uses submitAction property; edit mode label updated to Save changes.

pool-form.component.html: Submit button uses submitAction binding.

pool-form.component.spec.ts: Updated submit label test expectation for edit mode.

hosts.component.ts: Edit host modal submit button text changed to Save changes.

hosts.component.spec.ts: Updated test to match Save changes button text.

app.constants.ts: In the current workspace snapshot, there is no SAVE constant added under ActionLabelsI18n.

Signed-off-by: pujaoshahu <pshahu@redhat.com>
Signed-off-by: pujashahu <pshahu@redhat.com>
2026-07-22 14:34:06 +05:30
Jaya Prakash
091c0d01da
Merge pull request #70391 from Jayaprakash-ibm/wip-bluefs-spillover-config-validation
common/options: restrict spillover configuration options to valid ranges

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>
2026-07-22 14:18:46 +05:30
Hezko
12f33ad77c
Merge pull request #70035 from Hezko/fix-rbd-application
mgr/nvmeof: change pool application to nvmeof-meta instead of rbd
2026-07-22 11:42:13 +03:00
pujashahu
3ffe684ee1 mgr/dashboard: Add RADOS Namespace Support in NVMe-oF UI
Fixes: https://tracker.ceph.com/issues/78330

Signed-off-by: pujaoshahu <pshahu@redhat.com>
2026-07-22 14:09:39 +05:30
Tomer Haskalovitch
a419992330 mgr/dashboard: fix list_hosts and list_locations nvmeof cli commands empty table bug
Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>
2026-07-22 11:27:29 +03:00
Naman Munet
967e888682 mgr/dashboard: Edit should support setting one or more rgw service configuration individually, when multiple rgw service exist
fixes: https://tracker.ceph.com/issues/78332

Before
===
The configuration form had a TypeScript compilation error and could not properly support editing client-specific configurations. The form lacked the ability to:

- Configure different values for specific client entities (e.g., client.rgw.*, client.admin, client.radosgw-gateway)
- No way to Add/remove multiple client entity entries dynamically
- No way to provide autocomplete suggestions for existing client entities

After
===
Users can now edit configuration options individually for any client entity, with a clean UI that supports:

- Generic client entity support: Configure any client.* entity (RGW daemons, admin users, CephFS clients, etc.)
- Autocomplete with custom values: Select from existing client entities or enter custom ones (e.g., client.rgw.my-daemon, client.admin)
- Dynamic entry management: Add/remove multiple client configuration entries

Signed-off-by: Naman Munet <naman.munet@ibm.com>
2026-07-22 13:34:29 +05:30
Afreen Misbah
609d540833 mgr/dashboard: Fix spacings in donut chart in resiliency card
Fixes https://tracker.ceph.com/issues/78507

Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-22 13:33:07 +05:30
Afreen Misbah
7b29dcc7e6
Merge pull request #68495 from rhcs-dashboard/76169-inconsistant-tooltip-timestamp-format-performance-chart
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-22 13:25:57 +05:30
Afreen Misbah
d840b598ba
Merge pull request #69829 from rhcs-dashboard/77813-admin-user-permissions
mgr/dashboard: restricting admin user to revoke its own admin permissions

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-07-22 13:21:15 +05:30
Afreen Misbah
df499f3134
Merge pull request #66607 from rhcs-dashboard/carbonized-import-modal
mgr/dashboard: carbonized-import-modal

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Devika Babrekar <devika.babrekar@ibm.com>
2026-07-22 13:20:35 +05:30
Afreen Misbah
37a9e87bfc
Merge pull request #70412 from rhcs-dashboard/disable-telemetry
mgr/dashboard: disable ibm telemetry on CI env

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-22 13:04:07 +05:30
Dnyaneshwari Talwekar
1599d7fd50 mgr/dashboard: Subvolume group and subvolume not listed in NFS Share form on edit.
Signed-off-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

Fixes: https://tracker.ceph.com/issues/78281
2026-07-22 12:31:47 +05:30
Nizamudeen A
0cbf62c035 mgr/dashboard: replace gherkin-lint with prettier
I checked a couple of gherkin linters as an alternative but all of them
are not really well maintained. There are couple of new projeects but
even they are simply maintained by a single guy so i don't think it'll
get enough contributions later on. So instead, I am moving it to
prettier. Although we miss some functional validation, it'll take care
of some basic formatting.

Fixes: https://tracker.ceph.com/issues/78528
Signed-off-by: Nizamudeen A <nia@redhat.com>
2026-07-22 12:31:32 +05:30
Nizamudeen A
674605736a
Merge pull request #70361 from rhcs-dashboard/login-username-issue
mgr/dashboard: New dashboard user creation is successful with long characters > 256 but login fails


Reviewed-by: Nizamudeen A <nia@redhat.com>
2026-07-22 12:15:42 +05:30
Kautilya Tripathi
f7b898ab6c crimson/seastore: keep on-page checksum in sync after merge_content_to
When a rewrite transaction publishes via no-conflict handoff, merge_content_to
can update an already CRC-stamped initial-pending fixedkv page (copy-dest of a
concurrent split). It refreshed last_committed_crc but not the in-extent phy
checksum, so the page could be written with new content and a stale stamp.
Debug asserts only compare last_committed_crc to calc_crc32c(), so the mismatch
survived until a later cold read aborted with "fixedkv extent checksum
inconsistent".

Recalculate the CRC and call update_in_extent_chksum_field together in both
LBALeafNode and FixedKVInternalNode merge_content_to, matching
apply_delta_and_adjust_crc / update_lba_mappings.

Fixes: https://tracker.ceph.com/issues/77022
Signed-off-by: Kautilya Tripathi <kautilya.tripathi@ibm.com>
2026-07-22 11:32:25 +05:30
Nizamudeen A
0690deaa34 mgr/dashboard: disable ibm telemetry on CI env
looks like ibm telemetry is being run when you do `npm install` on CI
env. here's disabling that

Signed-off-by: Nizamudeen A <nia@redhat.com>
2026-07-22 09:21:50 +05:30
Kefu Chai
57f86c05a0
Merge pull request #69964 from sunyuechi/fix-crimson-pg-recovery-exception-scope
crimson/osd/pg_recovery: fix exception handler scope in recover_missing

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-22 09:04:46 +08:00
Yuri Weinstein
ccd0a90d4f
Merge pull request #70398 from yuriw/wip-yuriw-ptl-tool-auth-fail-fast
script/ptl-tool: fail fast and cleanly on invalid Redmine/GitHub credentials

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2026-07-21 16:56:30 -07:00
Yuri Weinstein
e04840b619
Merge pull request #70397 from yuriw/wip-yuriw-ptl-tool-new-qa-links
ptl-tool: add New QA Links to update-qa notification

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2026-07-21 16:39:59 -07:00
SrinivasaBharathKanta
4cb45f38b9
Merge pull request #70152 from sunyuechi/wip-osd-health-metrics-guard
osd: guard max_element end() deref in get_health_metrics
2026-07-22 04:41:48 +05:30
SrinivasaBharathKanta
d2003750cd
Merge pull request #70142 from sunyuechi/fix-ec-omap-iterator-invalidation
osd/ECBackend: fix iterator invalidation in omap_get
2026-07-22 04:41:04 +05:30
Adam C. Emerson
5a176d3f83
rgw/policy: Add rgw-policy-test command line tool
Annotated evaluation of a policy.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2026-07-21 17:55:08 -04:00
Adam C. Emerson
3a1686cf29
rgw: Add PrincipalIdentity for use in testing and utilities
This is an identity backed by a `Principal`, for use when we don't
have any sort of RGW driver.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2026-07-21 17:55:08 -04:00
Adam C. Emerson
7bdb627a85
rgw/policy: Allow logging policy evaluation
When the `rgw_copious_policy_logging` option is set to true, a
detailed trace of policy evaluation is logged.

Assisted-by: Claude Opus 4.6 <noreply@anthropic.com>
(I wanted to see how it performed on rebase conflict resolution.)
Assisted-by: Claude Fable 5 <noreply@anthropic.com>
(I figured I should have it look at the policy evaluation code for
bugs while it's included, and it found a few mistakes in output.)
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2026-07-21 17:55:08 -04:00
Adam C. Emerson
7582c3200f
rgw: Make rgw::auth::Identity libfmt-able
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2026-07-21 17:55:08 -04:00
Adam C. Emerson
05d82dab5f
rgw: Make ARNs libfmt-able
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2026-07-21 17:55:08 -04:00
Adam C. Emerson
e885b11c1c
log: Add StringEntry to avoid using ostream
Create a log entry where we can fill a buffer with `format` instead of
needing to go through stringstream.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2026-07-21 17:55:08 -04:00
Adam C. Emerson
8191c81344
include/types: Add std::unordered_multimap support
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2026-07-21 17:55:08 -04:00
Adam C. Emerson
c62ee110cf
rgw/policy: Add missing strings for actions
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2026-07-21 17:55:08 -04:00
Adam C. Emerson
12d88e88b3
rgw/policy: Properly spell RemoveClientIdFromOIDCProvider in table
Typo in a lookup table.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2026-07-21 17:55:08 -04:00
Tomer Haskalovitch
a2057e2ea0 mgr/nvmeof: change pool application to nvmeof-meta instead of rbd
Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>
2026-07-21 23:04:58 +03:00
Afreen Misbah
0f2fd79457 mgr/dashboard: add view alerts button in data resiliency tab
Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-22 01:04:57 +05:30
Yuri Weinstein
f65b96c097 script/ptl-tool: add unit tests for the credential fail-fast fix
Extract the Redmine auth check added in the previous commit into a
standalone verify_redmine_auth(R) function so it can be unit tested
in isolation, and add src/script/test_ptl_tool.py covering:

- verify_redmine_auth(): invalid key (AuthError), a key that
  authenticates but lacks permission (ForbiddenError), a valid key,
  and confirming unrelated Redmine errors (ServerError) still
  propagate normally rather than being misreported as a credentials
  problem.
- AuditReport.post_consolidated_review(): logs success on 2xx, logs
  an error (rather than failing silently) on a non-2xx GitHub
  response, never calls out to GitHub under --dry-run, and is a
  no-op when the report has no issues to post.

No behavior change to verify_redmine_auth() itself -- this is a pure
extraction plus tests. ptl-tool.py has no existing test suite, so
these are self-contained (mocked Redmine/requests, no network access,
no real git checkout needed) and run with plain pytest; see the
module docstring for the exact invocation.

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2026-07-21 09:58:00 -07:00
Yuri Weinstein
f7d688861a ptl-tool: use branch name as the QA link text, not a generic label
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2026-07-21 16:55:11 +00:00
Yuri Weinstein
8bb99f9aee script/ptl-tool: fail fast and cleanly on invalid Redmine/GitHub credentials
Invalid credentials were handled inconsistently:

- A missing GitHub/Redmine token was caught with a clear error at
  startup, but an invalid or expired one was not: the Redmine
  client is constructed lazily by python-redmine, so a bad
  ~/.redmine_key was never actually exercised until
  manage_qa_tracker() called R.project.get() -- which runs *after*
  PRs have been merged and the integration branch/tag has already
  been pushed to ceph-ci. The result was an unhandled traceback
  well after real side effects had already happened, with no
  actionable message.

- post_consolidated_review() posted the consolidated audit review
  to GitHub without checking the response status at all, so a bad
  or under-scoped token would fail silently there while every other
  GitHub write call in the file already checks status_code.

Fix both:

- Immediately after constructing the Redmine client, make a cheap
  R.user.get('current') call and catch AuthError/ForbiddenError,
  exiting with a clear message before any merging or pushing
  happens.
- Check the response status in post_consolidated_review() and log
  an error on failure, matching the pattern already used by the
  other GitHub POST call sites in this file.

Verified with py_compile plus live --dry-run runs against
tracker.ceph.com: a deliberately invalid PTL_TOOL_REDMINE_API_KEY
now exits 1 immediately with a clear message and no git side
effects, while a valid key proceeds exactly as before.

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2026-07-21 09:50:21 -07:00
Yuri Weinstein
7e5aa0a2c1 ptl-tool: add New QA Links to update-qa notification
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2026-07-21 16:39:20 +00:00
Afreen Misbah
04cea032ec
Merge pull request #68075 from syedali237/tests-rbd
mgr/dashboard (test) : RBD snapshot mirroring toggle test

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>
2026-07-21 22:04:39 +05:30
Afreen Misbah
a9a6255cd2 mgr/dashboard: Sanitize toast HTML and optimize notification service
- Escape user-controlled content before Carbon innerHTML rendering
- Whitelist localStorage deserialization keys, prune stale read map
- Replace deep equality with ID comparison in removeToast
- Gate ngAfterViewChecked DOM queries behind a dirty flag
- Deduplicate by title+type+message; show occurrences count
- Fix bell icon unread dot clearing on new occurrences
- Fix view more pre-selection via route queryParams observable
- Save individual notifications instead of HTML-merged messages
- Convert notifications bell component to signals with OnPush
- Add OnPush to notification-item and toast components
- Add e2e tests
- Fix styling of notification panel

Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-21 21:34:11 +05:30
Patrick Donnelly
091354ea59
Merge PR #70205 into main
* refs/pull/70205/head:
	script: s/Backports/Backport/

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-21 11:31:57 -04:00
Mykola Golub
ccd9d3e94b
Merge pull request #59902 from trociny/wip-68169
rgw_gc: don't fail when processing entry with nonexisting pool

Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>
2026-07-21 18:23:36 +03:00
Jaya Prakash
793b757d3d common/options: restrict spillover configuration options to valid ranges
Fixes: https://tracker.ceph.com/issues/78492

Signed-off-by: Jaya Prakash <jayaprakash@ibm.com>
2026-07-21 15:21:47 +00:00
Alex Ainscow
f0c13de500
Merge pull request #68939 from aainscow/zero_sized_object
test/osd: Fix event loop context violations in test fixtures

Reviewed-by: Bill Scales <bill_scales@uk.ibm.com>
2026-07-21 16:13:58 +01:00
Yuri Weinstein
1cadb4cd14
Merge pull request #70350 from yuriw/wip-yuriw-ptl-tool-push-log
ptl-tool: log the ceph-ci push (and the equivalent git command)

Reviewed-by: Patrick Donnelly <pdonnell@redhat.com>
2026-07-21 07:48:50 -07:00
John Mulligan
d0bd301842
Merge pull request #70229 from Sodani/shsodani_exo_fix
mgr/smb: add support for RGW shares with external Ceph clusters

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Anoop C S <anoopcs@cryptolab.net>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2026-07-21 10:40:51 -04:00
Afreen Misbah
8cb0932382 mgr/dashboard: Add View more link to toasts
- adds view more link to toast which takes to view full toast message
- allow notifications to be clickable in notification panel taking user to the particular notification
- add routes to eahc notitification in notificatio page
- extend duration for error notifications - as per carbon error notifications should persist longer since users need time to read

Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-21 19:38:51 +05:30
Afreen Misbah
0834020077 mgr/dashboard: Truncate notification toasts
- truncated to 3 lines
- maximum 3 visible toasts
- duplicated toasts show "first maessage (+n more)"

Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-21 19:38:40 +05:30
Sridhar Seshasayee
b74a7f9db7
Merge pull request #70357 from sseshasa/wip-fix-ok-to-upgrade-min-pg-errmsg
mgr/DaemonServer: Make an ok-to-upgrade error message more generic

Reviewed-by: Nitzan Mordechai <nmordech@ibm.com>
2026-07-21 19:38:18 +05:30
Afreen Misbah
64e8dc794d
Merge pull request #69924 from syedali237/dashboard/notif-dest
mgr/dashboard : migrated notification destination tabs to resource pages

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Abhishek Desai <abhishek.desai1@ibm.com>
2026-07-21 19:13:29 +05:30
Afreen Misbah
0077832b38
Merge pull request #70244 from rhcs-dashboard/fix-e2e
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-21 18:11:52 +05:30
Dnyaneshwari Talwekar
374e7a7796 mgr/dashboard: New dashboard user creation is successful with long characters > 256 but login fails
Signed-off-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

Fixes: https://tracker.ceph.com/issues/78439
2026-07-21 17:42:50 +05:30
Afreen Misbah
e197383c91
Merge pull request #70214 from rhcs-dashboard/fix-78232-main
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-21 17:24:27 +05:30
Sagar Gopale
2c133a33f5 mgr/dashboard: carbonized-import-modal
Fixes: https://tracker.ceph.com/issues/74162

Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-07-21 17:12:52 +05:30
Abhishek Desai
0d0a241ba7 mgr/dashboard: grant hosts read to block-manager role
Block-manager users were denied access to the NVMe/TCP page because
host listing APIs require hosts read permission.

Fixes: https://tracker.ceph.com/issues/77952
Signed-off-by: Abhishek Desai <adesai@redhat.com>
2026-07-21 17:08:56 +05:30
Afreen Misbah
dde3f89434
Merge pull request #70344 from rhcs-dashboard/fix-78423-main
mgr/dashboard: fix rgw service form realm/zg/zone selection

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-21 17:02:45 +05:30
Aashish Sharma
125bb2235e
Merge pull request #70216 from rhcs-dashboard/fix-78233-main
mgr/dashboard: set MOTD fails on FIPS enabled systems

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-21 15:03:15 +05:30
Syed Ali Ul Hasan
6db9394d1a mgr/dashboard: fix nfs toggle test
Fixes: https://tracker.ceph.com/issues/75986
Signed-off-by: Syed Ali Ul Hasan <syedaliulhasan19@gmail.com>
2026-07-21 14:51:47 +05:30
Afreen Misbah
8d446a82e1
Merge pull request #69830 from rhcs-dashboard/77815-smb-rate-limiting
Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
Reviewed-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-07-21 14:41:23 +05:30
Afreen Misbah
8176a0ec08
Merge pull request #69430 from rhcs-dashboard/77369-ec-profile-creation-form-simplification
Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-07-21 14:41:15 +05:30
Afreen Misbah
5f0c1234dc
Merge pull request #69639 from rhcs-dashboard/77570-disabling-multisite-actions-dropdown-for-readonly-user
Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-07-21 14:41:07 +05:30
Syed Ali Ul Hasan
5293360de7 mgr/dashboard: migrated notif dest table tabs to resource pages
- Fixes: https://tracker.ceph.com/issues/77541

Signed-off-by: Syed Ali Ul Hasan <syedaliulhasan19@gmail.com>
2026-07-21 14:39:59 +05:30
Devika Babrekar
38a06d81f1 mgr/dashboard: moving filter under cluster and audit logs tab
Signed-off-by: Devika Babrekar <devika.babrekar@ibm.com>
2026-07-21 14:22:36 +05:30
Mykola Golub
73535b105d rgw_gc: don't fail when processing entry with nonexisting pool
Fixes: https://tracker.ceph.com/issues/68169
Signed-off-by: Mykola Golub <mykola.golub@clyso.com>
2026-07-21 10:50:45 +03:00
Dnyaneshwari Talwekar
8dbcc8e3db mgr/dashboard: Irrelevant description shown for the Realm field on the RGW service creation page.
Signed-off-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>

Fixes: https://tracker.ceph.com/issues/78442
2026-07-21 13:01:08 +05:30
baum
c77c5eb8db
Merge pull request #69650 from baum/nvmeof-host-maintenance
cephadm: disable/enable NVMe-oF gateways during host maintenance
2026-07-21 14:09:24 +07:00
Naman Munet
3aa2b17a04 mgr/dashboard: fix dashboard cephadm e2e
fixes: https://tracker.ceph.com/issues/78279

Signed-off-by: Naman Munet <naman.munet@ibm.com>
2026-07-21 12:02:06 +05:30
Aashish Sharma
7927cf51e0 mgr/dashboard: set MOTD fails on FIPS enabled systems
Root cause: On FIPS-compliant systems, the OpenSSL provider rejects hashlib.md5() calls that lack the usedforsecurity=False flag, because MD5 is not an approved algorithm for security-sensitive use. This causes the MOTD set command — and the /api/motd create endpoint — to raise a ValueError and crash whenever a new MOTD is saved.

Fix: Pass usedforsecurity=False to hashlib.md5(). This tells the FIPS provider that MD5 is being used only as a non-cryptographic checksum (to detect whether a user has already dismissed a specific MOTD message), which is a legitimate use-case that FIPS allows.

Fixes: https://tracker.ceph.com/issues/78233

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2026-07-21 11:48:28 +05:30
Avan Thakkar
ef160631a6 mgr/smb: only resync clusters that reference a changed resource
_find_modifications() regenerated every cluster whenever a JoinAuth,
UsersAndGroups, TLSCredential, or ExternalCephCluster changed, since
these can be shared across clusters. Use the existing lookup
helpers to resync only the clusters that actually reference it.

For rgw creds resync everything(just like old way), as narrowing
it scope means scanning every share in the store (which may not scale well),
so not worth a change.

Fixes: https://tracker.ceph.com/issues/78245
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2026-07-21 11:45:30 +05:30
Aashish Sharma
7dfb439308 mgr/dashboard: fix rgw service form realm/zg/zone selection
Fixes: https://tracker.ceph.com/issues/78423

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2026-07-21 11:04:38 +05:30
Sridhar Seshasayee
afd0b924e2 mgr/DaemonServer: Make an ok-to-upgrade error message more generic
When no OSDs within a CRUSH bucket can be found to upgrade, the earlier error
message indicated the minimum number of PGs affected if any OSD was removed
from the set. For e.g.,

"Error EBUSY: unsafe to upgrade osd(s) at this time (at least X PG(s) will
become offline if any OSD out of the Y in CRUSH bucket 'foo' is stopped)"

This was inaccurate in some cases because the number of PGs affected is
determined only from the last tested OSD in the CRUSH bucket based on the sort
order (i.e., OSD with least number of PGs per OSD). Based on the CRUSH rules
and placement, this doesn't mean that the least number of PGs are affected if
this OSD is stopped. There may be other OSDs in the set that affect a smaller
number of PGs if made offline.

But with the way ok-to-upgrade logic uses the convergence factor, not all OSDs
would be tested for the offline pg check. Therefore, it is not possible to
accurately determine the minimum number affected PGs.

In view of the above, the error message is modified as shown below to be
slightly more generic but still convey the reason:

"Error EBUSY: unsafe to upgrade osd(s) at this time (one or more PG(s) will
become offline if any OSD out of the Y in CRUSH bucket 'foo' is stopped)"

Fixes: https://tracker.ceph.com/issues/78425
Signed-off-by: Sridhar Seshasayee <sridhar.seshasayee@ibm.com>
2026-07-21 10:43:28 +05:30
Aashish Sharma
b6b591dcce python-common/rgw: fix get_realm_tokens returning empty result for realms with a secondary zone
Issue: The Export Multi-site Realm Token feature in the Dashboard showed no token and no realm details for any realm that had a secondary zone configured (i.e. the realm had been replicated to another cluster). The API endpoint GET /api/rgw/realm/get_realm_tokens returned an empty list or raised a 500 Internal Server Error.

Fixes: https://tracker.ceph.com/issues/78232

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2026-07-21 10:33:08 +05:30
Yuri Weinstein
e820641861 ptl-tool: log the ceph-ci push (and the equivalent git command)
Previously, a real (non-dry-run) push of the integration branch/tag to
ceph-ci was silent -- only the "[DRY RUN] Would push ..." path logged
anything. Add matching log.info() calls on the real-push path, and
include the literal `git push <remote> <ref>` invocation in the message
so it can be copy-pasted and re-run manually if needed (e.g. after a
network blip, or to re-push without re-running the whole merge).

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2026-07-20 17:44:47 -07:00
Igor Fedotov
ec92d36e6e
Merge pull request #69030 from ifed01/wip-ifed-better-slow-ops-dump
os/bluestore,kv/rocksdbstore: more information about slow KV ops

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>
2026-07-21 01:25:16 +03:00
Jaya Prakash
42bf941429
Merge pull request #69971 from Jayaprakash-ibm/wip-fix-kerneldevice-aio-stop
blk/kernel : skip AIO thread for devices with size < block_size

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>
Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
2026-07-21 00:02:19 +05:30
Jaya Prakash
9598f49fda
Merge pull request #69271 from aclamk/aclamk-bs-fix-unsharing-on-remove
bluestore: Fix unsharing blobs on remove

Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
Reviewed-by: Jaya Prakash <jayaprakash@ibm.com>
2026-07-21 00:01:39 +05:30
Jaya Prakash
ee78db3ec0
Merge pull request #69633 from Prachi-Lasurkar/wip-77331-both_fragmentation_and_score_have_the_same_label
os/bluestore: Rename allocator score output label

Reviewed-by: Jaya Prakash <jayaprakash@ibm.com>
2026-07-21 00:00:20 +05:30
Jaya Prakash
998c16a0eb
Merge pull request #70217 from Jayaprakash-ibm/wip-discard-during-file-migration
os/bluestore: use _release_pending_allocations() during file migration

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>
Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
2026-07-20 23:59:43 +05:30
Afreen Misbah
04fff85bf0
Merge pull request #70191 from rhcs-dashboard/feat-unread-notification
mgr/dashboard: redesign notifications page for unread notifications

Reviewed-by: Devika Babrekar <devika.babrekar@ibm.com>
2026-07-20 22:09:56 +05:30
Abhishek Desai
21e7419557 mgr/dashboard : Fix access denied issue for readonly user and user role
fixes : https://tracker.ceph.com/issues/78418
Signed-off-by: Abhishek Desai <abhishek.desai1@ibm.com>
2026-07-20 20:37:06 +05:30
benhanokh
36374fc83f rgw: add rgw_enable_dedup_threads config to control per-RGW dedup participation
Add rgw_enable_dedup_threads (default true) and rgw_nfs_run_dedup_threads
(default false) config options to control whether a given RGW instance
participates in dedup background processing.

This follows the same pattern used by rgw_enable_gc_threads,
rgw_enable_lc_threads, and rgw_enable_restore_threads. The config is
evaluated at startup in init_dedup(); when disabled, the dedup Background
object is not created, the thread is not spawned, and the instance does
not register a RADOS watch on the dedup control object.

Disabled RGW instances are invisible to the dedup cluster coordination
mechanism -- they receive no notifications, cause no timeouts, and do
not grab shard tokens.

Signed-off-by: Gabriel BenHanokh <gbenhano@redhat.com>
Assisted-by: Cursor Opus-4.6 <cursoragent@cursor.com>
2026-07-20 17:25:52 +03:00
Gabriel Benhanokh
fc4249596a
Merge pull request #70310 from benhanokh/dedup_handle_notify_bug_fix
rgw/dedup: protects handle_notify processing of URGENT_MSG_RESTART ag…
2026-07-20 17:06:38 +03:00
Patrick Donnelly
bf9608037b
Merge PR #70190 into main
* refs/pull/70190/head:
	script/ptl-tool: add automatic HTTP retry adapter to requests session

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2026-07-20 09:46:19 -04:00
bluikko
0a1cfbc8ca
Merge pull request #70330 from bluikko/wip-doc-sphinx-warnings-nfs
doc/mgr: fix Sphinx warnings in nfs.rst
2026-07-20 20:45:04 +07:00
Patrick Donnelly
56ae3a65e5
Merge PR #70258 into main
* refs/pull/70258/head:
	.github/workflows/pr-checklist: run only on main

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2026-07-20 09:31:04 -04:00
Bill Scales
401e0a4b8b osd: Fix Valgrind UninitCondition error in ECCommon::read_result_t::print()
omap_complete in read_result_t can be uninitialized if ECSubReadReply
contains an empty map of omap_complete information.

Fixes: https://tracker.ceph.com/issues/76953

Signed-off-by: Bill Scales <bill_scales@uk.ibm.com>
2026-07-20 12:53:57 +01:00
Anthony D'Atri
9b761f07ee
Merge pull request #69011 from bluikko/wip-doc-man-ceph-monitor
doc/man: use Monitor consistently in ceph.rst
2026-07-20 07:48:06 -04:00
Venky Shankar
0d256f8062 Merge PR #52147 into main
* refs/pull/52147/head:
	PendingReleaseNotes: add an note about dmclock based scheduler for MDS
	doc/config-ref: mention about MDS dmclock configurations
	qa: use dmclock with fs:workload
	mds: add MDS dmClock scheduler for subvolume QoS support

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Robin H. Johnson <robbat2@orbis-terrarum.net>
2026-07-20 16:58:23 +05:30
Ville Ojamo
f71c97ea20 doc/mgr: fix Sphinx warnings in nfs.rst
Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>
2026-07-20 16:58:17 +07:00
Shweta Sodani
028b162d42 doc/mgr/smb: document manual RGW credentials for external cluster
Fixes: https://tracker.ceph.com/issues/77784
Signed-off-by: Shweta Sodani <ssodani@redhat.com>
2026-07-20 14:21:30 +05:30
Shweta Sodani
86f8616411 mgr/smb: Add test coverage for RGW share with external clusters
test cases include:
- RGW-only and mixed CephFS/RGW configurations
- Multiple shares and credential isolation

Fixes: https://tracker.ceph.com/issues/77784
Signed-off-by: Shweta Sodani <ssodani@redhat.com>
2026-07-20 14:14:28 +05:30
Shweta Sodani
a4b10fb820 mgr/smb: add support for RGW shares with external Ceph clusters
Implement support for creating RGW-backed SMB shares on external Ceph
clusters with user configuration.

Fixes: https://tracker.ceph.com/issues/77784
Signed-off-by: Shweta Sodani <ssodani@redhat.com>
2026-07-20 14:14:14 +05:30
Abhishek Desai
622483c52e mgr/dashboard: grant hosts read to smb-manager role
SMB cluster creation needs to list cluster hosts but the smb-manager
role lacked hosts read permission, causing Access Denied errors.

Fixes: https://tracker.ceph.com/issues/77952
Signed-off-by: Abhishek Desai <adesai@redhat.com>
2026-07-20 14:06:13 +05:30
Abhishek Desai
4bdd697d02 mgr/dashboard: grant pool read to cluster-manager role
Cluster-manager users were denied access to the CRUSH map page because
the underlying API requires pool read permission.

Fixes: https://tracker.ceph.com/issues/77952
Signed-off-by: Abhishek Desai <adesai@redhat.com>
2026-07-20 14:06:13 +05:30
Abhishek Desai
65034ed4e8 mgr/dashboard: exclude user scope from read-only system role
Read-only users could see the Settings icon but were denied access
to user management pages. Remove user read permission from the
read-only role so the administration menu stays hidden.

Fixes: https://tracker.ceph.com/issues/77952
Signed-off-by: Abhishek Desai <adesai@redhat.com>
2026-07-20 14:06:13 +05:30
Abhishek Desai
21c6df7d22 mgr/dashboard: hide Report an Issue for users without config-opt read
The Help menu option redirected read-only users to an Access Denied
page. Hide it when the user lacks config-opt read permission.

Fixes: https://tracker.ceph.com/issues/77952
Signed-off-by: Abhishek Desai <adesai@redhat.com>
2026-07-20 14:06:13 +05:30
Abhishek Desai
2fba7b733a mgr/dashboard: allow read-only users to view Silences page
Remove SilenceFormComponent from the silences list page, which
incorrectly required create permission and caused a blank page for
users with prometheus read access only.

Fixes: https://tracker.ceph.com/issues/77952
Signed-off-by: Abhishek Desai <adesai@redhat.com>

 Conflicts:
	src/pybind/mgr/dashboard/frontend/src/app/ceph/cluster/prometheus/prometheus-tabs/prometheus-tabs.component.html
        - Added permissions
2026-07-20 14:06:13 +05:30
SrinivasaBharathKanta
179b4aeed1
Merge pull request #69833 from NitzanMordhai/wip-nitzan-ceph-daemon-adding-timeout
pybind/ceph_daemon: add 10s timeout to admin socket client
2026-07-20 13:54:10 +05:30
Yuval Lifshitz
6a73f21660
Merge pull request #69200 from sujay-d07/rgw-kafka-gssapi-implementation
rgw/kafka: Adding missing support for GSSAPI bucket notification endpoint and relevant tests for it
2026-07-20 10:27:44 +03:00
Devika Babrekar
96ab03abe0 mgr/dashboard: Inconsistent tooltip timestamp format on performance card
Fixes: https://tracker.ceph.com/issues/76169

Signed-off-by: Devika Babrekar <devika.babrekar@ibm.com>
Co-authored-by: Aashish Sharma <Aashish.Sharma1@ibm.com>
2026-07-20 12:26:31 +05:30
Venky Shankar
192b4306b9
Merge pull request #69169 from dparmar18/i61482
mgr/nfs: warn when deprecated export/cluster delete commands are run

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2026-07-20 12:19:47 +05:30
Ville Ojamo
5b81b4eada doc/man: sync osd pool get/set flags in 8/ceph.rst
Add the flags listed for ceph osd pool set in ceph osd pool get.
Remove nonexistent flag debug_fake_ec_pool in ceph osd pool set.
Confirm the additions/removals from MonCommands.h.

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>
2026-07-20 13:08:12 +07:00
Hezko
314d194dcb
Merge pull request #70246 from Hezko/fix-force-param-err
pybind: Fix CLI error message when using force parameter if it doesn't exist
2026-07-20 01:15:02 +03:00
Matan Breizman
271817bbb8 crimson/seastore: fix LogNode overflow from unsigned underflow in expect_overflow
The can_ow branch of LogManager's expect_overflow computed:
```
remain = capacity() - get_last_pos() - reserved_len
```
all uint32 operands, once the log leaf node is near full
(get_last_pos() + reserved_len >= capacity()) this subtraction underflows.

The node never rolls to a new leaf, instead it overflows the
16 KB block at commit-time delta replay.

Captured at the replay overflow point (per-collection batching, 4K randwrite):

```
  LogNode OVERFLOW at replay: pos=16641 size=58 reserved_len=3098
                              reserved_size=6 capacity=16328 LIMIT=16384

  log_node.h : In function 'void LogKVNodeLayout::set_last_pos(uint32_t)',
               ceph_assert(pos <= LOG_NODE_BLOCK_SIZE)
```

Here reserved_len (3098) is accurate: ~13543 committed + 3098 pending =
16641 = pos, i.e. the node was already ~13.5 KB full and the predictor
should have rolled over. Instead capacity - get_last_pos - reserved_len
went negative (16328 - ~13273 - 3098 < 0), wrapped in uint32, and the
check passed; pos reached 16641 > 16384 and set_last_pos asserted.

This was exposed by the per-coll batching where many overwrites append in
a single txn.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-07-19 13:29:04 +00:00
Matan Breizman
32e28b9a70 crimson/seastore: cleanup collection-lock leftovers
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-07-19 13:29:04 +00:00
Matan Breizman
548f65f03e crimson/seastore: batch same-collection transactions
Previously, the per-collection ordering_lock provided both ordering and
single-holder exclusion, causing same-collection transactions to be serialized
one at a time.

Replace that with a per-collection queue and a single dispatch loop. When a
collection has a batch in flight, newly arriving transactions are queued.
The dispatch loop later drains the queue and merges transactions into one
SeaStore transaction (batch). Resulting in a one build, one ool_write and one journal record for the entire batch.

Only one batch is in flight per collection, so same-collection concurrency is
still prevented and ordering is preserved by the queue.

Callbacks are unchanged: FuturizedStore drains each caller's on_commit before
enqueueing, so only ops are merged. Each caller's promise is fulfilled when the
merged batch commits.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-07-19 13:29:04 +00:00
leonidc
1cdfb58f24
Merge pull request #67286 from leonidc/nvme-gw-show-all
nvmeofgw: introduce nvme-gw show-all
2026-07-19 13:47:10 +03:00
benhanokh
b4f1a63bfa rgw/dedup: protects handle_notify processing of URGENT_MSG_RESTART against bad messages
Tracker: https://ibm-ceph.atlassian.net/browse/IBMCEPH-16929

Signed-off-by: benhanokh <gbenhano@redhat.com>
2026-07-19 10:26:47 +03:00
leonidc
42b8bc661d
Merge pull request #69616 from leonidc/beacon_timer_in_gw_created
nvmeofgw: arm beacon timeouts for GWs in Created state
2026-07-19 09:41:00 +03:00
leonidc
f72cb692d6
Merge pull request #69490 from leonidc/fix_relocation_stretched
nvmeofgw: stretched cluster fix relocate
2026-07-19 09:40:38 +03:00
Patrick Donnelly
0d531d898c
Merge PR #70264 into main
* refs/pull/70264/head:
	debian/rules: Also exclude librgw and radosgw from dwz

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-18 21:35:14 -04:00
Patrick Donnelly
9afe94d201
Merge PR #70268 into main
* refs/pull/70268/head:
	common/lockdep: explicitly export g_lockdep visibility

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
2026-07-18 14:29:24 -04:00
Xuehan Xu
2c45b4a90c
Merge pull request #70068 from xxhdx1985126/wip-seastore-new-trans-cc
crimson/os/seastore/transaction: new conflict/no_conflict trans synchronization

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2026-07-18 17:31:47 +08:00
Jesse Williamson
d21a73039a Fixup for (EXPERIMENTAL) FoundationDB support
Better-defined generator semantics; many internal improvements,
a couple found bug fixes, minor feature enhancements.

Considerable simplification that streamlines and improves the
baseline performance of library internals. Updates tests and
example documentation.

  - Add inclusive()/exclusive() selector endpoints.
  - default is still: select { a, b } == [a, b).
  - endpoint semantics work for reads, pagination,
    range-erase, split planning.
  - tests: selector endpoint combinations; endpoint handling; multiple
    value types from generators..
  - runtime byte-pointer helpers now actually just runtime

Assisted-by: Codex:GPT-5

Signed-off-by: Jesse Williamson <jfw@ibm.com>
2026-07-17 18:25:03 -07:00
David Galloway
e92c0dfd58
Merge pull request #68911 from djgalloway/wip-63336
doc: sign-debs script for release process
2026-07-17 19:52:12 -04:00
Afreen Misbah
36ceca6b08 mgr/dashboard: redesign notifications page with master-detail layout
-  revamped UX
-  shows unread notifications
-  added back button
-  added clear all, and single notification delete

Fixes: https://tracker.ceph.com/issues/78209
Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-18 02:34:55 +05:30
David Galloway
a5fc039e7a
Merge pull request #70296 from djgalloway/wip-container-rocky10
container: default FROM_IMAGE to Rocky Linux 10
2026-07-17 15:57:48 -04:00
Oguzhan Ozmen
1543a248a4 rgw/admin: guard bilog autotrim on non-exporting (archive) zones
The background sync-log-trim thread already skips bucket trim on zones whose
sync module does not export data , but the admin command had no equivalent
guard, so running it on an archive zone could delete instance metadata the
zone is meant to retain.

So, refuse the command on non-exporting zones unless --yes-i-really-mean-it is
given.

Fixes: https://tracker.ceph.com/issues/78364
Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-07-17 19:26:46 +00:00
David Galloway
194e58aa9b container: default FROM_IMAGE to Rocky Linux 10
Since tentacle, the preferred base image for ceph containers has been
Rocky Linux 10, and the CI tag-naming logic in build.sh already assumes
rockylinux-10 is the default fromtag for every branch except reef and
squid.  The actual build default was never flipped, though: anything
that ran build.sh without FROM_IMAGE set (e.g. the release container
job in ceph-build) still got a CentOS Stream 9 base.

Flip the Containerfile ARG and the build.sh fallback to
docker.io/rockylinux/rockylinux:10 so umbrella and later build from
Rocky 10 by default.  Builds that want a different base can still pass
FROM_IMAGE explicitly, as the CI pipeline does.

Signed-off-by: David Galloway <david.galloway@ibm.com>
2026-07-17 14:34:44 -04:00
Shilpa Jagannath
3a64c5588d
Merge pull request #69293 from smanjara/wip-datalog-ec
rgw: fix log_remove() -EIO on non-existent FIFO shards
2026-07-17 10:13:17 -07:00
Patrick Donnelly
2ff0eaa28f
common/lockdep: explicitly export g_lockdep visibility
Aggressive Link-Time Optimization (LTO) and stricter linker defaults
in newer toolchains (like GCC 13 on Ubuntu 24.04) can strip or
localize external global variables if their visibility is not
explicitly defined. Tagging g_lockdep with default visibility ensures
it remains accessible in the dynamic symbol table for external
binaries like monmaptool.

Assisted-by: Gemini
Fixes: https://tracker.ceph.com/issues/78321
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-17 12:24:55 -04:00
Jesse Williamson
1f14c03514
Merge pull request #70054 from ceph/jfw-libfdb-reverse-select
libfdb: reverse-order selection
2026-07-17 09:20:46 -07:00
Ilya Dryomov
28b97363e3
Merge pull request #70232 from batrick/csc-membership-updates
doc/governance: csc membership updates

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Sage McTaggart <sagemct@ibm.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2026-07-17 18:03:50 +02:00
Bill Scales
9958e1399a osd: Fix Valgrind UninitCondition error in ECTransaction::Generate::Generate()
first_interval_in_write can be accessed before it has been initialized. This
is harmless as at worst it will just cause the 1st I/O to a FastEC PG to be
slightly slower than necessary by disabling some optimizations.

However it needs fixing so the teuthology Valgrind job can check for other
issues.

Fixes: https://tracker.ceph.com/issues/76953

Signed-off-by: Bill Scales <bill_scales@uk.ibm.com>
2026-07-17 17:00:20 +01:00
Afreen Misbah
0dfc834523
Merge pull request #69751 from rhcs-dashboard/fix-hardware-tab
mgr/dashboard: Fix hardware tab in health card

Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2026-07-17 20:31:46 +05:30
Oguzhan Ozmen
80f203ee2a rgw/bilog/trim: (non-functional) set some bilog trim events to level 1
Raise a few low-frequency, high-signal bilog trim events so they are
visible at the default log level:

Trim pace is bounded by the rgw trim config options and these events
fire at most once per bucket or once per trim cycle, so they are not
expected to be chatty.

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-07-17 14:24:16 +00:00
Oguzhan Ozmen
90f7bff07c rgw/bilog/trim: (non-functional) remove unused bucket_info from status shards CR
The peer -ENOENT handling in RGWReadRemoteStatusShardsCR no longer
consults the local bucket layout (whether Deleted flag set or not),
so its bucket_info member and the matching constructor arg are
not used anymore.

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-07-17 14:24:16 +00:00
Oguzhan Ozmen
35135ceef5 rgw/bilog/trim: fix stale bucket.instance metadata after peer deletion
When a peer returns -ENOENT for a bucket's sync status, the old code only
purged the log if the local layout already carried BucketLogType::Deleted.
That flag may never arrive once the deleting zone removes its own instance
(#70858), so trim aborted with -EINVAL and left bucket.instance metadata
orphaned.

But an all-peer -ENOENT is also ambiguous: a live bucket whose sync status is
not yet initialized looks the same. The Deleted flag is unreliable here,
and the local entrypoint is not authoritative during creation (the
instance syncs to peers before the entrypoint). So on all-peer -ENOENT,
confirm the bucket is really gone by reading its entrypoint from the
metadata master, which is authoritative and available immediately on
creation; if this zone is the metadata master its local read already
suffices. Only on confirmed deletion remove the orphaned instance
directly; on any failure, skip and retry on the next trim cycle.

Fixes: https://tracker.ceph.com/issues/70858
Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-07-17 14:24:16 +00:00
Oguzhan Ozmen
6cc8ee2043 rgw/test/multisite: tests for bilog autotrim -ENOENT interpretation
An all-peer -ENOENT does not by itself mean a bucket is deleted: a live
bucket whose sync status is not yet initialized returns -ENOENT too. Add
two testcases that verifies we do not accidentally delete instance of a
live bucket:

- test_bucket_log_trim_live_empty_bucket_not_removed: a data-less bucket
  never gets sync status, so every peer reports -ENOENT; autotrim must not
  remove its instance.
- test_bucket_log_trim_new_bucket_entrypoint_not_synced: a new bucket
  whose instance has synced to the secondary before its entrypoint must
  survive autotrim. A delay_meta_sync_bucket_entrypoint_store injection
  forces that window.

Tests: https://tracker.ceph.com/issues/70858
Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-07-17 14:24:16 +00:00
Oguzhan Ozmen
43518e2f3e rgw/test/multisite: test for bilog autotrim -ENOENT race after secondary reshard
Adds test_bucket_log_trim_enoent_race_after_reshard to reproduce the
exact failure from tracker #70858: when autotrim runs on the primary
zone first and fully cleans up (including removing its own bucket.instance
metadata), then an autotrim on the secondary zone asks the primary
and receives -ENOENT.

The secondary hasn't received the BucketLogType::Deleted
flag yet (metadata sync is stalled via delay injection), so the current code
left StatusShards{generation=0, shards=[]} which caused take_min_status()
to fail with -EINVAL, leaving the bucket.instance metadata permanently
orphaned.

Tests: https://tracker.ceph.com/issues/70858
Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-07-17 14:24:15 +00:00
Afreen Misbah
36b2e02d2e
Merge pull request #70046 from rhcs-dashboard/fix-IBMCEPH-15229
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-17 19:19:26 +05:30
Ronen Friedman
45cf397e7d
Merge pull request #70153 from ronen-fr/wip-rf-78124ecN
mon: require EC optimizations for Crimson EC pools

Reviewed-by: Alex Ainscow <aainscow@uk.ibm.com>
Reviewed-by: Bill Scales <bill_scales@uk.ibm.com>
2026-07-17 16:48:06 +03:00
Patrick Donnelly
d5f34ba8dd
.githubmap: correct Anthony's name
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-17 08:50:52 -04:00
Patrick Donnelly
6727eaba45
doc/governance: add Sage McTaggart to CSC
Fixes: https://tracker.ceph.com/issues/77238
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-17 08:50:52 -04:00
Patrick Donnelly
91856bacf4
.githubmap: add Sage McTaggart
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-17 08:50:52 -04:00
Patrick Donnelly
5b2cfd5e0b
doc/governance: add Kyle Bader
Fixes: https://tracker.ceph.com/issues/77572
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-17 08:50:52 -04:00
Patrick Donnelly
1ea97af1ad
.githubmap: add Kyle Bader
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-17 08:50:52 -04:00
Patrick Donnelly
bc478de29e
.githubmap: sort alphabetically
awk '/^#/ {print; next} {print | "sort --stable -f"}' .githubmap > .githubmap_sorted

This commit also removes two duplicates for zmc and nmshelke.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-17 08:50:12 -04:00
Devika Babrekar
c36c33a4d1 mgr/dashboard: updating SMB rate limiting form fields
Fixes: https://tracker.ceph.com/issues/77815
Signed-off-by: Devika Babrekar <devika.babrekar@ibm.com>
2026-07-17 17:05:53 +05:30
Devika Babrekar
31a3023ad6 mgr/dashboard:EC Profile creation form simplification and improvement
Fixes: https://tracker.ceph.com/issues/77369

Signed-off-by: Devika Babrekar <devika.babrekar@ibm.com>

 Conflicts:
	src/pybind/mgr/dashboard/frontend/src/app/ceph/pool/erasure-code-profile-form/erasure-code-profile-form-modal.component.html
-removed Directory field, as it is no longer needed in form
2026-07-17 17:05:35 +05:30
Devika Babrekar
bed41f0cb1 mgr/dashboard: Disable multisite Action dropdown for read-only user
Fixes: https://tracker.ceph.com/issues/77570
Signed-off-by: Devika Babrekar <devika.babrekar@ibm.com>

Conflicts:
	src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-multisite-details/rgw-multisite-details.component.html
- added disabled property on table component to disable the table
	src/pybind/mgr/dashboard/frontend/src/app/shared/datatable/table-actions/table-actions.component.html
- added disabled property on table component to disable the table
2026-07-17 17:04:50 +05:30
Vallari Agrawal
6020010ca1
mgr/cephadm: set mgr cap for NVMeoF to allow cmd "service dump"
Allow nvmeof daemons cap to run mgr command "service dump".
This helps to map gateway_id with hostname (info needed to resolve
nvmeof issue https://github.com/ceph/ceph-nvmeof/issues/1974)

Fixes: https://tracker.ceph.com/issues/78339

Signed-off-by: Vallari Agrawal <vallari.agrawal@ibm.com>
2026-07-17 16:57:01 +05:30
Afreen Misbah
ec6e3322e9
Merge pull request #69807 from syedali237/dashboard/accounts
mgr/dashboard : migrate accounts table tabs to resource pages

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>
om>
2026-07-17 16:36:23 +05:30
SrinivasaBharathKanta
5722fa0912
Merge pull request #69784 from sunyuechi/wip-osd-network-min-fix
osd: dump_osd_network min section now reports min not max
2026-07-17 15:24:49 +05:30
Xuehan Xu
fdad2c066b crimson/os/seastore/transaction: new conflict/no_conflict trans
synchronization

Use seastar::shared_mutex to synchronize transactions when committing
no_conflict ones.

Specifically, this commit achieves conflict/no_conflict trans
synchronization by:
1. The no_conflict trans acquires locks on all the stable extents which
   it modifies.
2. The conflicting trans acquires shared locks on all the stable extents
   which it modifies.
3. The no_conflict trans releases the locks after its committing.
4. The conflicting trans releases the shared locks once it enter the
   prepare pipeline phase.

This commits makes sure that:
1. only transactions accessing the extents that the no_conflict trans
   is committing will be blocked.
2. avoid persisting extents that a committing no_conflict trans is
   modified

Compared to the old approach, this commit's strategy is much more
straightforward and understandable.

This commit is more performant as the no_conflict trans only blocks
the trans the modified extent set of which intersects with it, while,
in the old approach, all other transactions will be blocked once a
conflicting trans is blocked because it's blocked after it enters the
prepare pipeline phase

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-17 17:51:04 +08:00
SrinivasaBharathKanta
6af4f7939b
Merge pull request #69870 from pablodelara/isal_update
isa-l: Update isa-l submodule from 2.32.0 to 2.32.1
2026-07-17 15:20:05 +05:30
SrinivasaBharathKanta
f0effae00a
Merge pull request #69822 from sunyuechi/wip-pick_address-iface
common/pick_address: match against the current iface in pick_iface
2026-07-17 15:19:09 +05:30
Afreen Misbah
685e738171 mgr/dashboard: Fix hardware tab health status
- fix overall health
- add warning health
- refactor

Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-17 14:57:59 +05:30
Afreen Misbah
ec822b631e
Merge pull request #70234 from rhcs-dashboard/fix-fs-snapshot-flaky
mgr/dashboard: resolve e2e cephfs snapshot test flakyness

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-17 14:12:05 +05:30
Ronen Friedman
8c69d4fde8
Merge pull request #69943 from ronen-fr/wip-rf-bucket
rgw/bucket_logging: fix dangling string_view from ternary temporary

Reviewed-by: Yuval Lifshitz <ylifshit@redhat.com>
2026-07-17 08:13:45 +03:00
Patrick Donnelly
12a823f9f1
Merge PR #70247 into main
* refs/pull/70247/head:
	mailmap, githubmap, organizationmap: add Anurag Raut

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-16 21:19:50 -04:00
Matthew N. Heler
18da8c4b90 s3tests: add tests for CopyObject tagging-directive
Add tests to validate that CopyObject honors x-amz-tagging-directive.

New tests:
- test_object_copy_retaining_tagging: default copy keeps the source tags
- test_object_copy_tagging_directive_copy: COPY keeps the source tags
- test_object_copy_tagging_ignored_without_directive: a tagging header is ignored without a REPLACE directive
- test_object_copy_replacing_metadata_and_copying_tagging: metadata REPLACE keeps the source tags
- test_object_copy_replacing_tagging: REPLACE applies the new tag set
- test_object_copy_replacing_tagging_with_none: REPLACE with no tags clears them
- test_object_copy_to_itself_replacing_tagging: retag onto the same key
- test_copy_object_replacing_tagging: replaced tags replicate across zones (multisite)

Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
2026-07-16 18:32:09 -05:00
Matthew N. Heler
e63f8d1aff rgw: honor CopyObject tagging-directive
CopyObject ignored x-amz-tagging-directive, so REPLACE never took
effect and copies kept the source object's tags. Honor the directive
independent of the metadata directive: REPLACE applies the given tags
(or clears them), COPY keeps the source tags.

Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
2026-07-16 18:32:09 -05:00
John Mulligan
7e4d2a6abc
Merge pull request #70218 from avanthakkar/fix-read-cap-smb-pool
mgr/cephadm: scope .smb pool read cap to its own namespace

Reviewed-by: Anoop C S <anoopcs@cryptolab.net>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Shweta Sodani <ssodani@redhat.com>
2026-07-16 18:43:40 -04:00
John Mulligan
2648de4556
Merge pull request #70238 from anoopcs9/update-ctdbd-dirs
cephadm/smb: Update CTDB bind mount for new socket path

Reviewed-by: Shweta Sodani <ssodani@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Xavi Hernandez <xhernandez@gmail.com>
2026-07-16 18:40:36 -04:00
Adam Emerson
af595ea7fb
Merge pull request #69876 from thmour/tmo-quota-fix
rgw: rgw_rest_account.cc: Fix quota variable types from int32_t to int64_t

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2026-07-16 17:03:20 -04:00
Adam Emerson
558d6c0683
Merge pull request #69774 from mheler/wip-72013
rgw/http: skip pause/resume when the request already finished

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2026-07-16 17:02:35 -04:00
Adam Emerson
99d4d2355d
Merge pull request #68191 from sunyuechi/xxhash-rvv
src/xxHash: bump xxHash submodule to enable RISC-V RVV optimizations

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-16 17:01:45 -04:00
Adam Emerson
423acf868d
Merge pull request #67725 from mheler/wip-rgw-transition-recompress
rgw: compress/decompress objects during lifecycle transitions

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2026-07-16 16:52:07 -04:00
David Galloway
f17083b81e debian/rules: Also exclude librgw and radosgw from dwz
Fixes: https://tracker.ceph.com/issues/78312

Signed-off-by: David Galloway <david.galloway@ibm.com>
2026-07-16 16:25:07 -04:00
Syed Ali Ul Hasan
e74f66c576 mgr/dashboard: migrated accounts table tabs to resource pages
- Fixed E2E tests for user, roles, accounts

- Fixes: https://tracker.ceph.com/issues/77778

- Fixes: https://tracker.ceph.com/issues/78254

Signed-off-by: Syed Ali Ul Hasan <syedaliulhasan19@gmail.com>
2026-07-17 00:54:16 +05:30
Afreen Misbah
80110691fb
Merge pull request #70200 from rhcs-dashboard/fix-footer-view-all
mgr/dashboard: Fix the footer - view all button css in notification panel

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-07-17 00:41:25 +05:30
David Galloway
708737d3ec
Merge pull request #67057 from tchaikov/wip-debian-dwz
debian/rules: exclude ceph-osd-crimson from dwz compression
2026-07-16 13:09:53 -04:00
Kautilya Tripathi
041cfa6eb6
Merge pull request #69978 from knrt10/fix-pg-backfill-scan
crimson/osd: discard stale primary backfill scan after backfill finishes
2026-07-16 22:30:34 +05:30
Anoop C S
c069e89cf7 cephadm/smb: Update CTDB bind mount for new socket path
Samba recently changed CTDB to derive its runtime paths from Samba's
configure options instead of using a monolithic CTDB_RUNDIR[1].
With current packaging[2], the CTDB socket path moved from
/var/run/ctdb/ctdbd.socket to /run/samba/ctdb/ctdbd.socket.

Update the CTDB runtime bind mount target from /var/run/ctdb to
/run/samba/ctdb to match the new CTDB_SOCKETDIR. Additionally,
create the CTDB pid directory, /run/ctdb, and lock directory,
/var/lib/samba/lock/ctdb, on the host side as they are hidden
when their parent directories are bind mounted over.

Ideally these directories should be created by sambacc but sambacc is
included in samba-container images that may be built with different
Samba packages where these path configurations might not hold true.

[1] https://gitlab.com/samba-team/samba/-/merge_requests/4631
[2] https://github.com/samba-in-kubernetes/samba-build/blob/main/packaging/samba-master.spec.j2

Fixes: https://tracker.ceph.com/issues/78309
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2026-07-16 22:21:38 +05:30
Patrick Donnelly
0763f905bd
.github/workflows/pr-checklist: run only on main
We have no checklist for backports.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-16 11:29:47 -04:00
Afreen Misbah
83da7b9683
Merge pull request #70245 from rhcs-dashboard/hide-auth
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-16 20:51:19 +05:30
Afreen Misbah
87f0163e95
Merge pull request #70230 from rhcs-dashboard/service-table-fix
Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2026-07-16 20:44:13 +05:30
Yuval Lifshitz
f853590538 qa/rgw/kafka: add valgrind exception for cyrus-sasl leak
bug is in 3rd party lib:
https://github.com/cyrusimap/cyrus-sasl/issues/843

Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
2026-07-16 14:37:55 +00:00
Kefu Chai
0b50da6588 debian/rules: exclude ceph-osd-crimson from dwz compression
When building with DWZ enabled, the debian packaging fails with:
```
  dh_dwz: error: Aborting due to earlier error
```
Running the dwz command manually reveals the root cause:
```
  $ dwz -mdebian/ceph-osd-crimson/usr/lib/debug/.dwz/x86_64-linux-gnu/ceph-osd-crimson.debug \
    -M/usr/lib/debug/.dwz/x86_64-linux-gnu/ceph-osd-crimson.debug -- \
    debian/ceph-osd-crimson/usr/bin/ceph-osd-crimson \
    debian/ceph-osd-crimson/usr/bin/crimson-store-nbd
  dwz: debian/ceph-osd-crimson/usr/bin/ceph-osd-crimson: Too many DIEs, not optimizing
  dwz: Too few files for multifile optimization
```
The dwz tool has a limit on the number of DWARF DIEs (Debug Information
Entries) it can process. The ceph-osd-crimson binary, being a large C++
executable with extensive template usage, exceeds this limit, causing
dwz to exit with status 1 and fail the build.

This change excludes only ceph-osd-crimson from dwz processing using
the -X flag, allowing other binaries in the package to still benefit
from DWARF compression while avoiding the build failure.

Please note, we always disable DWZ in ceph-build's ceph-dev-pipeline.

Fixes: https://tracker.ceph.com/issues/78303
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-16 22:13:55 +08:00
Aishwarya Mathuria
614860f320 crimson/os/seastore/rbm: fix pass_admin to pass admin command to ioctl
The coroutine conversion accidentally passed nullptr to NVME_IOCTL_ADMIN_CMD,
so Identify always failed and SeaStore fell back to st_blksize (e.g. 128K),
breaking journal roll_start alignment on mkfs.

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
2026-07-16 17:32:33 +05:30
Kefu Chai
5b8ba02b00
Merge pull request #69855 from sunyuechi/wip-crimson-ec-backend-transaction-fixes
crimson/osd/ec_backend: fix transaction translation

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Shraddha Agrawal <shraddhaag@ibm.com>
Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-16 19:04:14 +08:00
Kefu Chai
105f5bad28
Merge pull request #70103 from tchaikov/wip-crimson-first-old-map
crimson/osd: start from `first` when older maps are trimmed

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
Reviewed-by: Aishwarya Mathuria <amathuri@redhat.com>
2026-07-16 19:01:37 +08:00
Matan Breizman
068a9df456
Merge pull request #70119 from tchaikov/wip-seastore-journal-greedy-dispatch
crimson/os/seastore/journal: dispatch records greedily while io slots are free

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-16 13:10:21 +03:00
Aashish Sharma
edb62beba3
Merge pull request #70213 from rhcs-dashboard/fix-78230-main
monitoring: Rename Ceph Object - Sync Overview - Replication (objects) from Source Zone

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-16 15:18:21 +05:30
Anurag Raut
0b1f4f698d mailmap, githubmap, organizationmap: add Anurag Raut
Signed-off-by: Anurag Raut <anuragtraut08@ibm.com>
2026-07-16 14:57:10 +05:30
Ronen Friedman
6ce9633687
Merge pull request #70219 from ronen-fr/wip-rf-fragments-crimson
Crimson/net: cap outgoing message batch size to prevent packet fragment overflow

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2026-07-16 12:13:47 +03:00
Xuehan Xu
2b1ed9e100 crimson/os/seastore/transaction: drop the old trans-blocking mechinary
This old trans-blocking mechinary can be hard to maintain in the future

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-16 16:44:17 +08:00
Tomer Haskalovitch
e8be5e2697 pybind: fix error message when using force parameter if it doesn't exist in a command's arguments schema
Fixes: https://tracker.ceph.com/issues/78291
Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>
2026-07-16 10:38:14 +03:00
sujay-d07
93a357c93c qa/rgw: Adding GSSAPI testing task for Bucket Notification Testing
This adds a Kerberos setup task, tells the Kafka task to emit GSSAPI listeners and broker principals, and updates notification tests to pass Kerberos credentials through to the bucket-notification suite. It enables the RGW Kafka notification coverage to run against GSSAPI-secured brokers.

Signed-off-by: sujay-d07 <sujaydongre07@gmail.com>
2026-07-16 12:12:20 +05:30
sujay-d07
803f698200 rgw/testing: add GSSAPI testing in bucket notification tests
Signed-off-by: sujay-d07 <sujaydongre07@gmail.com>
2026-07-16 12:12:20 +05:30
sujay-d07
1e9997d9be rgw/kafka: add support for GSSAPI mechanism for Kafka Bucket Notification Endpoint
Signed-off-by: sujay-d07 <sujaydongre07@gmail.com>
2026-07-16 12:12:20 +05:30
Venky Shankar
7d65abc9c1 Merge PR #68813 into main
* refs/pull/68813/head:
	tools/cephfs: make journal pointer/header failure explicit
	doc/cephfs: add note explainin the new batching technique
	tools/cephfs: document --max-rss usage in first-damage.py
	qa: test cephfs-journal-recovery event recover_dentries summary
	tools/cephfs: flush every journal event as soon as it is scanned

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-16 11:43:24 +05:30
pujashahu
2bf1b6e794 mgr/dashboard: mgr/dashboard: Hide Authentication section when "Allow All Hosts" is enabled
Fixes: https://tracker.ceph.com/issues/78280

Signed-off-by: pujaoshahu <pshahu@redhat.com>
2026-07-16 11:42:07 +05:30
Venky Shankar
eaf50ec581 Merge PR #68924 into main
* refs/pull/68924/head:
	tools/cephfs_mirror: Fix failed-to-syncing state priority across metrics paths

Reviewed-by: Karthik U S <karthik.u.s1@ibm.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
2026-07-16 11:39:01 +05:30
Venky Shankar
1bc793d81c
Merge pull request #69466 from kotreshhr/fs-mirror-stale-stat-size
tools/cephfs-mirror: Fix truncated sync from stale crawl-time file size

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2026-07-16 11:38:26 +05:30
Devika Babrekar
22e6ac309a mgr/dashboard: Fixing control flow on submit button for pool creation page
Fixes: https://tracker.ceph.com/issues/77030
Signed-off-by: Devika Babrekar <devika.babrekar@ibm.com>

Conflicts:
	src/pybind/mgr/dashboard/frontend/src/app/ceph/rgw/rgw-storage-class-form/rgw-storage-class-form.component.html
- updated conditions to render ui
2026-07-16 11:34:31 +05:30
Devika Babrekar
31565e1d39 mgr/dashboard: disabling role clearning button for admin user
Signed-off-by: Devika Babrekar <devika.babrekar@ibm.com>
2026-07-16 10:29:32 +05:30
Devika Babrekar
95a096f2c6 mgr/dashboard: restricting admin user to revoke its own admin permissions
Fixes: https://tracker.ceph.com/issues/77813

Signed-off-by: Devika Babrekar <devika.babrekar@ibm.com>

Conflicts:
	src/pybind/mgr/dashboard/frontend/src/app/core/auth/user-form/user-form.component.html
-conflict while rebasing for SSO related parameters
	src/pybind/mgr/dashboard/frontend/src/app/core/auth/user-form/user-form.component.ts
-conflict while rebasing for SSO related variables
2026-07-16 10:29:32 +05:30
Ilya Dryomov
5b1f1643a2
Merge pull request #69992 from sunyuechi/wip-rbd-mirror-unsupported-mode-return
rbd-mirror: return after finishing on unsupported mirror mode

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2026-07-16 00:23:11 +02:00
Ilya Dryomov
50402b0596
Merge pull request #70027 from sunyuechi/rbd-mirror-fix-peerspec-weak-ordering
rbd-mirror: fix strict weak ordering in PeerSpec::operator<

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2026-07-16 00:22:24 +02:00
Ilya Dryomov
10d04194d5
Merge pull request #70033 from sunyuechi/rbd-mirror-status-updater-unlock
rbd-mirror: take MirrorStatusUpdater lock by value in queue_update_task

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2026-07-16 00:21:24 +02:00
Ilya Dryomov
54cd7760b0
Merge pull request #70039 from VinayBhaskar-V/wip-group-snap-list
pybind/rbd: handle non-existent groups properly in list_snaps()

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2026-07-15 21:37:12 +02:00
Ilya Dryomov
6497e5ff17
Merge pull request #69623 from sunyuechi/wip-iocache-reply-leak-teardown
tools/immutable_object_cache: don't leak in-flight replies on teardown

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2026-07-15 21:35:21 +02:00
John Mulligan
4423f22cff
Merge pull request #69892 from phlogistonjohn/jjm-teuth-remotectl
smb: add a teuthology test that exercises the gRPC over TCP interface of smb remote-control

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Rabinarayan Panigrahi <rapanigr@redhat.com>
2026-07-15 12:56:20 -04:00
Oguzhan Ozmen
7a3b1ae30f
Merge pull request #70006 from BBoozmen/wip-oozmen-75716-75715
rgw: copy_obj_data() record uncompressed size for a compressed source
2026-07-15 12:31:30 -04:00
Pedro Gonzalez Gomez
253729b919 mgr/dashboard: resolve e2e cephfs snapshot test flakyness
Signed-off-by: Pedro Gonzalez Gomez <pegonzal@ibm.com>
2026-07-15 18:30:56 +02:00
Sun Yuechi
517f9ce8f4 test/crimson/omap: enlarge values under ASan to shrink depth-driven tests
innernode_split_merge_balancing, omap_iterate, list and monotonic_inc grow
the tree to a fixed depth, so under ASan a larger value reaches the same
depth (same split/merge paths) with far fewer keys. It stays below the
per-kv limit so inserts keep the normal path.

Scale the extent-count bound in innernode_split_merge_balancing with the
value size accordingly.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-15 08:54:33 -07:00
Alex Ainscow
2f58c109e5
Merge pull request #69000 from aainscow/primaryfirst_fix
osd: get_primary_shard should not call pgtemp_undo_primaryfirst

Reviewed-by: Bill Scales <bill_scales@uk.ibm.com>
2026-07-15 16:37:02 +01:00
Bill Scales
12b57e0254 osd: Fix PeeringState::proc_lease crash caused by race hazard
Fix race hazard in PeeringState::proc_lease where it read
a std::optional value which could be modified by another
thread at the same time causing an abort.

There is already a lock to protect accesses to this state
to make them thread safe, but the code wasn't using it.

Fixes: https://tracker.ceph.com/issues/54932

Assisted-by: IBM Bob 2.0.1
Signed-off-by: Bill Scales <bill_scales@uk.ibm.com>
2026-07-15 16:15:28 +01:00
Jaya Prakash
de7c5598f8 os/bluestore: Avoid migrating active files during spillover cleaning
Fixes: https://tracker.ceph.com/issues/78234

Signed-off-by: Jaya Prakash <jayaprakash@ibm.com>
2026-07-15 14:28:58 +00:00
Naman Munet
6ba6faa37c mgr/dashboard: services page flickering with empty table
fixes: https://tracker.ceph.com/issues/78247

why the changes were made:
The Services table had autoSave enabled by default, which persists table state (including sort configuration) to browser localStorage. This created a conflict because:

The table uses server-side pagination ([serverSide]="true")
The table auto-reloads every 5 seconds ([autoReload]="5000")
On initialization, the table would load stale sort configuration from localStorage
This cached sort state would override the intended default sort (by service_name ascending)
User sort interactions would be saved to localStorage but could conflict with server-side sort handling

Signed-off-by: Naman Munet <naman.munet@ibm.com>
2026-07-15 19:41:49 +05:30
Bill Scales
b5ed9812f3 osd: Fix race hazard that caused duplicated message after OSD marked down
After an OSD that has not crashed is marked down there is a race hazard
where a message can be deliever to the OSD a second time. Different
asserts are hit depending on which message is duplicated.

This issue is seen in tests which mark osds down as an error inject
and very occasionally in the field where there are other issues
(e.g. network problems) that are causing osds which have not
crashed to be marked down.

The fix is to change the order in which the OSD processes a
new epoch and OSDMap which marks the OSD as down, it must
first consume the map so that old messages will be filtered
out and then call rebind to reset all the connections.

Fixes: https://tracker.ceph.com/issues/58417

Assisted-by: IBM Bob 2.0.1
Signed-off-by: Bill Scales <bill_scales@uk.ibm.com>
2026-07-15 14:54:56 +01:00
Sun Yuechi
d440950eff client: don't use uninitialized keyid in fscrypt_dummy_encryption
When add_fscrypt_key() failed it jumped to the err label, which then
read the uninitialized keyid buffer into a key specifier and issued a
key removal for it. add_fscrypt_key() only populates keyid on success,
so this read uninitialized stack memory. Return the error directly
instead, since no key was added and there is nothing to remove.

Also stop overwriting the original error with remove_fscrypt_key()'s
return value on the policy-set failure path, so the real failure is
reported to the caller.

Fixes: https://tracker.ceph.com/issues/78246
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-15 21:28:52 +08:00
Sun Yuechi
5985acca83 client: fix _wrap_name reporting success on encryption failure
_wrap_name() returns bool, but two encryption error paths did "return r"
with r < 0, which converts to true. Callers then treated the failure as
success and used the unencrypted name. Return false instead.

Fixes: https://tracker.ceph.com/issues/78246
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-15 21:28:45 +08:00
Sun Yuechi
4a6e51c51a client: fix double unlock of client_lock in mount()
mount() holds client_lock through a unique_lock (cl), but the dummy
encryption path unlocked and relocked the underlying mutex directly,
bypassing cl. On fscrypt_dummy_encryption() failure it returns early
without relocking, so cl's destructor then unlocks an already-unlocked
mutex, which is undefined behavior. Only reachable when
client_fscrypt_dummy_encryption is enabled.

Unlock/relock through cl so its ownership state stays consistent.

Fixes: https://tracker.ceph.com/issues/78246
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-15 21:28:09 +08:00
kobi ginon
1d0eaf08ee
Merge pull request #69420 from kginonredhat/issue-77250-clean
qa/mgr: fix influx module_selftest health warning noise
2026-07-15 16:04:35 +03:00
Venky Shankar
991cfb9609 Merge PR #69320 into main
* refs/pull/69320/head:
	qa:/cephfs: separate out ceph-fuse upgrade tests under fs:upgrade
	qa/cephfs: reject ceph-fuse upgrade jobs to reduce failure noise

Reviewed-by: Kotresh Hiremath Ravishankar <khiremat@redhat.com>
2026-07-15 17:26:59 +05:30
Yuval Lifshitz
483fe17c2d
Merge pull request #70188 from yuvalif/wip-yuval-78214
test/kafka: support archived kafka versions
2026-07-15 14:54:37 +03:00
Ronen Friedman
3b09d27d4d rimson/net: cap outgoing message batch size to prevent seastar packet fragment overflow
During heavy backfill, sweep_out_pending_msgs_to_sent() could accumulate
more than 65535 buffer fragments in a single bufferlist, overflowing
seastar's uint16_t packet fragment counter and triggering an assertion.
Limit the sweep to crimson_osd_max_send_buffers fragments
per write; remaining messages are sent in subsequent dispatch iterations.

Fixes: https://tracker.ceph.com/issues/75600
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2026-07-15 11:37:32 +00:00
Afreen Misbah
cec298caa5 mgr/dashboard: add hardware health unit tests and status constants
Added Python unit tests and frontend unit tests

Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-15 16:55:40 +05:30
Afreen Misbah
7ec4b48e8a mgr/dashboard: fix hardware health card icon rendering
Hardware category icons  were rendering as empty SVGs because of wrong reference

Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-15 16:55:38 +05:30
Afreen Misbah
b269dc7e14 mgr/dashboard: fix hardware status parsing for health
Atollon return status as a plain string ("OK") instead of
the dict format ({"health": "OK", "state": "Enabled"}) used by
other BMCs. This caused KeyError/AttributeError.

Added _get_health() helper that handles both formats:
- dict with nested health key
- plain string status
- missing or non-dict components

Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-15 16:54:46 +05:30
Avan Thakkar
d60e8b8054 mgr/cephadm: scope .smb pool read cap to its own namespace
Scope the .smb pool read cap to the same namespace already
used by the adjacent cluster-meta rwx cap.

Fixes: https://tracker.ceph.com/issues/78235
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2026-07-15 16:52:00 +05:30
Jaya Prakash
10f0380bd4 os/bluestore: use _release_pending_allocations() during file migration
Fixes: https://tracker.ceph.com/issues/78234

Signed-off-by: Jaya Prakash <jayaprakash@ibm.com>
2026-07-15 11:07:08 +00:00
Afreen Misbah
9e1d1ce9f6 mgr/dashboard: Fix the footer - view all button css in notification panel
Fixes https://tracker.ceph.com/issues/78222

Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-15 16:27:24 +05:30
Ronen Friedman
2d4a87a178
Merge pull request #69940 from ronen-fr/wip-rf-336681-1upd
crimson/mgr: prevent pg stats report pile-up in overloaded store

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2026-07-15 12:33:22 +03:00
Sagar Gopale
cec4a14faa mgr/dashboard: add validation for username
Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-07-15 14:22:00 +05:30
Ali Masarwa
a0c926c71b
Merge pull request #70077 from AliMasarweh/wip-alimasa-global-cors
RGW|global CORS: fix missing headers

Reviewed-by: Naman Munet <naman.munet@ibm.com>
2026-07-15 11:49:09 +03:00
Aashish Sharma
ce24915123 monitoring: Rename Ceph Object - Sync Overview - Replication (objects) from Source Zone
Replication (objects) from Source Zone title is misleading, the panel actually shows How many objects are being replicated per second

Fixes: https://tracker.ceph.com/issues/78230

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2026-07-15 13:51:23 +05:30
Kefu Chai
9889bb6e3e
Merge pull request #69455 from sunyuechi/wip-crimson-messenger-thrash-mem
crimson/test: shrink messenger-thrash max_message_len to 1MB

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-15 15:06:01 +08:00
Sun Yuechi
e25810d879 Dockerfile.build: install sccache from distro packages when available
Prefer the distro-packaged sccache (installed via dnf) over downloading a
release tarball with curl. Fall back to the curl download when the package
is absent or on non-dnf distros, so existing behavior is unchanged.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-15 11:53:53 +08:00
Kefu Chai
c98598d388 script: s/Backports/Backport/
when referencing the "Backport" field in tracker.ceph.com, we
used "Backports". but the field is named "Backport" not "Backports".
in this change, we replace "Backports" with "Backport" when it
references this field.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-15 11:31:11 +08:00
Kefu Chai
d3d08a106d
Merge pull request #70056 from tchaikov/wip-debian-remove-unused-deps
debian: drop unused deps from ceph-mgr-modules-core.requires

Reviewed-by: David Galloway <david.galloway@ibm.com>
2026-07-15 11:06:10 +08:00
Laura Flores
8926e0cce3 qa/suites/upgrade: ignore expected telemetry warning
We already re-opt into telemetry immediately after upgrading,
but this warning still sometimes comes up and is detected
by the badness check. Since we are following the normal
procedure to silence the warning, the next step is to
just ignore this.

Fixes: https://tracker.ceph.com/issues/77382
Signed-off-by: Laura Flores <lflores@ibm.com>
2026-07-14 16:34:28 -05:00
Jesse F. Williamson
f393f3a585 Tweak block_generator<> to use transactors!
A pleasant improvement to the implementation.

Signed-off-by: Jesse F. Williamson <jfw@ibm.com>
2026-07-14 12:29:05 -07:00
mheler
bf449d4db2
Merge pull request #68922 from mheler/wip-rgw-cloud-tier-503-backoff
rgw/cloud-tier: retry on 503 from remote endpoint
2026-07-14 13:44:13 -05:00
Kotresh HR
0bb27c7de1 tools/cephfs-mirror: report epoch time in last snap sync time
Store sync completion time (last_synced_snap.sync_time_stamp)
as utime_t (Unix epoch) instead of coarse monotonic time so
the metric is meaningful to operators.

Fixes: https://tracker.ceph.com/issues/76506
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-07-14 20:37:04 +05:30
Patrick Donnelly
2ac7e1ea76
Merge PR #69886 into main
* refs/pull/69886/head:
	doc: squid 19.2.5 release notes

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-14 10:07:47 -04:00
Patrick Donnelly
1a1693b6e4
script/ptl-tool: add automatic HTTP retry adapter to requests session
Long-running local git operations and Redmine API calls can cause the idle
keep-alive connection to api.github.com to be closed by the remote server.
When the script later attempts to post automated PR comments, it fails with
http.client.RemoteDisconnected.

Fix this by mounting an urllib3 Retry adapter to the requests.Session,
automatically re-establishing dropped connections up to 3 times on
transient network or server errors (500, 502, 503, 504).

Assisted-by: Gemini
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-14 09:38:38 -04:00
Igor Fedotov
881d8c9144
Merge pull request #68503 from ifed01/wip-ifed-fix-ncb-expand
os/bluestore: do not add expanded space to allocator after allocmap recovery.

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>
2026-07-14 16:23:36 +03:00
Yuval Lifshitz
c0f1ffb960 test/kafka: support archived kafka versions
Fixes: https://tracker.ceph.com/issues/78214

Signed-off-by: Yuval Lifshitz <ylifshit@ibm.com>
2026-07-14 13:20:14 +00:00
Sun Yuechi
ae789567a2 cmake: fix sccache job limits when dist is unavailable
The "disabled" search only matches sccache's Disabled("disabled")
payload, not "dist-client feature not selected" or the NotConnected
states. Those reach the success branch, so the job counts become
SchedulerStatus-NOTFOUND, JOB_POOLS is never defined, and ninja
rejects the manifest. Key off the JSON lookup's error instead to
cover every non-dist state.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-14 06:15:35 -07:00
Redouane Kachach
d24cc5b497
Merge pull request #61670 from ShwetaBhosale1/fix_issue_69502_nfs_ratelimiting_doc
mgr/nfs: Documentation for NFS ratelimiting commands

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2026-07-14 14:30:52 +02:00
Kefu Chai
d2f6acf548
Merge pull request #70168 from sunyuechi/pybind-tempita-build-dir
pybind: write Tempita-processed pyx to the build directory

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2026-07-14 19:59:20 +08:00
Anthony D'Atri
b68f5a1d6f
Merge pull request #70138 from anthonyeleven/improvededup
doc/radosgw: Improve s3_objects_dedup.rst
2026-07-14 06:45:45 -04:00
Shweta Bhosale
2567dcce88 doc: mgr/nfs: Documentation for NFS ratelimiting commands
Fixes: https://tracker.ceph.com/issues/69502
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-07-14 14:57:32 +05:30
Alex Ainscow
84fc350a1f test/osd: check the offset cache after slice_map()
Recompute ro_start and ro_end with compute_ro_range() and compare them
against the cached values, and cover single-shard maps.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-14 02:08:02 -07:00
Sun Yuechi
0d1b69a32c osd/ECUtil: fix slice bounds in slice_map()
slice_map() accumulated the slice bounds against the wrong operands,
and included parity shards in ro_start and ro_end, which only track
data.

Introduced by 9e2841ab16 ("osd: Introduce optimized EC").

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-14 02:08:02 -07:00
Afreen Misbah
ef47d68738
Merge pull request #69968 from rhcs-dashboard/fix-rgw-cache-invalid
mgr/dashboard: Stale RGW user/account metadata cache in mgr after delete and recreate

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-14 13:46:08 +05:30
Sun Yuechi
d1ed18a720 pybind: write Tempita-processed pyx to the build directory
distutils_add_cython_module() runs setup.py with WORKING_DIRECTORY set
to the source directory, so every build leaks
{rados,cephfs,rbd,rgw}_processed.pyx into the source tree as untracked
files. Write the file to CYTHON_BUILD_DIR (already exported by that
CMake function) when set, keeping the cwd fallback for standalone
invocations.

The .pyx no longer sits next to the bindings' .pxd files, so add each
binding's source directory to include_path. Drop the build_dir
argument: it has no effect on absolute source paths.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-13 22:34:33 -07:00
Mohit Agrawal
1aa027d8f1 test/crimson: Add request_budget_retry in test_backfill.cc
Signed-off-by: Mohit Agrawal <mohit84.agrawal@gmail.com>
2026-07-14 10:28:56 +05:30
Mohit Agrawal
c6782dce47 crimson/osd: fix backfill deadlock in BackfillState::Waiting
BackfillState::Enqueuing unconditionally posts RequestWaiting{}
when budget available return false. Waiting can exit only via
ObjectPushed which requires an in-flight push. If the budget
is exhausted with no pushes in flight no objectPush will ever
arrive and the PG stalls permanently.

Solution: Introduce a dedicated BudgetBlocked state. When budget_available()
is false and tracked_objects_completed()==true post RequestBudgetBlocked{}
instead of RequestWaiting{}.BudgetBlocked calls get_backfill_throttle()
to suspend until a slots free up, then fires BudgetAvailable{} which
transitions directly back to Enqueuing to dispatch the next batch of pushes.

Fixes: https://tracker.ceph.com/issues/77804
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
2026-07-14 10:28:03 +05:30
Kefu Chai
a0897c7ecf
Merge pull request #69972 from vitaly-goot/vgoot/crimson/osd/fix-purged-peer-boot-crash
crimson/osd: fix boot crash when a peer OSD was purged across an osdm…

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2026-07-14 10:49:44 +08:00
Patrick Donnelly
bf1010e8bb
Merge PR #70162 into main
* refs/pull/70162/head:
	doc,.githubmap: add Aviv to CSC

Reviewed-by: Ilya Dryomov <idryomov@redhat.com>
Reviewed-by: Joseph Mundackal <jmundackal@bloomberg.net>
2026-07-13 20:12:48 -04:00
Patrick Donnelly
dc779d5f88
Merge PR #70165 into main
* refs/pull/70165/head:
	script/ptl-tool: handle cherry-pick conflicts on branch-specific commits

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2026-07-13 20:11:13 -04:00
Redouane Kachach
32cb979e34
Merge pull request #70154 from rkachach/fix_issue_78179
cephadm: use release/autobuild .asc instead of .gpg for repo_gpgkey

Reviewed-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-07-14 00:22:46 +02:00
Afreen Misbah
0afa737338
Merge pull request #69475 from rhcs-dashboard/nvme-tls-cephadm-signed
Reviewed-by: pujaoshahu <pshahu@redhat.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-14 02:23:19 +05:30
Afreen Misbah
4522b9be68
Merge pull request #70034 from rhcs-dashboard/bucket-validator-fix
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-14 01:58:09 +05:30
Patrick Donnelly
eef55dbb75
script/ptl-tool: handle cherry-pick conflicts on branch-specific commits
Previously, ConflictSimulationCheck only anticipated merge conflicts
when simulating upstream cherry-picks. If a branch-specific commit (such
as test repo branch adjustments or backport metadata changes) failed to
apply cleanly to the target base branch, the script threw an unhandled
GitCommandError and crashed with an unhandled traceback.

Wrap branch-specific commit cherry-picks in a try...except block. When a
conflict occurs, abort the cherry-pick, record a "Rebase Required"
simulation failure in the audit report, and either prompt the user in
interactive mode or gracefully halt check execution in CI mode.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-13 14:35:42 -04:00
Patrick Donnelly
7a0202fa62
Merge PR #70144 into main
* refs/pull/70144/head:
	releng: refactor audit workflow routing and decouple state management

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2026-07-13 14:00:18 -04:00
Redouane Kachach
ad5f741664
cephadm: use release/autobuild .asc instead of .gpg for repo_gpgkey
release.gpg no longer exists at download.ceph.com/keys/ (moved to
/keys/old/, last updated 2020-08-21). Only release.asc is current and
maintained. autobuild.gpg still resolves but is inconsistent with this
and could rot the same way.

repo_gpgkey() now returns the .asc URLs for both release and autobuild
keys. rpm --import and apt accept ASCII-armored keys directly, so no
other code changes are needed.

Fixes 404s seen in qa/workunits/cephadm/test_repos.sh and reported by
users running cephadm add-repo on current distros.

Fixes: https://tracker.ceph.com/issues/78179
Signed-off-by: Redouane Kachach <rkachach@ibm.com>
2026-07-13 19:30:04 +02:00
Patrick Donnelly
6e5a1af9d8
releng: refactor audit workflow routing and decouple state management
Previously, workflow routing and PR state management were split between
the GitHub Actions workflow and the underlying PTL script, leading to
stale branch protection states, API pagination bugs, and blocked merges
during manual overrides.

This refactors the workflow into a centralized state machine and decouples
status reporting from the audit tool:

* Decouple State Management: Remove `--audit-label` from `ptl-tool.py` and
  move all commit status and label transitions (`releng-audit-pass`,
  `releng-audit-fail`) into a dedicated workflow reporter step.
* Non-Blocking Review Comments: Update `ptl-tool.py` in CI mode to post
  audit findings as `COMMENT` events rather than `REQUEST_CHANGES`,
  preventing stale review states from blocking PRs after an override.
* Add Step Summaries: Implement `write_ci_summary()` in `ptl-tool.py` to
  output rich Markdown diagnostic reports directly to `GITHUB_STEP_SUMMARY`.
* Atomic Override Handling: Add `setOverrideStatus()` and `executeOverride()`
  to push an immediate `success` commit status when `/audit override` or the
  override label is applied, unblocking required branch protection checks.
* Safe SHA & Label Lookup: Replace the unpaginated Bash `curl` request with
  a reusable `getPrSha()` helper and evaluate overrides directly within the
  JavaScript router.
* Handle Cancellations Safely: Map `cancelled` workflow outcomes to an
  `error` commit status rather than defaulting to `failure`.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-13 13:17:25 -04:00
Patrick Donnelly
da0228750b
doc,.githubmap: add Aviv to CSC
Aviv was elected to join the CSC in [1].

[1] https://tracker.ceph.com/issues/77573

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-13 11:35:46 -04:00
Redouane Kachach
0156ce1be2
Merge pull request #69847 from yaelazulay-redhat/issues_77248_skip_unpublished_stable_repos_in_test_repos_sh
qa/workunits: skip unpublished stable repos in test_repos.sh

Reviewed-by: Redouane Kachach <rkachach@ibm.com>
2026-07-13 16:54:25 +02:00
Anthony D'Atri
d3ef6ba429 doc/radosgw: Improve s3_objects_dedup.rst
Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
2026-07-13 10:49:37 -04:00
Redouane Kachach
75ee6b497c
Merge pull request #70079 from ShwetaBhosale1/fix_issue_78094_duplicate_entries_for_nfs_haproxy_hosts
mgr/cephadm: deduplicate HAProxy_Hosts in NFS ganesha.conf

Reviewed-by: Redouane Kachach <rkachach@ibm.com>
2026-07-13 16:30:11 +02:00
John Mulligan
690a13b8aa
Merge pull request #69932 from avanthakkar/add-smb-rgw-creds-uri-deps
mgr/cephadm: track SMB rgw_creds_uri as a daemon dependency

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Shweta Sodani <ssodani@redhat.com>
Reviewed-by: Anoop C S <anoopcs@cryptolab.net>
2026-07-13 09:51:20 -04:00
Matan Breizman
913357c627
Merge pull request #69730 from Matan-B/wip-matanb-seastore-metrics-txn
seastore: Record slow transaction histograms

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-13 16:24:49 +03:00
Abhishek Desai
3407863f99 mgr/dashboard : Enable cephadm-signed TLS for NVMeOf
fixes: https://tracker.ceph.com/issues/77406
Signed-off-by: Abhishek Desai <abhishek.desai1@ibm.com>
2026-07-13 18:34:26 +05:30
Patrick Donnelly
958f26b3dc
Merge PR #70135 into main
* refs/pull/70135/head:
	script/ptl-tool: update QA run description to a table
	script/ptl-tool: map supported PR labels to Redmine QA tags
	script/ptl-tool: add --qe-label option for automated QA workflows
	script/ptl-tool: detect scrambled commits and clean up visualizer
	script/ptl-tool: improve ordering of review sections
	script/ptl-tool: cleanup conflict review
	script/ptl-tool: hide outdated bot reviews

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2026-07-13 08:24:29 -04:00
Yael Azulay
a691404841 qa/workunits: skip unpublished stable repos in test_repos.sh
Probe download.ceph.com before each stable add-repo and skip releases
that are not published for the current distro. This avoids apt/yum
failures on newer OS versions (e.g. Ubuntu 24.04/noble) where older
stable trees like quincy and reef were never published.

Fixes: http://tracker.ceph.com/issues/77248
Signed-off-by: Yael Azulay <yazulay@redhat.com>
2026-07-13 15:10:44 +03:00
Sun Yuechi
068b27cac8 osd: guard max_element end() deref in get_health_metrics
slow_op_pools can be empty (all poolids 0/out-of-range) while
slow_op_types is not, making max_element return end(); dereferencing
it is UB.

Fixes: https://tracker.ceph.com/issues/78184
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-13 20:10:42 +08:00
Redouane Kachach
0a924daf55
Merge pull request #63617 from ShwetaBhosale1/fix_nfs_nodeid_issue
mgr/cephadm: During upgrade if NFS cluster is using old nodeids, keep using old node ids

Reviewed-by: Adam King <adking@redhat.com>
2026-07-13 13:27:25 +02:00
Shweta Bhosale
e1d196472d mgr/cephadm: deduplicate HAProxy_Hosts in NFS ganesha.conf
When enable_haproxy_protocol is set, _haproxy_hosts() can collect the
same address more than once. Store collected IPs in a set and return a
list so HAProxy_Hosts contains only unique entries.

Fixes: https://tracker.ceph.com/issues/78094
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-07-13 15:42:13 +05:30
Kefu Chai
55ccaac10e
Merge pull request #69604 from sunyuechi/wip-perfcountertype-unit-init
mgr/MMgrReport: default-initialize PerfCounterType members

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-13 18:07:13 +08:00
Matan Breizman
ae1e4fdcbf crimson/os/seastore: cleanup duplicate DO_TRANSACTION sample
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-07-13 09:57:37 +00:00
Redouane Kachach
06a65d4a05
Merge pull request #69906 from ShwetaBhosale1/fix_issue_77901_nfs_service_deps
mgr/cephadm: only add deps for set NFS spec fields and skip upgrade reconfig for defaults

Reviewed-by: Adam King <adking@redhat.com>
2026-07-13 11:22:57 +02:00
Redouane Kachach
f09d9ba259
Merge pull request #66136 from ShwetaBhosale1/fix_issue_73718_add_option_to_nfs_cluster_create_cmd_to_accept_ingress_spec
mgr/nfs: Add parameter to nfs cluster create command to set ingress placement

Reviewed-by: Ashwin M. Joshi <ashjosh1@in.ibm.com>
2026-07-13 11:22:15 +02:00
Redouane Kachach
2db920a400
Merge pull request #69963 from ShwetaBhosale1/fix_issue_77954_start_disabled_services_while_existing_maintenance_mode
cephadm: start disabled services on host-maintenance exit

Reviewed-by: Redouane Kachach <rkachach@ibm.com>
2026-07-13 11:20:43 +02:00
Redouane Kachach
177897b4f8
Merge pull request #70084 from ShwetaBhosale1/fix_issue_78099_fix_qa_test_nfs.py
qa: fixed the NFS protocol version in test_nfs.py

Reviewed-by: Redouane Kachach <rkachach@ibm.com>
2026-07-13 11:18:48 +02:00
Matty Williams
8c7ccb2d06
Merge pull request #69267 from MattyWilliams22/ec-omaps-enabled
osd: Enable support for omaps in EC pools

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Alex Ainscow <aainscow@uk.ibm.com>
Reviewed-by: Bill Scales <bill_scales@uk.ibm.com>
2026-07-13 10:05:03 +01:00
Venky Shankar
40594bd539 qa:/cephfs: separate out ceph-fuse upgrade tests under fs:upgrade
Fixes: http://tracker.ceph.com/issues/76780
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2026-07-13 12:24:10 +05:30
Venky Shankar
70acbb08ae qa/cephfs: reject ceph-fuse upgrade jobs to reduce failure noise
Subsequent commit will introduce a dedicated ceph-fuse upgrade
sub-suite (under fs:upgrade) to still keep track of failing
jobs.

Fixes: http://tracker.ceph.com/issues/76780
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2026-07-13 12:18:30 +05:30
Sun Yuechi
1dd8cb8bd5 osd/ECBackend: fix iterator invalidation in omap_get
The removed_ranges erase loop in omap_get() called erase(out_it->first),
which invalidates out_it, then did ++out_it on the dead iterator (UB).
When adjacent keys were erased this skipped keys that should have been
removed:

    Expected: { "k01", "k05", "k06" }
    Actual:   { "k01", "k03", "k05", "k06" }

Extract the loop into a static remove_keys_in_ranges() helper so it can
be unit tested without an ObjectStore, and rewrite it as a range erase:
removed_ranges is a sorted map of non-overlapping [start, end) ranges.

Fixes: https://tracker.ceph.com/issues/78177
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-13 14:45:15 +08:00
Shweta Bhosale
12e2928f81 mgr/cephadm: During upgrade if NFS cluster is using old nodeids, keep using it
Fixes: https://tracker.ceph.com/issues/71509
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-07-13 11:12:55 +05:30
nbalacha
a5590d99c3
Merge pull request #69983 from nbalacha/wip-nbala-rgw-s3tests-1
tests/rgw: improvements for the bucket versioning feature in s3-tests
2026-07-13 10:02:54 +05:30
Kefu Chai
e61de15ddb mgr: forward the root-logger fallback at the record's own level
ceph_module.mgr_log() emitted every record at dout(0): a record either
passed the Python-side gate and went unconditionally to both the log
file and the in-memory recent buffer, or never reached the C++ log at
all.  The two-level debug_mgr semantics (file level / memory level)
never applied, so capturing third-party DEBUG logs meant sending each
record to the log file and a log_max_recent slot; with the kubernetes
client logging multi-megabyte HTTP response bodies at DEBUG, the
recent buffer alone could grow to tens of gigabytes.

Pass the record's level through mgr_log() and emit at the mapped ceph
level: CRITICAL/ERROR at 0, WARNING at 1, INFO at 2, DEBUG at 20.
INFO maps to 2 so it stays visible in the log file at the default
debug_mgr=2/5; DEBUG maps to 20 so floods are dropped there, while
debug_mgr=2/20 captures third-party DEBUG in the in-memory buffer
without touching the log file.  The Python-side gate follows the
higher of the two debug_mgr levels so memory-only records still
reach the C++ side.

Fixes: https://tracker.ceph.com/issues/77633
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-13 10:18:21 +08:00
Kefu Chai
3705db8974 mgr: route the root-logger fallback through a module-independent sink
MgrRootHandler captures third-party logs from the Python root logger
and emits them through the _ceph_log method of whichever mgr module
last configured logging.  A handler on the root logger outlives any
single module, so once that module unloads the handler emits through
a stale pointer until another module reconfigures logging.

Add a free mgr_log() function to the ceph_module binding, emitting
via dout() like PyModuleRunner::log but with no PyModule involved,
and rewrite MgrRootHandler to route through it.  The handler no
longer references a module instance, so it cannot go stale.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-13 10:12:04 +08:00
Patrick Donnelly
ef6a63dce1
script/ptl-tool: update QA run description to a table
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-12 20:49:30 -04:00
Patrick Donnelly
0bc6dfe560
script/ptl-tool: map supported PR labels to Redmine QA tags
Introduce a supported whitelist of component tags (e.g., rgw, cephfs,
rbd, cephadm) and automatically extract matching labels from the queued
GitHub pull requests during integration branch builds.

When building or updating a Redmine QA tracking ticket:
* Append matching component tags directly to each PR line in the ticket
  description (e.g., `(tags: rgw, cephfs)`).
* Populate the `tag_list` attribute on the Redmine issue to seamlessly
  integrate with the RedmineUP Tags plugin.
* When updating an existing QA ticket, preserve any pre-existing tags
  already attached to the issue and merge them with the active PR tags.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-12 20:49:30 -04:00
Patrick Donnelly
6f306f79e5
script/ptl-tool: add --qe-label option for automated QA workflows
Introduce the `--qe-label <pr-label>` option to streamline daily integration
testing and Redmine QA tracking for backports.

Using `--qe-label` automatically implies `--integration` and `--pr-label`,
and manages Redmine QA tracking tickets via custom field 45 (Ceph PR Label):

* Searches for open QA tickets matching the specified PR label and target
  release branch.
* If exactly one open ticket is found, prompts the user to either update
  the existing ticket or create a new one.
* If no open ticket is found, automatically defaults to creating a new
  QA ticket with the label stored in its metadata.
* Adds pre-flight validation to ensure all labeled PRs target the exact
  same base branch, preventing ambiguous or mixed-release merges.
* Halts execution if multiple ambiguous open QA tickets exist for the
  same label and release branch.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-12 20:49:30 -04:00
Patrick Donnelly
aaeb264d29
script/ptl-tool: detect scrambled commits and clean up visualizer
Refactor the Commit Parity Visualizer generation to use a deterministic
ordered grouping loop instead of a state machine. Missing commits are now
reliably printed within their respective original PR blocks, and any extra
backport commits are isolated at the bottom of the table.

In addition, add automated checks to detect "scrambled" commits during
backports:
* Verify that commits from multiple upstream PRs are not interleaved or
  scrambled together.
* Verify that commits within an individual upstream PR are applied in
  their exact original chronological sequence.

If scrambled or out-of-order commits are detected, log a failure in CI mode
or prompt the user for interactive resolution to ensure clean backport
history and reduce regression risks.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-12 20:49:30 -04:00
Patrick Donnelly
55b6a0f3eb
script/ptl-tool: improve ordering of review sections
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-12 20:49:30 -04:00
Patrick Donnelly
9c7bb73765
script/ptl-tool: cleanup conflict review
Put it at the end, collapse it, and improve suggestions.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-12 20:49:29 -04:00
Patrick Donnelly
f0f1bc3a1f
script/ptl-tool: hide outdated bot reviews
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-12 20:49:29 -04:00
Kefu Chai
9e656e9655
Merge pull request #69957 from ifed01/wip-ifed-fix-fio-msgr-build
test/fio_ceph_messenger: fix building errors

Reviewed-by: Max Kellermann <max.kellermann@ionos.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-13 08:41:57 +08:00
Patrick Donnelly
c21040049e
Merge PR #70140 into main
* refs/pull/70140/head:
	script/ptl-tool: use img to fix table width

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2026-07-12 19:56:46 -04:00
Hezko
2b2be0b72c
Merge pull request #69861 from Hezko/msg-whem-empty
mgr/dashboard: NVMeoF CLI dont show empty table when no data for read commands
2026-07-13 02:36:50 +03:00
Patrick Donnelly
0a8c75e7cd
Merge PR #70139 into main
* refs/pull/70139/head:
	releng-audit: clear state labels and set pending status on comment triggers
	.github/workflows/releng-audit: cleanup branches

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2026-07-12 14:35:37 -04:00
Ronen Friedman
6314ccfd24
Merge pull request #69482 from fultheim/crimson-omap-extent-retirement-tests
test/crimson/seastore: assert omap extent retirement on split/merge/balance

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2026-07-12 20:56:22 +03:00
Ronen Friedman
522df726eb
Merge pull request #69454 from fultheim/seastore-test-remap-enoent
test/crimson/seastore: tolerate enoent in concurrent remap/overwrite helpers

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2026-07-12 20:55:14 +03:00
Patrick Donnelly
785821e69f
script/ptl-tool: use img to fix table width
Otherwise the browser will shrink the table horizontally to the point
it's unreadable.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-12 13:06:50 -04:00
Patrick Donnelly
01f53cc1a7
releng-audit: clear state labels and set pending status on comment triggers
When /audit retest or /audit test-branch is invoked via an issue_comment event, GitHub Actions does not natively bind the execution to the PR's HEAD commit SHA. As a result, the check widget on the PR conversation tab remains in its previous state (often showing as failed), and any lingering releng-audit-fail or releng-audit-pass labels remain attached.

This introduces a triggerAuditRun helper function to the workflow router that:

- Removes any existing releng-audit-fail or releng-audit-pass labels from the pull request to reset the label state machine and prevent subsequent synchronize events from halting prematurely.
- Explicitly creates a pending commit status on the PR's HEAD SHA via the GitHub REST API, ensuring visual consistency and feedback inside the PR UI while manual or test-branch audits execute in the background.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-12 13:05:57 -04:00
Patrick Donnelly
6509cefb81
.github/workflows/releng-audit: cleanup branches
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-12 13:05:52 -04:00
Afreen Misbah
49ea86d184
Merge pull request #69468 from syedali237/dashboard/hosts-overview
mgr/dashboard: revamped hosts resource page overview 

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-12 22:34:39 +05:30
Ronen Friedman
1116547df1 mon/osdmonitor: streamline enable_pool_ec_optimizations() error returns
using 'expected'

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2026-07-12 16:34:13 +00:00
Ronen Friedman
38da15738c mon: require EC optimizations for Crimson EC pools
Fixes: https://tracker.ceph.com/issues/78124
Co-authored-by: Bill Scales <bill_scales@uk.ibm.com>
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2026-07-12 16:32:58 +00:00
Patrick Donnelly
c326ec3d66
Merge PR #70130 into main
* refs/pull/70130/head:
	.github/workflows/releng-audit: add support for queue label

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2026-07-12 11:46:06 -04:00
SrinivasaBharathKanta
9335813d14
Merge pull request #69777 from sunyuechi/wip-ecomapjournal-bind-by-reference
osd/ECOmapJournal: bind by reference when clearing omap keys
2026-07-12 20:45:52 +05:30
SrinivasaBharathKanta
eed9a5a8f8
Merge pull request #69743 from sunyuechi/wip-osd-scheduler-breaks
osd/scheduler: add missing breaks in PGRecoveryMsg::run
2026-07-12 20:45:17 +05:30
SrinivasaBharathKanta
fbeb1d4e54
Merge pull request #69544 from amathuria/wip-amat-fix-merge-not-ready-warn-check
mon/OSDMonitor: only log PG merge backoff for crimson pools
2026-07-12 20:44:39 +05:30
SrinivasaBharathKanta
03be466bd9
Merge pull request #69746 from sunyuechi/wip-osd-comparison
osd: fix osd_reqid_t comparison strict weak ordering
2026-07-12 20:41:32 +05:30
Syed Ali Ul Hasan
9b1ba0712f mgr/dashboard: revamped hosts overview resource page
- Added a new shared overview component for the resource page

- Added a new host-action service

- Fixes: https://tracker.ceph.com/issues/76712

Signed-off-by: Syed Ali Ul Hasan <syedaliulhasan19@gmail.com>
2026-07-12 19:09:00 +05:30
Patrick Donnelly
5de67b6872
Merge PR #70134 into main
* refs/pull/70134/head:
	.github/workflows/releng-audit: support testing via alternate branch

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2026-07-12 09:14:51 -04:00
Patrick Donnelly
c1b2ffb899
Merge PR #70133 into main
* refs/pull/70133/head:
	script/ptl-tool: add PR label to QA run metadata

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2026-07-12 09:14:12 -04:00
Patrick Donnelly
b7fa210f04
.github/workflows/releng-audit: add support for queue label
This allows trigger a new audit on a mass batch of PRs instead of
individually.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-12 09:13:45 -04:00
Ronen Friedman
f85368ca86
Merge pull request #69853 from ronen-fr/wip-rf-lba1
crimson/os/seastore: documenting the LBA manager

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2026-07-12 09:00:48 +03:00
Ronen Friedman
64ab7813a5
Merge pull request #69966 from ronen-fr/wip-rf-scrubDiag
osd/scrub: fix pg_scrubber.h diagram

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2026-07-12 08:58:20 +03:00
Patrick Donnelly
5708bd5fe0
.github/workflows/releng-audit: support testing via alternate branch
Allow authorized maintainers and release managers to test audit workflow
changes on a pull request using an alternate repository branch.

When a user comments `/audit test-branch [branch-name]` on a PR:
* Verify user authorization and confirm the target branch exists via the
  GitHub API (defaulting to `testing/releng-audit` if unspecified).
* If valid, activate audit execution and set the `checkout_ref` output
  to dynamically override the repository checkout target.
* If the branch does not exist or the user lacks permissions, fail the
  job cleanly and leave an explanatory comment.

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-11 15:20:36 -04:00
Patrick Donnelly
a6ebddf3c5
script/ptl-tool: add PR label to QA run metadata
To easily open all the PRs associated with the QA run (assuming those
PRs are still labeled).

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-11 14:22:27 -04:00
Jesse Williamson
a4a5f15e48 Add support for reverse-order selection.
Updates tests, examples.

Assisted-by: Codex:GPT-5

Signed-off-by: Jesse Williamson <jfw@ibm.com>
2026-07-11 10:04:51 -07:00
John Mulligan
b481de3afa qa/workunits/smb: add tests that use grpc api directly
Add a test case that uses the API directly instead of through the
ceph-smb-ctl command line. Ideally, this is validation of access tot the
api and a precursor to using the grpc api as a component throughout the
smb workunit tests.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2026-07-11 10:38:32 -04:00
John Mulligan
f3a7fad80a qa/workunits/smb: add option to modify share without waiting
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2026-07-11 10:38:31 -04:00
John Mulligan
bb01b31129 qa/workunits/smb: add grpc dependencies to the tests
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2026-07-11 10:38:31 -04:00
John Mulligan
79ed468612 qa/suites/cephadm/smb: add new test case for remote control grpc
We recently added testing for the remote control service via the
local unix socket but had no coverage for the gRPC based
mode of operation. This adds a test case that configures and deploys
the needed TLS certs and sets up and tests remote control with gRPC.

Fixes: https://tracker.ceph.com/issues/76676

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2026-07-11 10:38:31 -04:00
John Mulligan
5e4890efb7 qa/workunits/smb: add test for remote control over gRPC/TCP
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2026-07-11 10:38:31 -04:00
John Mulligan
b77f6fa701 qa/workunits/smb: add a method for getting testdir path
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2026-07-11 10:38:31 -04:00
John Mulligan
e77b9efd2d qa/workunits/smb: add an argument to set up volume mappings
Add an argument to the cephadm shell wrapper function that
makes it easy and obvious where to add volume mappings.
This will be used in a future change to map in a directory
containing TLS certificates and such.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2026-07-11 10:38:31 -04:00
John Mulligan
bddb807fab qa/tasks: pass testdir path to smb workunit
Ensure that the testdir (parent dir of generated tls certs)
is made available to the smb test configuration used by
the workunit tests.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2026-07-11 10:38:31 -04:00
John Mulligan
655bd58510 qa/tasks: add support for embedding ssl/tls certs with templates
Update template.py jinja2 templating, to add support for extracting the
content of tls certs, generated by the ssl_keys task.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2026-07-11 10:38:31 -04:00
kobi ginon
93ff14d091
Merge branch 'main' into issue-77250-clean
Signed-off-by: kobi ginon <153318313+kginonredhat@users.noreply.github.com>
2026-07-11 13:27:02 +03:00
Patrick Donnelly
8f7a85855e
Merge PR #70126 into main
* refs/pull/70126/head:
	.github/workflows/releng-audit: keep failed state on other label events

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2026-07-10 21:00:47 -04:00
Patrick Donnelly
d80ccc8cc1
Merge PR #69464 into main
* refs/pull/69464/head:
	script/ptl-tool: do not update PRs for private QA

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2026-07-10 21:00:13 -04:00
David Galloway
1701c02065
Merge pull request #70127 from djgalloway/wip-78111
script/buildcontainer-setup: fix package install on debian trixie
2026-07-10 18:11:59 -04:00
Patrick Donnelly
e83c663c47
.github/workflows/releng-audit: keep failed state on other label events
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-10 16:40:47 -04:00
Patrick Donnelly
3f721c0760
Merge PR #69149 into main
* refs/pull/69149/head:
	qa/suites/upgrade: ignore fs down variant

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2026-07-10 16:35:01 -04:00
Patrick Donnelly
c70e8df7e1
Merge PR #69463 into main
* refs/pull/69463/head:
	script/ptl-tool: address datetime warning

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-10 16:34:10 -04:00
Patrick Donnelly
3967140782
Merge PR #69335 into main
* refs/pull/69335/head:
	.github/workflows/releng-audit: catch override removal error

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
2026-07-10 16:33:22 -04:00
David Galloway
b5697987ae script/buildcontainer-setup: fix package install on debian trixie
Debian removed software-properties-common from the archive in trixie,
so the flat apt-get install list fails there. The package was only
needed to provide add-apt-repository for llvm.sh, which installs
clang-19 from apt.llvm.org. Trixie ships clang-19 natively, so install
it from the distro instead; run-make.sh's prepare() then finds clang-19
and skips llvm.sh entirely. Ubuntu and older Debian releases still have
software-properties-common and keep the previous behavior.

Fixes: https://tracker.ceph.com/issues/78111
Signed-off-by: David Galloway <david.galloway@ibm.com>
2026-07-10 16:12:01 -04:00
Alex Ainscow
99531101ef
Merge pull request #68878 from aainscow/scrubber_ut
test/osd: Add scrubber to EC / Peering test fixture to check validity.
2026-07-10 20:06:10 +01:00
Yuri Weinstein
69836a3dd1 doc: squid 19.2.5 release notes
Fixed date for release.
Incorporated review comments (trimmed MDS/OSD Notable Changes bullets
per batrick, removed pwlc-epoch bullet per anthonyeleven/idryomov).

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2026-07-10 11:18:36 -07:00
Afreen Misbah
03752e6d6d
Merge pull request #70069 from rhcs-dashboard/htmllint
mgr/dashboard: add more html lint rules and fixer

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-10 22:47:07 +05:30
Patrick Donnelly
2623b29fef
Merge PR #70099 into main
* refs/pull/70099/head:
	.github: fix check-license permission issue

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
2026-07-10 12:58:05 -04:00
Yuri Weinstein
c9ac93fdbb qa/tests: fix POOL_FULL ignorelist pattern in upgrade tests
\(POOL_FULL\) only matches the "Health check failed/update" cluster-log
format (code wrapped in parens). It misses the "Health check cleared"
format, where the code is not parenthesized (see src/mon/Monitor.cc),
so a pool clearing its full state mid-upgrade-test was not ignorelisted.
Drop the escaped parens so the pattern matches both forms, consistent
with other bare entries already in this file (OSD_ROOT_DOWN,
MDS_INSUFFICIENT_STANDBY).

Fixes: https://tracker.ceph.com/issues/78149
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2026-07-10 08:53:36 -07:00
Kefu Chai
1d0df87756
Merge pull request #69776 from tchaikov/wip-mgr-selftest-influx-ignorelist
qa/suites/rados/mgr: ignorelist MGR_INFLUX_DB_LIST_FAILED

Reviewed-by: Sridhar Seshasayee <sridhar.seshasayee@ibm.com>
2026-07-10 23:52:13 +08:00
Anthony D'Atri
b1224d5d6b
Merge pull request #70058 from bluikko/wip-doc-toctree-202607
doc: add orphan files to toctree
2026-07-10 10:40:22 -04:00
Kefu Chai
0ada4fba7f crimson/os/seastore/journal: dispatch records greedily while io slots are free
`RecordSubmitter` dispatches a small record (or the pending batch) only
once the journal is fully drained (`state == IDLE`): `submit()` takes
the direct-write path only at IDLE, and `decrement_io_with_flush()`
flushes the pending batch only after *all* outstanding io has
completed.  The remaining dispatch triggers never fire for 4K-class
records: `preferred_fullness` measures the padding ratio of a single
block, and small records do not reach the batch capacity.

At low-to-mid client queue depth this degenerates into strict
alternation: one journal io in flight, each batch carrying only the
records that arrived during the previous device write.  Measured with
4K randwrite QD20 against a single crimson OSD (circular-bounded
journal): io_depth 1.07, 1.9 records/io, ~2.2ms `submit_record`
latency -- 1.5x the raw device write+flush latency, since each record
first waits ~0.5x for the in-flight io to drain.  `io_depth_limit`
(default 5) is never approached.

Dispatch whenever the device has a free io slot instead:

- `submit()`: take the direct-write path whenever `state != FULL`;
  records are written greedily while slots are free, and batching
  kicks in once `io_depth_limit` is reached.
- `decrement_io_with_flush()`: flush the pending batch into the freed
  slot instead of waiting for IDLE.

Ordering is unaffected: a batch's `write_base` is assigned at dispatch
from `journal_allocator.get_written_to()`, so the on-disk layout
follows submission order; completions are reported in order via the
`device_submission` exit turnstile and the exclusive `finalize` phase,
so a record is never acked before all earlier records are durable.
Replay stops at the first invalid record, and anything beyond such a
hole was never acked.  The same concurrency already occurs through the
`SUBMIT_FULL` path under high load; this extends it to the
low-fullness regime.

## Benchmark

Ceph 21.3.0 dev (RelWithDebInfo), single crimson OSD, 4 reactors
pinned to 4 cores, fio rbd engine on 4 disjoint cores, size-1 pool,
24 GiB prefilled image, consumer NVMe without PLP (~0.7ms fdatasync).

4K randwrite QD20, 180s, `RANDOM_BLOCK_SSD` (circular-bounded journal):

| metric                  | before | after         |
|-------------------------|-------:|--------------:|
| IOPS                    |  5,800 | 10,918 (+88%) |
| avg client latency      |  3.4ms | 1.83ms        |
| avg journal io_depth    |   1.07 | 3.18          |
| records per io          |    1.9 | 1.01          |
| `submit_record` latency |  2.2ms | 1.05ms        |

4K randwrite, 45s per queue depth, `RANDOM_BLOCK_SSD`:

| iodepth |   1 |     4 |    16 |      64 |
|---------|----:|------:|------:|--------:|
| before  | 452 | 1,270 | 3,470 | ~10,300 |
| after   | 536 | 1,908 | 9,337 |  15,333 |

4K randwrite QD20, 180s, `SEGMENTED` (segmented journal and ool
writer, both built on `RecordSubmitter`):

| metric               | before | after         |
|----------------------|-------:|--------------:|
| IOPS                 |  2,728 | 6,123 (+124%) |
| avg client latency   |  7.3ms | 3.26ms        |
| avg journal io_depth |     ~1 | 2.26          |

A classic BlueStore OSD on the same 4-core/QD20 harness does 10,243
IOPS; crimson previously trailed it by 1.8x and now matches it.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-10 22:35:52 +08:00
Avan Thakkar
2063d84c4d mgr/cephadm: track SMB rgw_creds_uri as a daemon dependency
Add rgw_creds_uri as a Dep.FIELD in get_dependencies and include it in
needs_redeploy_prefixes so a change triggers REDEPLOY.

Also fix _save_pending_rgw_config to use cred_map.get() instead
of a bare dict lookup.

Fixes: https://tracker.ceph.com/issues/77763
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2026-07-10 19:58:47 +05:30
Nizamudeen A
9603feb31d mgr/dashboard: fix all ts eslint errors
as per the newer eslint rules. i limited it to only prettifying the code
instead of changing anything functional. running `npm run fix` did the
magic

Fixes: https://tracker.ceph.com/issues/77112
Signed-off-by: Nizamudeen A <nia@redhat.com>
2026-07-10 19:57:11 +05:30
John Mulligan
735dac9f92
Merge pull request #70045 from phlogistonjohn/jjm-smb-doc-port-update
doc/mgr: update out of date paragraph in smb.rst

Reviewed-by: Anoop C S <anoopcs@cryptolab.net>
Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
2026-07-10 09:08:02 -04:00
John Mulligan
980b01206e
Merge pull request #69702 from avanthakkar/fix-smb-sqlite-error-handling
mgr/smb: gracefully handle transient sqlitedb errors

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Rabinarayan Panigrahi <rapanigr@redhat.com>
2026-07-10 09:07:05 -04:00
John Mulligan
1049a49631
Merge pull request #69907 from Sodani/shsodani_ceph_rgw_new_module
mgr/cephadm: add feature-based REDEPLOY logic for SMB services

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Anoop C S <anoopcs@cryptolab.net>
2026-07-10 09:05:06 -04:00
John Mulligan
78c0a6adaa
Merge pull request #69950 from avanthakkar/fix-failed-health-daemon-check
mgr/cephadm: suppress CEPHADM_FAILED_DAEMON for services being removed

Reviewed-by: Rabinarayan Panigrahi <rapanigr@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2026-07-10 09:03:16 -04:00
Redouane Kachach
d313177ee0
Merge pull request #66930 from Shubhaj1810/2368819
mgr/cephadm: support monitoring_addr and bind_addr for nfs

Reviewed-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-07-10 13:22:33 +02:00
Nizamudeen A
daa89763ad mgr/dashboard: fix all prettier warnings
basically ran `npm run fix:html` and commited everything

Fixes: https://tracker.ceph.com/issues/77112
Signed-off-by: Nizamudeen A <nia@redhat.com>
2026-07-10 16:52:18 +05:30
Nizamudeen A
ac187a8220 mgr/dashboard: fix html lint errors according to new rules
mostly accessibility issues like

 - A label component must be associated with a form element
   Fixed by adding `for` to labels and sometimes replacing label with
legend

 - Elements with interaction handlers must be focusable
 - click must be accompanied by either keyup, keydown or keypress event for accessibility
 - <button> should have content

Fixes: https://tracker.ceph.com/issues/77112
Signed-off-by: Nizamudeen A <nia@redhat.com>
2026-07-10 16:20:48 +05:30
Nizamudeen A
8ae8c8177f mgr/dashboard: add more html lint rules and fixer
deprecate the older eslint packages and install the unified official
packages for eslint and migrate .eslintrc to eslint.config.js.

also bump prettier

Fixes: https://tracker.ceph.com/issues/77112
Signed-off-by: Nizamudeen A <nia@redhat.com>
2026-07-10 16:20:27 +05:30
Afreen Misbah
944f1b6b01
Merge pull request #69302 from rhcs-dashboard/77132-Crete-osds-conversion-full-to-wide-tearsheet
mgr/dashboard: Converting Create OSDs tearsheet type from Full to Wide

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-10 14:39:28 +05:30
mohit84
6746f7e97e
Merge pull request #69827 from mohit84/mclock_perf_counter
crimson/osd: add per-class mClock scheduler perf counters
2026-07-10 12:27:16 +05:30
Kefu Chai
ac4171291d crimson/osd: start from first when older maps are trimmed
_handle_osd_map() computes start = superblock.get_newest_map() + 1. A
freshly created OSD has get_newest_map() == 0, so start stays 1. When
such an OSD joins a cluster whose oldest map is already trimmed past
epoch 1, the gap [start, first - 1] is entirely below the trim lower
bound, so neither osdmap_subscribe guard above fires, and start stays 1.

store_maps() then stores only [first..last], but committed_osd_maps()
iterates from start=1 and calls get_local_map(1), which reads the
never-stored osdmap.1, hits ENOENT, and aborts. The OSD crash-loops
without ever activating a PG.

Clamp start to first in that case: the maps below first are gone, and
first is a full map, so it's a valid starting point. Classic OSD does
the same unconditionally via
start = std::max(superblock.get_newest_map() + 1, first).

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-10 14:51:33 +08:00
Afreen Misbah
cf6c7412f2
Merge pull request #69692 from rhcs-dashboard/feature/new-create-role-design
mgr/dashboard: Align RGW Account Role forms with updated UX design

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-10 11:39:59 +05:30
Avan Thakkar
4ae80fb446 mgr/smb: gracefully handle transient sqlitedb errors
Translate sqlite3.DatabaseError into a generic StoreUnavailable in
sqlite_store.py and handle it in cli.py and module.py instead of
letting it surface as an unhandled exception

Fixes: https://tracker.ceph.com/issues/78125
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2026-07-10 11:29:59 +05:30
Kefu Chai
4af78378a9 qa/tasks/mgr: bounce the mgrs once per test class
The dashboard backend API suite occasionally fails in CI on the job's
7200 second timeout alone, with every test passing. Suite runtime
varies with builder load: identical runs of the same 311 tests took
anywhere from about 5400 to over 7000 seconds, and teardown needs
roughly another 180 seconds on top. On the runs that landed at the
high end, only 60 to 80 seconds of margin were left before the
timeout fired. Cutting setup overhead gives the suite that margin
back.

DashboardTestCase.setUpClass() restarted every mgr twice: once in
MgrTestCase.setup_mgrs(), and again right after in _assign_ports(),
whose only job is to set the per-mgr dashboard port, which has to
happen while the mgrs are down.

Let subclasses declare their module ports in a MODULE_PORTS class
attribute instead. setup_mgrs() assigns them while the daemons are
already stopped, so the single restart it performs picks them up and
the second stop/fail/restart cycle disappears.

The assignment loop moves to _assign_module_ports(). _assign_ports()
keeps its old stop/assign/restart contract for callers that reassign
ports outside class setup (test_prometheus, test_dashboard,
test_module_selftest) and delegates to the shared helper.

With 66 dashboard test classes per API suite run this removes one
full mgr bounce per class, roughly 10 to 15 minutes of wall clock in
the CI runs examined.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-10 13:54:32 +08:00
Avan Thakkar
00f61bb64d mgr/cephadm: suppress CEPHADM_FAILED_DAEMON for services being removed
When a service is deleted, a daemon belonging to it may transiently
enter a failed state before orphan cleanup removes it. This happens
when there is a race between dependency-changed REDEPLOY and a
service removal going on paralelly.

Skip daemons in update_failed_daemon_health_check whose service is
already in spec_deleted.

Fixes: https://tracker.ceph.com/issues/77944
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2026-07-10 11:21:56 +05:30
Nizamudeen A
31f32da51a .github: fix check-license permission issue
RCA done by copilot inside the github actions

Root cause: the workflow is passing the default token to a third‑party action that calls PR/diff APIs, and in this run that token doesn’t have the required permission scope for that request.

From the failing job log:

There seems to be an error in an API request
This is usually due to using a GitHub token without the adequate scope
From the job definition (.github/workflows/check-license.yml):

It uses JJ/github-pr-contains-action
Token passed is ${{ github.token }}
Fix
Grant explicit read permissions needed for pull request metadata/content in this workflow (or job), so the token has consistent scope when the action fetches diff data.

Why this solves it
The action needs to read PR diff data. Explicitly setting pull-requests: read (and contents: read) prevents under-scoped default token behavior that triggers the API failure.

Signed-off-by: Nizamudeen A <nia@redhat.com>
2026-07-10 11:12:58 +05:30
Ronen Friedman
1f342d4f27
Merge pull request #69938 from ronen-fr/wip-rf-nodp-crimson
qa/crimson: ignore PG_NOT_DEEP_SCRUBBED in all crimson tests

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2026-07-10 07:46:14 +03:00
Kefu Chai
30849c1196
Merge pull request #70094 from phlogistonjohn/jjm-fix-issue78117
pybind/mgr: disable type checking in diskprediction_local

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-10 10:37:38 +08:00
John Mulligan
228c85a3e3
Merge pull request #69930 from ceph/Add-more-mac-configs
mgr/smb: add fruit:nfs_aces=no for macOS compatibility mode

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Anoop C S <anoopcs@cryptolab.net>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2026-07-09 19:47:33 -04:00
Patrick Donnelly
46cdb8669b
Merge PR #69465 into main
* refs/pull/69465/head:
	script/ptl-tool: source githubmap only when producing credits

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-09 19:46:32 -04:00
John Mulligan
25f76754e0
Merge pull request #70083 from phlogistonjohn/jjm-smb-teuth-workunit-pin
qa/workunits/smb: work around test failures by pinning smbprotocol version

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Anoop C S <anoopcs@cryptolab.net>
2026-07-09 19:46:25 -04:00
Adam Emerson
9f77776598
Merge pull request #68920 from adamemerson/wip-76094
rgw: Fix crashes on realm reload

Reviewed-by: Casey Bodley <cbodley@redhat.com>
Reviewed-by: Matthew N. Heler <matthew.heler@hotmail.com>
Reviewed-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-07-09 16:45:54 -04:00
John Mulligan
fc49390a36 pybind/mgr: disable type checking in diskprediction_local
Disable type checking in the __preprocess function of
diskprediction_local. This function is already collecting a lot of
`type: ignore` comments and adding more is getting difficult due to
the structure of this function. This time we were hitting errors
like:

diskprediction_local/predictor.py:160: error: Value of type "NDArray?[Any]" is not indexable  [index]
diskprediction_local/predictor.py:161: error: "NDArray?[Any]" has no attribute "shape"  [attr-defined]

on python 3.12.

Fixes: https://tracker.ceph.com/issues/78117

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2026-07-09 16:12:11 -04:00
yima77
2097e2f920
Merge pull request #70048 from yima77/cls-update-cmpomap-for-cas-use
rgw/cls: Make cmpomap workable for atomic updates with different values
2026-07-09 14:57:52 -04:00
Casey Bodley
ba6eb32e88
Merge pull request #69841 from cbodley/wip-qa-rgw-crypt-kmip-re-renable
Revert "Reapply "qa/rgw/crypt: disable failing kmip testing""

Reviewed-by: Adam Emerson <aemerson@redhat.com>
2026-07-09 13:21:04 -04:00
VinayBhaskar-V
56bcf706cf pybind/rbd: handle non-existent groups properly in list_snaps()
Calling `list_snaps()` on a non-existent group crashed the entire
Python process with a `free(): invalid pointer` error. This happened
because an unexpected `-ENOENT` error returned by `rbd_group_snap_list2`
and the exception was raised without updating the num_group_snaps to 0
from 10. Later the Cython `__dealloc__` wrapper attempted to free garbage
of uninitialized pointer spaces resulting in a crash.

Fix this behavior by resetting `self.num_group_snaps` to 0 inside
`GroupSnapIterator` before raising exception to prevent memory
corruption and accessing uninitialized pointers during cleanup call

Now `list_snaps()` consistently raise an `ObjectNotFound` error when
invoked on a non-existent group.
Also Extended the `TestGroups` test fixture with `self.dne_group` to
validate the expected error path for `list_snaps()`.

Fixes: https://tracker.ceph.com/issues/78034
Signed-off-by: VinayBhaskar-V <vvarada@redhat.com>
2026-07-09 21:46:21 +05:30
Ville Ojamo
8ed03627fd doc: add orphan files to toctree
Fix three Sphinx warnings and make the documents discoverable.

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>
2026-07-09 23:04:04 +07:00
bluikko
36f3b30643
Merge pull request #70057 from bluikko/wip-doc-radosgw-adminops-ref
doc/radosgw: use ref instead of external link definition
2026-07-09 23:03:06 +07:00
Yixin Jin
b36fc5500b rgw/cls: Make cmpomap workable for atomic updates with different values
1. Make cmp_vals() to take in ObjectOperation to allow
it work with ObjectWriteOperation.

2. Add comments for cmp_set_vals() regarding how different
omap values may be set.

3. Add unit tests to verify the transactional nature of
ObjectWriteOperation used when setting different omap values.

Fixes: https://tracker.ceph.com/issues/76622
Signed-off-by: Yixin Jin <yjin77@yahoo.ca>
2026-07-09 11:13:53 -04:00
Shweta Sodani
3c39960168 mgr/cephadm/smb: Add feature-based REDEPLOY logic for SMB services
Implement intelligent action selection for SMB daemon updates by tracking
feature changes as dependencies. When SMB service features change (e.g.,
adding cephfs-proxy, remote-control, keybridge), the system now triggers
a REDEPLOY instead of RECONFIG, as these changes require container
structure modifications.

Fixes: https://tracker.ceph.com/issues/77762

Signed-off-by: Shweta Sodani <ssodani@redhat.com>
2026-07-09 20:20:47 +05:30
Shweta Bhosale
19dea175a3 qa: fixed the NFS protocol version in test_nfs.py
Fixes: https://tracker.ceph.com/issues/78099
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-07-09 20:02:26 +05:30
John Mulligan
ce5ceabfae qa/workunits/smb: work around test failures by pinning smbprotocol version
Work around new tests failures (in the TestHostsAccessToggle1 tests)
where the error/exception handling has apparently changed by just
pinning the version (by excluding the new v1.17.0 and newer versions).

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2026-07-09 10:28:13 -04:00
Kotresh HR
a06b744e71 tools/cephfs_mirror: Fix failed-to-syncing state priority across metrics paths
When a mirrored directory hits the consecutive failure threshold, the
failed flag stays set until the next successful sync. On retry, the
directory can be actively syncing while failed remains true, but
dump_sync_stat() and update_directory_current_sync_perf_counters()
checked failed before current_syncing_snap and kept reporting "failed".

Introduce get_dir_sync_state() and use it in peer_status, omap
persistence, and per-directory perf counters so syncing takes precedence
over the sticky failed flag.

Fixes: https://tracker.ceph.com/issues/76616
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-07-09 19:44:36 +05:30
Venky Shankar
1257604372 PendingReleaseNotes: add an note about dmclock based scheduler for MDS
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2026-07-09 19:41:34 +05:30
Venky Shankar
11555bcaad doc/config-ref: mention about MDS dmclock configurations
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2026-07-09 19:41:34 +05:30
Venky Shankar
59ea63c7e6 qa: use dmclock with fs:workload
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2026-07-09 19:41:34 +05:30
Yongseok Oh
19b5342824 mds: add MDS dmClock scheduler for subvolume QoS support
Signed-off-by: Yongseok Oh <yongseok.oh@linecorp.com>
Signed-off-by: Venky Shankar <vshankar@redhat.com>
2026-07-09 19:41:34 +05:30
Kefu Chai
5b2aee80a7
Merge pull request #69520 from sunyuechi/warnings-crimson-cmake
crimson/cmake: restrict -Wno-non-virtual-dtor to C++ sources

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-09 21:59:06 +08:00
Redouane Kachach
d899c9fd7d
Merge pull request #69998 from rkachach/fix_issue_77991
qa/cephadm: fix hardcoded 'sshd' unit name in setup_ca_signed_keys

Reviewed-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-07-09 15:44:02 +02:00
Redouane Kachach
368ecc5d05
Merge pull request #68290 from lumir-sliva/cephadm/fix-unknown-state-health-warning
mgr/cephadm: don't treat unknown daemon state as error

Reviewed-by: Redouane Kachach <rkachach@ibm.com>
2026-07-09 15:42:10 +02:00
Redouane Kachach
a46915de23
Merge pull request #69882 from guits/node-proxy-spec-fix
node-proxy: fix NodeProxySpec deserialization from JSON

Reviewed-by: Adam King <adking@redhat.com>
2026-07-09 15:38:47 +02:00
Redouane Kachach
84f94f092c
Merge pull request #69984 from rkachach/fix_issue_77980
mgr/cephadm: pass RGW zonegroup hostnames as custom SANs

Reviewed-by: Ashwin M. Joshi <ashjosh1@in.ibm.com>
2026-07-09 15:34:15 +02:00
Kautilya Tripathi
e06a677f13
Merge pull request #69668 from knrt10/fix-buffer-pos
crimson/osd: guard recovery clone overlap with try_lock_for_read
2026-07-09 18:33:32 +05:30
Ali Masarwa
8b64889285 RGW|global CORS: fix missing headers
Signed-off-by: Ali Masarwa <amasarwa@redhat.com>
2026-07-09 15:46:55 +03:00
Nizamudeen A
0ad33c0965 mgr/dashboard: fix unncessary traceback when bucket not exist
UI has an async validator which calls the GET bucket API to make sure
the bucket name doesn't exist, but the proxy
was not properly handling the http_status_codes which results in raising
a massive traceback in logs whenever you type things in the bucket name
field. So handling that gracefully by capturing the proper status codes
for both RequestException and DashboardException

BEFORE
```
File "/usr/share/ceph/mgr/dashboard/services/exception.py", line 47, in dashboard_exception_handler
return handler(*args, **kwargs)
File "/lib/python3.9/site-packages/cherrypy/_cpdispatch.py", line 54, in _call_
return self.callable(*self.args, **self.kwargs)
File "/usr/share/ceph/mgr/dashboard/controllers/_base_controller.py", line 263, in inner
ret = func(*args, **kwargs)
File "/usr/share/ceph/mgr/dashboard/controllers/_rest_controller.py", line 193, in wrapper
return func(*vpath, **params)
File "/usr/share/ceph/mgr/dashboard/controllers/rgw.py", line 357, in get
result = self.proxy(daemon_name, 'GET', 'bucket', {'bucket': bucket})
File "/usr/share/ceph/mgr/dashboard/controllers/rgw.py", line 213, in proxy
raise DashboardException(e, http_status_code=http_status_code, component='rgw')
dashboard.exceptions.DashboardException: RGW REST API failed request with status code 404
(b'{"Code":"NoSuchBucket","Message":"","RequestId":"tx00000f14e08c1af0d5615-006'
b'71f54a7-3bc6-default","HostId":"3bc6-default-default"}')
2024-10-28T09:08:55.990+0000 7f89e045b640  0 [dashboard INFO request] [::ffff:10.74.18.122:51853] [GET] [500] [0.007s] [admin] [200.0B] /api/rgw/bucket/bucket-das
```

AFTER
```
Jul 08 09:28:28 ceph-node-00 ceph-mgr[2243]: [dashboard ERROR dashboard.rest_client] RGW REST API failed GET req status: 404
Jul 08 09:28:28 ceph-node-00 ceph-mgr[2243]: [dashboard INFO dashboard.services.exception] Dashboard Exception: RGW REST API failed request with status code 404
                                             (b'{"Code":"NoSuchBucket","Message":"","RequestId":"tx00000c029a81e5d154844-006'
                                              b'a4e183c-14251-default","HostId":"14251-default-default"}')
Jul 08 09:28:28 ceph-node-00 ceph-mgr[2243]: [dashboard INFO dashboard.tools] [::ffff:192.168.100.1:60408] [GET] [404] [0.078s] [admin] [200.0B] /api/rgw/bucket/testsa
```

Fixes: https://tracker.ceph.com/issues/78038
Signed-off-by: Nizamudeen A <nia@redhat.com>
2026-07-09 16:15:19 +05:30
Mohit Agrawal
cfb382b82c crimson/osd: add per-class mClock scheduler perf counters
Fixes: https://tracker.ceph.com/issues/77806
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
2026-07-09 16:07:15 +05:30
Adam Kupczyk
72fe925e93 os/bluestore: Simplify _maybe_unshare_on_remove
Unify compressed and uncompressed blobs handling.

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
2026-07-09 09:33:36 +00:00
Adam Kupczyk
96a1a2e25d os/bluestore: Make maybe_unshare_extents use allocations
Blob is tracking whole AUs in SharedBlob tracker.
But the extents may account for just a portion of AUs.
We need to compare what would Blob release against SharedBlob tracker.

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
2026-07-09 09:33:36 +00:00
Adam Kupczyk
1cebcd999d os/bluestore: Modify maybe_unshare_on_remove
Use Blob* as key for processing 'expect' shared blobs.
Opens access to bluestore_blob_t.

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
2026-07-09 09:33:35 +00:00
Adam Kupczyk
e3d24c9827 os/bluestore: Fix unsharing compressed blobs
Unsharing compressed blobs used object offsets, not disk offsets.

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
2026-07-09 09:33:35 +00:00
Adam Kupczyk
50a1037da8 os/bluestore: Split BlueStore::_do_remove()
Split BlueStore::_do_remove into 2 parts:
- part that removes object (in _do_remove)
- part that processes head object and tries to unshare blobs
  (new function _maybe_unshare_on_remove)

No logic changes.

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
2026-07-09 09:33:35 +00:00
Adam Kupczyk
29bae567fc test/objectstore/store_test: Add more tests for unsharing blobs
More test verifying if bluestore is properly revering
shared blobs to regular blobs.

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
2026-07-09 09:33:35 +00:00
Dhairya Parmar
0e9af4b99e tools/cephfs: make journal pointer/header failure explicit
iIf the journal pointer and/or header is missing or corrupt, the tool silently
returns with no-op. There is no dentry recovery happening if any of the objects
are missing but it should be explicit to the user.

This still doesn't change returning errnos on failures to maintain status quo
until https://tracker.ceph.com/issues/77913 is addressed

One important change: The condition to scan events is now changed from
header_present to header_valid. The reason for this can be understood by
reading JournalScanner::scan_header -- if the header could not be decoded, it
is set to null, using this in JournalTool::recover_dentries would segfault and
other corruptions include bad magic or inconsistent offsets. A recovery should
not proceed with distorted magic/offset.

Fixes: https://tracker.ceph.com/issues/76422
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2026-07-09 14:35:21 +05:30
Dhairya Parmar
627af6cfc7 doc/cephfs: add note explainin the new batching technique
Partially-Fixes: https://tracker.ceph.com/issues/76422
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2026-07-09 14:35:20 +05:30
Dhairya Parmar
d611da1c25 tools/cephfs: document --max-rss usage in first-damage.py
Partially-Fixes: https://tracker.ceph.com/issues/76422
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2026-07-09 14:35:20 +05:30
Dhairya Parmar
fc3ca7725b qa: test cephfs-journal-recovery event recover_dentries summary
with capping the maximum RSS to 500 MiB

Partially-Fixes: https://tracker.ceph.com/issues/76422
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2026-07-09 14:35:20 +05:30
Dhairya Parmar
aa2db56d08 tools/cephfs: flush every journal event as soon as it is scanned
NOTE: exclusive to only `event recover_dentries`

JournalScanner::scan() runs a loop and loads the entire decoded journal in an
`EventMap` called `events`. This can severely shoot up the RSS if the journal
is huge (unflushed journal) making the tool less efficient.

The way to curb the memory bomb is to scan and flush journal in batches. The
flush boundary is based on the batch size and at event EVENT_SUBTREEMAP or
EVENT_SUBTREEMAP_TEST(subtree event(s) mark a clear boundary). The knob to
control this setting is via providing `--max-rss <bytes>` with the journal
tool command. This patch divides the provided limit by three to create the
batch since it is observed that the decoded journal events weight on an avg
~3x the encoded stream.

Also: Currently, the events map retains all decoded events until all dentries
are recovered, causing unnecessary memory retention.

Fix this by erasing each event from the map right after it is processed.
While the primary RSS bloat is caused by dnbl holding the encoded stream
in memory, this incremental cleanup provides a minor optimization until
a permanent fix is implemented (https://tracker.ceph.com/issues/77909).

Partially-Fixes: https://tracker.ceph.com/issues/76422

Co-authored-by: Patrick Donnelly <pdonnell@ibm.com>
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2026-07-09 14:35:20 +05:30
Ronen Friedman
d98e84a618 qa/crimson: ignore PG_NOT_DEEP_SCRUBBED in all crimson tests
As we do not yet scrub scheduling in Crimson, the
OSDs do not perform deep scrubs. last_deep_scrub_stamp thus
stays at epoch 0 for all PGs. The monitor flags these as overdue
immediately, causing spurious test failures — particularly in tests
that restart OSDs (e.g. crimson_fio_restart), where PGs reaching
active+clean state trigger the health check.

Fixes: https://tracker.ceph.com/issues/77929
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2026-07-09 08:26:47 +00:00
Ilya Dryomov
bb59fd9b18
Merge pull request #70042 from abitdrag/wip-doc-add-resync-note
doc/rbd: clarify mirror resync snapshot behavior

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2026-07-09 10:16:03 +02:00
Tomer Haskalovitch
586a4e3d6c mgr/dashboard: dont show empty table when no data for read commands
Fixes: https://tracker.ceph.com/issues/77853
Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>
2026-07-09 10:24:04 +03:00
Super User
d80a038a00 doc/rbd: clarify mirror resync snapshot behavior
Add documentation explaining that rbd mirror resync only copies
image state and data up to the last mirror-snapshot for
snapshot-based mirroring. Document the requirement to create
a new mirror-snapshot on current primary so that new changes
(including resize operation) gets reflected on secondary after
resync.

Fixes: https://tracker.ceph.com/issues/78050

Signed-off-by: Miki Patel <miki.patel132@gmail.com>
2026-07-09 12:26:04 +05:30
Ronen Friedman
7071fdcabe
Merge pull request #69942 from ronen-fr/wip-rf-339463backfl
crimson/osd: fix backfill deadlock when only delete operations remain

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Mohit Agrawal <moagrawa@redhat.com>
2026-07-09 09:45:35 +03:00
Sagar Gopale
5f5e4093ad mgr/dashboard: Align RGW Account Role forms with updated UX design
Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-07-09 12:12:32 +05:30
Kefu Chai
392f520d2f
Merge pull request #69614 from dkelson/fix-rdma-fd-eventfd-abort
msg/async/rdma: don't abort on an invalid per-connection eventfd

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-09 14:31:51 +08:00
Nizamudeen A
e56e144187
Merge pull request #69766 from rhcs-dashboard/angular-19.2.x
mgr/dashboard: bump angular to 19.2.25

Reviewed-by: Naman Munet <nmunet@redhat.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-09 11:42:21 +05:30
Shraddha Agrawal
cfe7cfb8a7
Merge pull request #69913 from shraddhaag/wip-shraddhaag-enable-fastec-tests
src/test/librados: enable FastEC tests with crimson
2026-07-09 10:56:16 +05:30
Ville Ojamo
4b6a4a69df doc/radosgw: use ref instead of external link definition
Use a validated intra-docs link using the ref role instead of abusing
external link definitions.

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>
2026-07-09 12:10:40 +07:00
Kefu Chai
1f2dc5fbd6
Merge pull request #69893 from tchaikov/wip-rgw-web-cache
common/web_cache: switch from std::shared_mutex to ceph::shared_mutex

Reviewed-by: Marcel Lauhoff <marcel.lauhoff@clyso.com>
2026-07-09 12:55:30 +08:00
Nizamudeen A
abfa3f8988
Merge pull request #69712 from rhcs-dashboard/table-selection
mgr/dashboard: persist table selection across server side page changes

Reviewed-by: Nizamudeen A <nia@redhat.com>
2026-07-09 10:18:31 +05:30
Kefu Chai
4e545493e8 debian: drop unused deps from ceph-mgr-modules-core.requires
CherryPy and packaging are listed as runtime dependencies of
ceph-mgr-modules-core, but none of the modules shipped in that package
import either one. CherryPy is used by the dashboard, cephadm, and
prometheus modules, which list it in their own .requires; packaging is
not imported by any shipped mgr code. Both entries therefore pull
python3-cherrypy and python3-packaging into ceph-mgr-modules-core for no
reason.

Drop them. The remaining entries (natsort, requests, python-dateutil,
pyyaml) match what the core modules import; prettytable is declared
separately in debian/control.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-09 10:40:25 +08:00
Kautilya Tripathi
8a73bcfecc crimson/osd: guard recovery clone overlap with try_lock_for_read
Crimson recovery could use a neighbor clone as a clone_subset source
during osd_recover_clone_overlap without checking whether that clone
was safe to read. Classic OSD takes a read lock on the neighbor clone
via try_lock_for_read() before using it.

Under snap trim + recovery stress, recovery could clone_range from a
clone that snap trim was modifying, producing wrong object content
(ceph_test_rados: incorrect buffer at pos 1462272).

calc_clone_subsets() and calc_head_subsets() now return clone overlap
candidates in preference order only. ReplicatedRecoveryBackend locks
the first usable neighbor clone from each list and holds those read
locks until push/pull recovery completes. RecoveryCloneLockManager
releases locks in its destructor so move-assign cannot leak a held
lock and block snap trim.

Fixes: https://tracker.ceph.com/issues/76499
Signed-off-by: Kautilya Tripathi <kautilya.tripathi@ibm.com>
2026-07-09 07:41:45 +05:30
Kefu Chai
458deaec58
Merge pull request #70036 from bluikko/wip-doc-remove-rtd-sphinx-search
doc: remove unsupported readthedocs-sphinx-search

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Dan Mick <dan.mick@redhat.com>
Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-09 08:16:31 +08:00
Lumir Sliva
1430b88553
rgw/beast: dispatch connection close on strand during pause
AsioFrontend::pause() (and stop()) call ConnectionList::close() from
the realm reloader thread. That invokes socket.close() on every live
connection from a thread that isn't the connection's asio strand, so
the reactor deregisters each socket's descriptor_state racing against
the coroutine that handles the connection. If a keep-alive connection
is starting its next async_read_some() on its strand at the same time,
epoll_reactor::start_op() dereferences the descriptor_state that close
has already freed and radosgw segfaults in an io_context_pool thread.

This matches the race pattern fixed for timeout_timer in 86ad0b891b:
any caller that touches socket state from outside the strand needs to
be serialized with the coroutine.

Store the coroutine's strand executor on Connection and dispatch the
close onto that strand in ConnectionList::close(). The lambda captures
an intrusive_ptr to Connection so the object lives until the close
actually runs, since connections.clear() unlinks the list entry right
after dispatch.

Tracker: https://tracker.ceph.com/issues/76094
Signed-off-by: Lumir Sliva <61183145+lumir-sliva@users.noreply.github.com>

Update `unittest_rgw_asio_frontend` to run the `io_context` and
execute in a coroutine to match the environment in RGW.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2026-07-08 17:06:08 -04:00
Adam C. Emerson
3ab380f6f8
common/async: io_context_pool no longer moves from init function
Previously if we received an rvalue function object we would call a
moved-from object on all but the first thread. In practice this didn't
cause an issue, but only because of the functions we were passing.

Fixes: https://tracker.ceph.com/issues/76094
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2026-07-08 17:06:08 -04:00
Adam C. Emerson
3a099061cf
rgw: Fix name and add operators for io_context holder
This is essentially a special case of an optional and it should act
like one.

I asked the original author to updated it but they didn't want to, so
I did it myself.

Fixes: https://tracker.ceph.com/issues/76094
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2026-07-08 17:06:08 -04:00
Adam C. Emerson
e8639cc9d4
rgw/rados: Initialize things before starting threads that use them
To prevent a race condition/crash on realm reload where sync was
trying to put objects through quota before quota was initialized.

Fixes: https://tracker.ceph.com/issues/76094
Fixes: https://tracker.ceph.com/issues/77883
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2026-07-08 17:06:08 -04:00
Adam C. Emerson
17f994f1c2
rgw: Check going_down in other-than-interval_set loop
Without this we can end up in an infinite loop where we just fill the
log with -ECANCELED from `get_next()`. Duplicates a check in the
`blocked_operations` loop.

Fixes: https://tracker.ceph.com/issues/76094
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2026-07-08 17:06:08 -04:00
Afreen Misbah
1ecc86bc7c
Merge pull request #70040 from rhcs-dashboard/fix-78051-main
mgr/dashboard: fix RGW - Export realm token form

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-08 22:42:56 +05:30
bluikko
b70badedfd
Merge pull request #70024 from bluikko/wip-doc-sphinx-warnings-202607
doc: fix Sphinx complaints and some rendering errors
2026-07-09 00:03:59 +07:00
John Mulligan
9dff563eac doc/mgr: update out of date paragraph in smb.rst
We still had a part of the document saying we only support the default
port. We added service colocation via custom_ports/bind_addrs a while
back so the doc has been out of date.

Signed-off-by: John Mulligan <jmulligan@redhat.com>
2026-07-08 12:49:11 -04:00
Devika Babrekar
b9e272c469 mgr/dashboard: fixing manual deployment option for osd creation flow
Signed-off-by: Devika Babrekar <devika.babrekar@ibm.com>
2026-07-08 20:11:27 +05:30
Shraddha Agrawal
a4d69dc20c src/test/librados: enable FastEC tests with crimson
This commit enables fast EC tests such that they are run for crimson
OSDs. This is done towards an effort to establish reliable fast EC
support in crimson.

These tests where previously enabled as part of another PR but
ultimately were not merged due to insufficient confidence.

Fixes: https://tracker.ceph.com/issues/77910
Signed-off-by: Shraddha Agrawal <shraddha.agrawal000@gmail.com>
2026-07-08 18:23:14 +05:30
Lumir Sliva
b4363bb25e mgr/cephadm: don't treat unknown daemon state as error
cephadm ls may transiently report state='unknown' for daemons that are
still starting up. The state mapping in _refresh_host_daemons mapped
'unknown' to DaemonDescriptionStatus.error, which caused
update_failed_daemon_health_check to fire CEPHADM_FAILED_DAEMON for
daemons that were simply mid-startup. The warning clears on the next
refresh once the daemon reaches a real state, but it causes spurious
QA failures across many test suites.

Map 'unknown' to DaemonDescriptionStatus.unknown instead — the enum
value already exists and is handled by to_str() and the dashboard.
get_error_daemons() only checks for .error, so unknown daemons no
longer trigger the health warning.

Signed-off-by: Lumir Sliva <61183145+lumir-sliva@users.noreply.github.com>
2026-07-08 14:48:12 +02:00
Shubha Jain
01d5bec5e6
tests: add coverage for virtual_ip bind_addr in nfs
Signed-off-by: Shubha Jain <SHUBHA.JAIN1@ibm.com>
2026-07-08 18:13:15 +05:30
Shubha Jain
f70b3a2934
mgr/cephadm,nfs: support monitoring_addr and bind_addr for NFS
Add support for configuring monitoring_addr and bind_addr
for NFS daemons managed by cephadm.

Also restores required RADOS configuration blocks and
updates tests accordingly.

Fixes: https://tracker.ceph.com/issues/74403
Signed-off-by: Shubha Jain <SHUBHA.JAIN1@ibm.com>
2026-07-08 18:13:14 +05:30
Ville Ojamo
77356f1f78 doc: remove unused readthedocs-sphinx-search
Remove readthedocs-sphinx-search from the RTD build environment because
it seems to be unused and unsupported.

RTD has archived the readthedocs-sphinx-search repo since Apr 2025.
The docs website does not seem to show such function, the search box
in the top-left corner requires a search to be submitted and is not
live.

Fixes: https://tracker.ceph.com/issues/72918
Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>
2026-07-08 19:31:08 +07:00
Shwetha Acharya
d5b730736f mgr/smb: add fruit:nfs_aces=no for macOS compatibility mode
When macOS client compatibility is enabled, add the global Samba
configuration option 'fruit:nfs_aces = no' to disable NFS ACE
handling in the fruit VFS module. This improves compatibility
with macOS SMB clients by preventing conflicts between NFS-style
ACLs and macOS extended attributes.

The configuration is automatically applied when:
- Creating a new cluster with --client-compat=macos
- Updating an existing cluster to macOS mode

Signed-off-by: Shwetha K Acharya <Shwetha.K.Acharya@ibm.com>
2026-07-08 17:48:58 +05:30
Matan Breizman
8e5606a58a crimson/os/seastore: report do_transaction latency in milliseconds
The do_transaction latency histograms were bucketed and summed in
microseconds, but with seastar::lowres_clock (~task_quota, ~0.5ms) and
the ms-scale buckets we keep - microsecond values only add noise to the
exported numbers.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-07-08 11:59:20 +00:00
Matan Breizman
dd9cdd7293 crimson/os/seastore: drop sub-ms do_transaction latency buckets
The do_transaction latency histograms now source their samples from
seastar::lowres_clock (~0.5ms default). Drop the sub 1ms buckets.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-07-08 11:59:20 +00:00
Matan Breizman
784598bd54 crimson/os/seastore: use seastar::lowres_clock for do_transaction timing
do_transaction timing runs on the hot path and used for
latency histograms. std::chrono::steady_clock's per-call clock_gettime
is too costly for these needs. seastar::lowres_clock uses cached now()
at every task_quota poll (~0.5ms) - which is granular enough.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-07-08 11:59:19 +00:00
Matan Breizman
3ba4b80f67 crimson/os/seastore: per-stage do_transaction breakdown for tail txns
on hot shards a small fraction of transactions dominate latency,collecting
one histogram for all is not enough.

Add two more histograms of the same strcutre for txns that take more
than 5ms and more than 10ms.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-07-08 11:58:01 +00:00
Matan Breizman
824e267a33 crimson/os/seastore: measure collection ordering_lock hold time
collock_wait already tells us how long a transaction waits for the
per-collection ordering_lock, but not how long the lock is then held.

comparing hold to wait time can point at either slow in-lock or
contention of a hot pg.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-07-08 11:56:17 +00:00
Matan Breizman
24d9c936f0
Merge pull request #69670 from Matan-B/wip-matanb-seastore-onode-metrics
crimson/os/seastore/onode: measure onode-tree lookup cost

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-07-08 14:53:59 +03:00
Aashish Sharma
95b56b2a43 mgr/dashboard: fix RGW - Export realm token form
When there are multiple realms in rgw multi-site, the realm name/token fields overflow the modal content area

Fixes: https://tracker.ceph.com/issues/78051

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2026-07-08 17:19:31 +05:30
Sun Yuechi
ad6de06803 rbd-mirror: fix strict weak ordering in PeerSpec::operator<
The mon_host tier branched on "mon_host < rhs.mon_host" but returned the
same comparison, so mon_host > rhs.mon_host fell through to key, both
a < b and b < a could hold.

Only Mirror::m_pool_replayers reaches operator< (via std::pair key).

Fixes: https://tracker.ceph.com/issues/78049
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-08 19:40:22 +08:00
Sun Yuechi
b450195246 rbd-mirror: take MirrorStatusUpdater lock by value in queue_update_task
queue_update_task() took the lock by rvalue reference, so its early
returns never released it. In handle_update_task() that left m_lock
held across the on_finish->complete() calls; during shutdown a
completion can re-enter try_remove_mirror_image_status() and re-lock
m_lock, deadlocking.

Take the unique_lock by value so it unlocks on every return path.

Fixes: https://tracker.ceph.com/issues/78047
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-08 19:34:52 +08:00
Sun Yuechi
9759952725 rbd-mirror: return after finishing on unsupported mirror mode
The mirror-mode switch default in handle_get_mirror_info() calls
finish(-EOPNOTSUPP), which deletes this, then breaks and falls
through to the common code, dereferencing the null *m_state_builder and
freeing the request a second time.

Fixes: https://tracker.ceph.com/issues/78043
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-08 18:57:29 +08:00
Afreen Misbah
9e8d7ef611
Merge pull request #69839 from rhcs-dashboard/fix-duplicate-acc-name
mgr/dashboard: add account name duplicate check 

Reviewed-by: Nizamudeen A <nia@redhat.com>
2026-07-08 16:25:03 +05:30
Adam Kupczyk
c08f25ee53
Merge pull request #69540 from Jayaprakash-ibm/wip-bluestore-allocator-perf
os/bluestore: add new allocator perf counters for  allocator lock wait and processing latency
2026-07-08 11:37:30 +02:00
Aashish Sharma
7c55b80669
Merge pull request #69470 from rhcs-dashboard/fix-77400-main
mgr/dashboard: fix version string overlap, incomplete version on upgrades page

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-08 14:10:10 +05:30
Matan Breizman
27060dccdf
Merge pull request #69331 from Matan-B/wip-matanb-cyanstore-fix
crimson/osd: fix CyanStore boot with crimson OSD

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-08 10:19:34 +03:00
Ville Ojamo
d0dc93a22e doc: fix Sphinx complaints and some rendering errors
Fix 25 criticals, errors, and warnings from Sphinx.
Some of the changes did not change rendering but most fixed unintended
rendering, broken link etc.
Change Mon to Monitor in ceph-secrets.rst.

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>
2026-07-08 12:54:16 +07:00
Kefu Chai
83f218a909
Merge pull request #69783 from sunyuechi/openruyi-spec
ceph.spec.in: add support for openRuyi

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-08 13:45:08 +08:00
Nizamudeen A
fcbbca8949 mgr/dashboard: teardown http requests for feature-toggle
angular 19.2.latest is aggressively tearing down the test env which
forces the active rxjs timer to emit one execution as it collapses which
produces a ghost http in the case of request wrapped in the timer
service. so for now flushing all the requests manually to prevent it

Fixes: https://tracker.ceph.com/issues/77729
Signed-off-by: Nizamudeen A <nia@redhat.com>
2026-07-08 10:51:50 +05:30
Afreen Misbah
84881e99f0
Merge pull request #69917 from rhcs-dashboard/edit-host-access
mgr/dashboard: NVMe-oF – Enable editing host access from the Subsystem Overview page

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-08 10:17:38 +05:30
Nizamudeen A
13383ddfa2 mgr/dashboard: bump angular to 19.2.25
Fixes: https://tracker.ceph.com/issues/77729
Signed-off-by: Nizamudeen A <nia@redhat.com>
2026-07-08 10:11:34 +05:30
Laura Flores
387871c86d
Merge pull request #68325 from ljflores/wip-tracker-75414
qa/suites/upgrade: add POOL_FULL variations to ignorelist

Reviewed-by: Radosław Zarzyński <Radoslaw.Adam.Zarzynski@ibm.com>
2026-07-07 14:57:02 -05:00
Ilya Dryomov
d91fba350b
Merge pull request #69988 from idryomov/wip-77982
qa/suites/rbd/valgrind: pin to centos_9.stream instead of rpm_latest

Reviewed-by: Ramana Raja <rraja@redhat.com>
2026-07-07 21:16:04 +02:00
Kefu Chai
d5e9cf5e52
Merge pull request #69949 from sunyuechi/fix-subshellkilled-typo
test/subprocess: add missing space in SubshellKilled subcommand

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-07 21:51:52 +08:00
Redouane Kachach
7b2217caf3
qa/cephadm: fix hardcoded 'sshd' unit name in setup_ca_signed_keys
setup_ca_signed_keys() hardcoded `systemctl restart sshd`, which fails
on Debian/Ubuntu where the SSH server unit is named `ssh.service`
rather than `sshd.service`. Try `ssh` first, fall back to `sshd` if
that fails, grouped so the fallback only fires on restart failure and
doesn't mask a failed config write.

Fixes: https://tracker.ceph.com/issues/77991
Signed-off-by: Redouane Kachach <rkachach@ibm.com>
2026-07-07 15:47:11 +02:00
Samarah Uriarte
d7f2fba919
Merge pull request #69889 from samarahu/standalone-lifecycle-sqlite
rgw/posix: Implement SQLite lifecycle in POSIXUserDB

Reviewed-by: Daniel Gryniewicz <dang1@ibm.com>
2026-07-07 08:30:31 -05:00
Oguzhan Ozmen
5addb80aba rgw: copy_obj_data() record uncompressed size for a compressed source
When copying an object and a compressor/encryption processor is configured
for the destination, the bucket-index accounted_size was set from `ofs`. If
the source is compressed but the destination is not (re-)compressed (e.g.
compression is off, or rgw_compression_type=random rolled no-compression),
then `ofs` is the compressed size, not the logical size.

In multisite this diverges from the peer zone, which recomputes its own
accounted_size via fetch_remote_obj() (already corrected in #63306).
ListObjectVersions then reports different sizes per zone.

Pass along the source's logical size (astate->accounted_size, derived from the
object's own attrs and correct for plain/compressed/encrypted) into
copy_obj_data() and use it when the destination is not compressed. The
rewrite_obj() and transition_obj() callers pass 0 and keep the prior
ofs-based behaviour. This is the copy_obj analog of the fetch_remote_obj
fix in #63306.

No on-disk/encoding change; corrects newly copied objects only (existing
index entries are unaffected) and is safe across a mixed-version upgrade.

Fixes: https://tracker.ceph.com/issues/75716
Fixes: https://tracker.ceph.com/issues/75715
Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-07-07 13:00:17 +00:00
Afreen Misbah
b35999b643 mgr/dashboard: fix cancel button
Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-07 18:28:32 +05:30
Ilya Dryomov
75c4e2897f qa/suites/rbd/valgrind: pin to centos_9.stream instead of rpm_latest
This used to be the case before commit d4b977afdc ("qa/distros:
rename centos_latest.yaml to rpm_latest.yaml") and subsequent changes
to enable Rocky 10.  When paired with valgrind, python_api_tests* jobs
fail persistently on Ubuntu and Rocky, see [1].  Switch back until that
is resolved (or for as long as centos_9.stream remains in the mix).

[1] https://tracker.ceph.com/issues/74864

Fixes: https://tracker.ceph.com/issues/77982
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2026-07-07 13:40:57 +02:00
Afreen Misbah
c6dafdc553
Merge pull request #69693 from rhcs-dashboard/fix-77596-create
mgr/dashboard: Implement backword support for Create existing nvme-oF gateway groups through UI

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-07-07 17:03:22 +05:30
Matan Breizman
1cbf39b594 crimson/osd: fix CyanStore boot with crimson OSD
- Remove dead cyan_store.h includes from pg.cc and shard_services.cc.
- Fix CyanStore::mkfs to write type="cyanstore" instead of "memstore"
- Handle missing device class in OSD::start: stores that return an empty
  device class (e.g CyanStore) now fall back to is_rotational=false with
  a warning instead of throwing

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-07-07 10:48:44 +00:00
Matan Breizman
9899e7bfeb crimson/os/seastore/onode: measure onode-tree lookup cost
Before optimizing the onode lookup paths we should measure the existing
cost if each possible optimization.

The per-node search is already binary search(see search_result_bs_t
binary_search). Possible costs are:
- per-level node traversal
- STAGE_STRING ns/oid memcmp (rbd has long shared oid prefixes)

Adding the following stats to be exposed as seastar-metrics (not logs).

Keep the new sampling per-comparison overhead out unless needed for
measurment runs.

Candidate optimization leads from the numbers:
* cross-node string/prefix dedup (marked as TODOs in stage.h) this could
  result in cheaper compares for shared (rbd) prefixes.
* shrink onode_layout_t - marked as onode.h TODO.
  stop inlining oi and ss to get more objects per leaf.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-07-07 10:40:32 +00:00
Nithya Balachandran
e6d3e23b36 tests/rgw: verify current versions for bucket versioning
Verify that the current version is updated correctly with the
bucket versioning feature.

Fixes: https://tracker.ceph.com/issues/77981

Signed-off-by: Nithya Balachandran <nithya.balachandran@ibm.com>
2026-07-07 10:38:46 +00:00
Redouane Kachach
3ceee3ad3f
mgr/cephadm: pass RGW zonegroup hostnames as custom SANs
Fix RGW certificate generation so `zonegroup_hostnames` and wildcard
SANs are passed to `get_certificates()` using the correct keyword
argument. Previously, RGW built the expected SAN list from
`zonegroup_hostnames` and `wildcard_enabled`, but passed it
positionally so it was interpreted as ips list.

Also add a unit test to cover the RGW `zonegroup_hostnames` +
`wildcard_enabled` path and ensure SANs are passed as `custom_sans`.

Fixed other calls to enforce the usage of keywords for arguments just
to avoid similar issues in the future.

Fixes: https://tracker.ceph.com/issues/77980
Signed-off-by: Redouane Kachach <rkachach@ibm.com>
2026-07-07 12:37:53 +02:00
Nithya Balachandran
33f5de6f61 tests/rgw: mark versioning tests in s3-tests
Mark the versioning tests in RGW s3-tests.

Fixes: https://tracker.ceph.com/issues/77981

Signed-off-by: Nithya Balachandran <nithya.balachandran@ibm.com>
2026-07-07 10:27:08 +00:00
Aashish Sharma
045ec191a8 mgr/dashboard: fix version string overlap, incomplete version on upgrades page
Fixes: https://tracker.ceph.com/issues/77400

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2026-07-07 14:32:34 +05:30
Kotresh HR
f2ee574aab tools/cephfs-mirror: Fix truncated sync from stale crawl-time file size
Drop AT_STATX_DONT_SYNC from stat calls in the crawler thread that
populate SyncEntry objects. The flag could leave a stale file size from
readdir, causing datasync to truncate files to the wrong length.

For RemoteSync, use inode attrs from ceph_readdirplus_r instead
of a redundant ceph_statxat.

Fixes: https://tracker.ceph.com/issues/77393
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-07-07 14:14:53 +05:30
Kefu Chai
6c89866b53
Merge pull request #69865 from sunyuechi/wip-osd-asok-route-einval
osd: report -EINVAL to on_finish in asok_route_to_pg catch

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-07 16:27:25 +08:00
Naman Munet
6bf1347dbc mgr/dashboard: Stale RGW user/account metadata cache in mgr after delete and recreate
fixes: https://tracker.ceph.com/issues/77544

Signed-off-by: Naman Munet <naman.munet@ibm.com>
2026-07-07 11:25:18 +05:30
anrao19
dd7b7812d3
Merge pull request #69117 from linuxbox2/wip-tracker-76924
rgwlc:  faithfully store lifecycle filters
2026-07-07 11:09:05 +05:30
anrao19
dd126cee8c
Merge pull request #69086 from harsimran-05/AddingHeadObjectMetricstoProm
rgw: add HEAD Object op metrics to perf counters and Prometheus export
2026-07-07 11:08:48 +05:30
Kautilya Tripathi
7b214cba9a crimson/osd: discard stale primary backfill scan after backfill finishes
do_request_primary_scan() is an async coroutine.  Backfill can complete
(and on_pg_clean() can destroy backfill_state) while the coroutine is
suspended in scan_for_backfill_primary(), e.g. waiting on an object
context load.  Resuming and delivering PrimaryScanned then dereferences
a null backfill_state.

Discard the scan result if backfill_state is gone; the backfill has
already finished successfully.

Fixes: https://tracker.ceph.com/issues/77623
Signed-off-by: Kautilya Tripathi <kautilya.tripathi@ibm.com>
2026-07-07 09:35:40 +05:30
Venky Shankar
d73bf27fa9
Merge pull request #68635 from chrisphoffman/wip-74110
qa: Use mount class in test_fscrypt to read/write and to compare trees

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2026-07-07 08:41:40 +05:30
Vitaly Goot
91d7c74685 crimson/osd: fix boot crash when a peer OSD was purged across an osdmap batch
OSD::committed_osd_maps() captures old_map, then for each peer in
old_map->get_all_osds() that transitioned up->down it marks the peer's
cluster address down. The address was fetched from the *new* osdmap via
osdmap->get_cluster_addrs(osd_id), but osd_id comes from old_map. If that
peer was purged (fully removed) in the new map, get_cluster_addrs hits
ceph_assert(exists()) and the OSD aborts on boot.

This makes it impossible to decommission a crimson cluster: purging any
OSD leaves surviving crimson OSDs unable to reboot once they replay an
osdmap epoch range spanning that purge, since they abort on this assert
during boot.

Look the address up in old_map instead: it is guaranteed to contain
osd_id (that is where the id came from) and holds its last-known cluster
address, which is exactly what we want to mark down.

Signed-off-by: Vitaly Goot <vgoot@akamai.com>
2026-07-06 20:05:59 +00:00
Jaya Prakash
70fdcaae86 blk/kernel : skip AIO thread for devices with size < block_size
Fixes: https://tracker.ceph.com/issues/76354

Signed-off-by: Jaya Prakash <jayaprakash@ibm.com>
2026-07-07 00:45:19 +05:30
Ronen Friedman
d519c5f8e5 osd/scrub: fix pg_scrubber.h diagram
The diagram depicts the relationships between the main
Scrubber components and interfaces. The previous version of
the diagram was outdated, and it is now fixed to match the code.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2026-07-06 19:19:25 +03:00
Daniel Gryniewicz
5f01d5c058 RGW - Standalone - Add a standalone build target and RPM
Add a build target that builds standalone RGW in addition to normal RGW,
and add an RPM for it that only installs the necessary bits.

Signed-off-by: Daniel Gryniewicz <dang@fprintf.net>
2026-07-06 11:19:42 -04:00
Daniel Gryniewicz
9a0e3c70c1 Fix tools/tests link with bluestore disabled
Signed-off-by: Daniel Gryniewicz <dang@fprintf.net>
2026-07-06 11:19:42 -04:00
pujashahu
f6d0164a05 mgr/dashboard: Implement backword support for Create existing nvme-oF gateway groups through UI
Fixes : https://tracker.ceph.com/issues/77632

Signed-off-by: pujaoshahu <pshahu@redhat.com>
2026-07-06 20:29:39 +05:30
Abhishek Desai
4ac6a8298f mgr/dashboard : Enable cephadm-signed TLS for NVMeOf
fixes: https://tracker.ceph.com/issues/77406
Signed-off-by: Abhishek Desai <abhishek.desai1@ibm.com>

(cherry picked from commit 14162473bb6a6993ec729056b35dc514cad2228e)
Signed-off-by: pujashahu <pshahu@redhat.com>
2026-07-06 20:20:39 +05:30
Sun Yuechi
2c736cf70e crimson/osd/pg_recovery: fix exception handler scope in recover_missing
Member access binds tighter than ?:, so the handler attached only to the
recover_object() branch and exceptions from recover_object_with_throttle()
escaped unhandled. Parenthesize the ternary to cover both branches.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-06 22:44:35 +08:00
Shweta Bhosale
e9b85dfa74 mgr/nfs: Add parameter to nfs cluster create to pass ingress placement
Fixes: https://tracker.ceph.com/issues/73718

Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-07-06 19:51:25 +05:30
Shweta Bhosale
a0b7c9447d cephadm: start disabled services on host-maintenance exit
NFS and keepalived are deployed without systemd enable (DISABLED_SERVICES).
When a host exits maintenance, re-enabling ceph-<fsid>.target only starts
enabled units, so these daemons remain stopped.

Fixes: https://tracker.ceph.com/issues/77954
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-07-06 19:40:20 +05:30
Sridhar Seshasayee
5937363e43
Merge pull request #69578 from sseshasa/wip-osd-perf-counter-pg-rebuild-stats
osd/PeeringState: add perf counters for PG rebuild times

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2026-07-06 18:33:52 +05:30
Afreen Misbah
e3a65dcfd4
Merge pull request #69217 from rhcs-dashboard/wip-fix-user-creation-validation
mgr/dashboard: fix-user-creation-validation

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Pedro Gonzalez Gomez <pegonzal@redhat.com>
2026-07-06 17:59:32 +05:30
Igor Fedotov
410238719a test/fio_ceph_messenger: fix building errors
"
home/if/ceph.2/src/test/fio/fio_ceph_messenger.cc: In function ‘void
put_ceph_context()’:
/home/if/ceph.2/src/test/fio/fio_ceph_messenger.cc:144:9: error:
expected primary-expression before ‘(’ token
  144 |     dout(0) <<  ostr.str() << dendl;
      |         ^
In file included from /home/if/ceph.2/src/include/Context.h:20,
                 from /home/if/ceph.2/src/msg/Message.h:32,
                 from /home/if/ceph.2/src/msg/Messenger.h:29,
                 from
/home/if/ceph.2/src/test/fio/fio_ceph_messenger.cc:12:
/home/if/ceph.2/src/common/dout.h:190:5: error: ‘_dout_cct’ was not
declared in this scope
  190 |     _dout_cct->_log->submit_entry(std::move(_dout_e));
      \
      |     ^~~~~~~~~
/home/if/ceph.2/src/common/dout.h:216:15: note: in expansion of macro
‘dendl_impl’
  216 | #define dendl dendl_impl
....
"

Fixes: https://tracker.ceph.com/issues/77953
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
2026-07-06 14:53:31 +03:00
Shweta Bhosale
8855c439b4 mgr/cephadm: only add deps for set NFS spec fields, skip upgrade reconfig for defaults
Only add deps when the user has set a non-default value.  In
choose_next_action(), ignore False/None-valued entries in the symmetric
diff so daemons upgrading from the old behavior are not reconfigured
solely because default deps disappeared

Fixes: https://tracker.ceph.com/issues/77901
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-07-06 17:14:49 +05:30
Redouane Kachach
8d6d1a1f5c
Merge pull request #66037 from ShwetaBhosale1/fix_issue_73442_nfs_getting_restarted_twice_in_case_of_reboot
mgr/cephadm: Stop NFS service/daemon from starting automatically after reboot, cephadm to manage startup

Reviewed-by: Adam King <adking@redhat.com>
2026-07-06 12:31:08 +02:00
Redouane Kachach
9d5142ea08
Merge pull request #67025 from ShwetaBhosale1/fix_issue_74492_set_nfs_default_protocol_as_4
mgr/cephadm: Updated NFS default protocol to v4 and v3 is enabled only when enable_nfsv3 is set in the spec

Reviewed-by: Redouane Kachach <rkachach@ibm.com>
Reviewed-by: Adam King <adking@redhat.com>
2026-07-06 12:30:37 +02:00
Redouane Kachach
33f5816726
Merge pull request #68890 from ShwetaBhosale1/fix_76579_nfs_precheck_for_placement_count_per_host
python-common: Adding precheck for placement count_per_host in NFSServiceSpec

Reviewed-by: Adam King <adking@redhat.com>
2026-07-06 12:29:35 +02:00
Redouane Kachach
e3b03eba03
Merge pull request #69551 from VallariAg/fix-nvmeofclient-daemon_name
mgr/dashboard: Fix daemon_name for NVMeoFClient

Reviewed-by: Redouane Kachach <rkachach@ibm.com>
Reviewed-by: Kushal Deb <Kushal.Deb@ibm.com>
2026-07-06 12:28:27 +02:00
Redouane Kachach
881185d52c
Merge pull request #69541 from ShwetaBhosale1/fix_issue_77463_fix_nfs_export_remove
mgr/nfs: read full conf object when removing export URL

Reviewed-by: Adam King <adking@redhat.com>
2026-07-06 12:27:28 +02:00
Redouane Kachach
ae0cfd0d43
Merge pull request #69849 from adk3798/cephadm-delete-config-deps
mgr/cephadm: clear daemon_config_deps entry for removed daemons

Reviewed-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-07-06 12:26:49 +02:00
Redouane Kachach
d5922477ba
Merge pull request #69899 from ShwetaBhosale1/fix_issue_77891_delete_config_keys_set_by_service_spec_config_field
mgr/cephadm: remove spec.config options when a service is removed

Reviewed-by: Adam King <adking@redhat.com>
2026-07-06 12:26:12 +02:00
pujashahu
6573766548 mgr/dashboard: NVMe-oF – Enable editing host access from the Subsystem Overview page
Fixes: https://tracker.ceph.com/issues/77912

Signed-off-by: pujaoshahu <pshahu@redhat.com>
2026-07-06 13:51:55 +05:30
Sun Yuechi
6aa58ca1ec test/subprocess: add missing space in SubshellKilled subcommand
9abf2388f2 rewrote "sh -c cat" as SHELL "-c cat", which concatenates
to "/bin/sh-c cat", so the subshell tries to execute a non-existent
"/bin/sh-c". The test still passed because the SIGTERM sent by kill()
usually wins the race against the shell startup; under CI load the
shell occasionally prints

  /bin/sh: line 1: /bin/sh-c: No such file or directory

into the stderr pipe first, ASSERT_TRUE(buf.empty()) fails, and the
skipped join() turns this into a SIGABRT in ~SubProcess().

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-06 01:06:09 -07:00
Aashish Sharma
0a3d6e3e34
Merge pull request #69282 from rhcs-dashboard/fix-77115-main
mgr/dashboard: rbd-mirroring - hide create/import token buttons for read-only users

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-06 11:51:57 +05:30
Naman Munet
8dafeeb22e mgr/dashboard: add account name duplicate check
fixes: https://tracker.ceph.com/issues/77828

Signed-off-by: Naman Munet <naman.munet@ibm.com>
2026-07-06 10:56:11 +05:30
Sun Yuechi
4598137a81 ceph.spec.in: add support for openRuyi
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-06 01:22:53 +08:00
Sun Yuechi
274fa2e27c ceph.spec.in: switch more build deps to pkgconfig() and python3dist()
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-06 01:22:53 +08:00
Kefu Chai
5bc195949a
Merge pull request #69637 from xxhdx1985126/fix-balancer-upmap-underfull-scan
osd: speed up upmap underfull fallback

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2026-07-05 21:31:38 +08:00
Ronen Friedman
a5807c0878 rgw/bucket_logging: fix dangling string_view from ternary temporary
Clearing a compilation warning:

The ternary operator created a temporary std::string to unify
const std::string& and const char* branches, leaving the
string_view pointing to a destroyed object. Replace with
find() + if/else to assign directly from stable storage.

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2026-07-05 12:47:27 +00:00
Ronen Friedman
f64896d838 crimson/osd: fix backfill deadlock when only drop operations remain
BackfillState::Waiting requires an ObjectPushed event to transition
back to Enqueuing, but drop (remove) operations are fire-and-forget
and never generate ObjectPushed. When a backfill round enqueues only
drops and no pushes, the FSM permanently stalls in the Waiting state,
holding the backfill reservation and blocking all other PGs from
backfilling to that OSD.

Fix by adding a pending-pushes counter to ProgressTracker to
distinguish "only drops remain" from "pushes still in-flight". When
all pushes are complete and the namespace is exhausted, drain remaining
drop entries and advance last_backfill to MAX so that
OP_BACKFILL_FINISH triggers the normal completion path via
RequestDone.

The classic OSD avoids this by design: drops are never tracked as
in-flight operations in recover_backfill() and are drained inline.

Fixes: https://tracker.ceph.com/issues/77936
Assisted-by: Claude <claude.ai>
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2026-07-05 12:33:02 +00:00
Ronen Friedman
5be3fa605f
Merge pull request #69905 from ronen-fr/wip-rf-328786-emptyflush
crimson/osd: skip store flush for empty peering transactions

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
Reviewed-by: Jose J Palacios-Perez <perezjos@uk.ibm.com>
2026-07-05 15:28:41 +03:00
Ali Masarwa
73d7c9cc9b
Merge pull request #69638 from AliMasarweh/wip-nbalacha-posix-versioned
rgw/posix: fix delete markers

reviewed-by: Daniel Gryniewicz <dang@redhat.com>
2026-07-05 14:54:26 +03:00
Sun Yuechi
2d85d9e158 tools/immutable_object_cache: don't leak in-flight replies on teardown
CacheSession::send() deleted the reply in the async_write completion
handler. On teardown asio drops the pending handler without running it,
leaking the reply. Delete it right after encoding into bl instead; bl
owns the payload, so the write is unaffected.

Fixes: https://tracker.ceph.com/issues/77552
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-05 04:34:45 -07:00
NitzanMordhai
30b846988e
Merge pull request #69349 from NitzanMordhai/wip-nitzan-mgr-dispatch-throttle-increase
mgr: fix dispatch throttle bottleneck causing OSD connection timeouts
2026-07-05 13:14:25 +03:00
Kefu Chai
6503cfbbba
Merge pull request #69353 from batrick/librados-nspace-dout
librados: print nspace only if present

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-05 17:43:35 +08:00
Kefu Chai
1299bc382b
Merge pull request #69819 from sunyuechi/wip-blk-spdk-env-opts-init
blk/spdk: call spdk_env_opts_init() before setting pci options

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2026-07-05 17:42:58 +08:00
Kefu Chai
bc6ebfff79
Merge pull request #69053 from tchaikov/wip-seastore-overwrite-update-checksum
crimson/seastore: skip LBA-leaf crc check for in-place delta-overwrite

Reviewed-by: Myoungwon Oh <ohmyoungwon@gmail.com>
2026-07-05 08:41:27 +08:00
Kefu Chai
80b9067286 blk,build: drop BlueStore PMEM support
BlueStore's PMEMDevice backend targets byte-addressable persistent
memory exposed as a DAX device, in practice Intel Optane DC persistent
memory. Intel Optane has been discontinued since 2021 and there is no
mainstream replacement, so the backend has effectively no hardware left
to run on. It was always experimental and has seen no functional change
since the DML/DSA offload landed in 2022; every commit to it since has
been a formatting sweep.

Keeping it builds in a chain of dependencies that nothing else needs:
libpmem, libndctl and libdaxctl, plus dml for the DSA offload that only
works on a Sapphire Rapids CPU. Remove the backend and everything tied
to it.

RBD persistent write-back cache (WITH_RBD_RWL) still uses PMDK's
libpmemobj, so the pmdk submodule, WITH_SYSTEM_PMDK, Buildpmdk and
Findpmdk stay. WITH_SYSTEM_PMDK now depends on WITH_RBD_RWL alone, and
Buildpmdk no longer builds ndctl support.

- remove the WITH_BLUESTORE_PMEM option, HAVE_BLUESTORE_PMEM and
  HAVE_LIBDML
- delete src/blk/pmem/PMEMDevice.{cc,h} and the pmem block_device_t
- drop the pmem value from the bdev_type option
- remove the --bluestore-pmem vstart flag
- drop the libpmem, libndctl and libdaxctl build deps from debian and
  rpm, keeping libpmemobj for RWL
- delete the Finddml, Findndctl and Finddaxctl cmake modules
- remove the DSA/DML and ndctl documentation

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-05 08:37:21 +08:00
Anthony D'Atri
a73267fe1c
Merge pull request #69793 from anthonyeleven/caps
doc: Improve src/common/options/mds-client.yaml.in
2026-07-04 15:29:58 -04:00
Ronen Friedman
febf0c6263 crimson/mgr: prevent pg stats report pile-up in overloaded store
When AlienStore's thread pool is saturated (e.g. during PG splitting),
pool_statfs() calls inside OSD::get_stats() can get stuck behind
hundreds of queued transactions in the free_slots semaphore. Because
Client::report() dispatched get_stats() unconditionally every 5 seconds,
hundreds of stuck pool_statfs() calls accumulated, further exhausting
the semaphore and making recovery impossible.

Fix by tracking whether a get_stats() call is already in flight and
skipping new dispatches until the current one completes. Emit a
cluster-log warning every ~60 seconds when stats reporting is stuck.

This limits the damage from thread pool starvation to a single consumed
free_slot instead of an unbounded pile-up.

Fixes: https://tracker.ceph.com/issues/77930
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2026-07-04 16:56:33 +00:00
Anthony D'Atri
93c3f67031
Merge pull request #67779 from anthonyeleven/egregrioustypo
doc/rados/operations: Fix typos in erasure-code.rst
2026-07-04 10:15:30 -04:00
adatri
d524306b51 doc/rados/operations: Fix typos in erasure-code.rst
Signed-off-by: adatri <anthony.datri@gmail.com>
2026-07-04 10:05:33 -04:00
Kefu Chai
d8dde402b6 mgr: add heap admin socket command
the mon, osd and mds register a "heap" asok command that forwards to
ceph_heap_profiler_handle_command, but the mgr never did, so
`ceph daemon mgr.<id> heap stats` returns "no valid command found".
that leaves no way to tcmalloc-profile a leaking mgr through its admin
socket, which is what chasing mgr memory growth needs.

register the same command on MgrStandby's admin socket. its hook lives
for the whole process, so it works on both the standby and the active
mgr. heap output goes to the bufferlist, as on the other daemons, while
status keeps using the formatter.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-04 20:41:57 +08:00
Ronen Friedman
40e8ea0c1b
Merge pull request #69838 from ronen-fr/wip-rf-322936
crimson/os/alienstore: make CnLog::_flush non-blocking

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
Reviewed-by: Jose J Palacios-Perez <perezjos@uk.ibm.com>
2026-07-04 13:57:56 +03:00
Afreen Misbah
bd5fbb852e
Merge pull request #69815 from rhcs-dashboard/subsystem-resourcepage
mgr/dashboard: NVMe-oF – Subsystem Resource page enhancements


Reviewed-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-07-04 15:12:43 +05:30
Afreen Misbah
7bf1d637da
Merge pull request #69937 from tchaikov/wip-promql-test-deterministic
monitoring/ceph-mixin: scope test queries per dashboard

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-04 15:11:37 +05:30
Kefu Chai
171ee626fe
Merge pull request #69656 from tchaikov/wip-crimson-common-dump-metric-value-map
crimson/common: use the value_map passed to dump_metric_value_map()

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
2026-07-04 10:03:47 +08:00
Kefu Chai
fd78d1e7d2
Merge pull request #69697 from tchaikov/wip-crimson-throttle-releaser
crimson/osd: mark ThrottleReleaser [[nodiscard]]

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2026-07-04 10:03:15 +08:00
Kefu Chai
89cb0fc152 monitoring/ceph-mixin: scope test queries per dashboard
get_dashboards_data() keeps every query in a single map keyed by
"<panel title>-<legendFormat>", but that id is not unique: several
dashboards have a panel titled "IOPS", "OSDs" or "Throughput". The
dashboard read last overwrites the earlier entry, so one query shadows
another and never gets tested. glob() walks the files in filesystem
order, so which query survives, and whether the test passes, depends on
readdir.

run-tox-promql-query-test hit this in "Test IOPS Read"
(ceph-cluster.feature): the scenario feeds ceph_osd_op_r but the id
resolved to a CephFS pool panel.

    FAILED:
      expr: "sum(rate(ceph_pool_rd{cluster=~"mycluster|",  pool_id=~"UNSET VARIABLE"}[1m]))", time: 1m,
          exp:"{} 2.5E+01"
          got:"nil"
    HOOK-ERROR in after_scenario: AssertionError:

pool_id is "UNSET VARIABLE" because the scenario does not set $mdatapool,
and no ceph_pool_rd series were fed, so the query returns nil. It only
fails when readdir returns cephfsdashboard after ceph-cluster-advanced,
so the earlier change that walked nested rows and made the winner
deterministic did not fix it; it fixed which query wins, not that the
wrong one can win.

Key the queries per dashboard (data['queries'][<dashboard>][<id>]) and
have each .feature name its dashboard in a Background step. The lookup is
confined to that dashboard, so a title/legend used elsewhere no longer
shadows it. A duplicate within one dashboard is still an error, unless it
is in a collapsed row.

ceph-cluster.feature covers ceph-cluster-advanced; the rest map to one
dashboard each.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-04 09:40:07 +08:00
pujashahu
e6a2a8d996 mgr/dashboard: NVMe-oF – Subsystem Resource page enhancements
fixes:https://tracker.ceph.com/issues/77789

Signed-off-by: pujaoshahu <pshahu@redhat.com>

# Conflicts:
#	src/pybind/mgr/dashboard/frontend/src/app/ceph/block/nvmeof-group-form/nvmeof-group-form.component.html
#	src/pybind/mgr/dashboard/frontend/src/app/ceph/block/nvmeof-subsystem-overview/nvmeof-subsystem-overview.component.html
#	src/pybind/mgr/dashboard/frontend/src/app/ceph/block/nvmeof-subsystem-overview/nvmeof-subsystem-overview.component.spec.ts
#	src/pybind/mgr/dashboard/frontend/src/app/ceph/block/nvmeof-subsystem-overview/nvmeof-subsystem-overview.component.ts
2026-07-04 03:23:05 +05:30
Afreen Misbah
c11c7144fc
Merge pull request #69850 from rhcs-dashboard/edit-auth-subsystem
mgr/dashboard: NVMe-oF – Add edit authentication support for subsystem

Reviewed-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-07-04 01:48:27 +05:30
Anthony D'Atri
f45a073d9d doc: Improve src/common/options/mds-client.yaml.in
Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
2026-07-03 15:47:34 -04:00
Igor Fedotov
df23e844bc kv/Rocksdbstore: change output format for short txc dump.
Plus refactors txc dumping  code to permit Unit Testing for it.

Here is the new output sample:
Short form:
 DF:D*3,DF:O,MF:A,PF:O*2,PF:P,...*2
Verbose form:
PutCF(prefix = O, key = 'key', val len = 5)
PutCF(prefix = P, key = 'key2', val len = 6)
PutCF(prefix = O, key = 'key3', val len = 6)
DeleteCF(prefix = O, key = 'A1')
DeleteCF(prefix = D, key = 'A1')
DeleteCF(prefix = D, key = 'A2')
DeleteCF(prefix = D, key = 'A3')
MergeCF(prefix = A, key = 'A5', val len = 6)
<...>*2

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
2026-07-03 19:12:13 +03:00
Igor Fedotov
7dd1f39a70 os/bluestore: rename bluestore_log_op_verbose to
bluestore_log_op_verbose_kv_txc.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
2026-07-03 19:12:03 +03:00
pujashahu
6c923d6c57 mgr/dashboard: NVMe-oF – Add edit authentication support for subsystem
Fixes: https://tracker.ceph.com/issues/77840

Signed-off-by: pujaoshahu <pshahu@redhat.com>
2026-07-03 21:25:08 +05:30
pujashahu
ef6af190a7 mgr/dashboard: NVMe-oF – Subsystem Resource page enhancements
fixes:https://tracker.ceph.com/issues/77789

Signed-off-by: pujaoshahu <pshahu@redhat.com>
2026-07-03 21:25:08 +05:30
Afreen Misbah
71d61fb3e0
Merge pull request #69441 from rhcs-dashboard/edit-gateway
mgr/dashboard: Implement support for editing existing nvme-oF gateway groups through UI

Reviewed-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-07-03 20:14:51 +05:30
Imran Imtiaz
0272ea2254
Merge pull request #69883 from imran-imtiaz/wip-dashboard-clone-snap-id
mgr/dashboard: support snapshot_id in clone API for group snapshots
2026-07-03 12:46:45 +01:00
Afreen Misbah
6ed2ceda90
Merge pull request #69243 from rhcs-dashboard/bug-3990-pr2
mgr/dashboard : Add gateway filtering support for subsystem and namespace lists

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-07-03 16:57:42 +05:30
John Mulligan
4d59f1a835
Merge pull request #69912 from Sodani/shsodani_rgw_caps_final
mgr/smb: Allow pool creation for empty RGW buckets

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Anoop C S <anoopcs@cryptolab.net>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2026-07-03 07:09:49 -04:00
pujashahu
c3efbebd04 mgr/dashboard: Implement support for editing existing nvme-oF gateway groups through UI
Fixes: https://tracker.ceph.com/issues/77386

Signed-off-by: pujaoshahu <pshahu@redhat.com>
2026-07-03 15:11:52 +05:30
pujashahu
507765d71e mgr/dashboard: Implement backword support for Create existing nvme-oF gateway groups through UI
Fixes : https://tracker.ceph.com/issues/77632

Signed-off-by: pujaoshahu <pshahu@redhat.com>
(cherry picked from commit 26b7e9e4849f8f0f1d044cb23cc94a5ce910061d)
2026-07-03 13:41:19 +05:30
pujashahu
eb36d949e9 mgr/dashboard: Implement NVMe-oF Gateway Group Resource Overview Page
Fixes: https://tracker.ceph.com/issues/77383

Signed-off-by: pujaoshahu <pshahu@redhat.com>
(cherry picked from commit 254ab5383505962460d06b483dd931bf91c1f0b8)
2026-07-03 13:36:44 +05:30
Abhishek Desai
517f6da8a9 mgr/dashboard : Enable cephadm-signed TLS for NVMeOf
fixes: https://tracker.ceph.com/issues/77406
Signed-off-by: Abhishek Desai <abhishek.desai1@ibm.com>

(cherry picked from commit 9623f6847cecf7f2e79f461420fce4821fd1b159)
2026-07-03 13:32:08 +05:30
pujashahu
b7708cd711 mgr/dashboard : Add gateway filtering support for subsystem and namespace lists
Fixes: https://tracker.ceph.com/issues/77068

Signed-off-by: pujaoshahu <pshahu@redhat.com>
(cherry picked from commit e838bb19897ed4a8f6a548eca08e4fdd726e62f5)

 Conflicts:
	src/pybind/mgr/dashboard/frontend/src/app/ceph/block/nvmeof-namespaces-list/nvmeof-namespaces-list.component.spec.ts
	src/pybind/mgr/dashboard/frontend/src/app/ceph/block/nvmeof-subsystems/nvmeof-subsystems.component.spec.ts
2026-07-03 13:22:06 +05:30
Kefu Chai
440cba650a
Merge pull request #69856 from tchaikov/wip-seastar-sanitizer
crimson: keep seastar's sanitizers in lockstep with WITH_ASAN

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2026-07-03 15:30:31 +08:00
Afreen Misbah
8bbcd33f20
Merge pull request #69439 from rhcs-dashboard/fix-4877
mgr/dashboard: Implement NVMe-oF Gateway Group Resource Overview Page

Reviewed-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-07-03 12:49:40 +05:30
Afreen Misbah
2785810838
Merge pull request #69753 from rhcs-dashboard/atollon-fixes
mgr/dashboard: Add compression panels and PSU temperature

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
2026-07-03 12:48:59 +05:30
Afreen Misbah
16814fa200
Merge pull request #69757 from rhcs-dashboard/atollon-alerts
mgr/dashboard: Add node proxy hardware alerts


Reviewed-by: Aashish Sharma <aasharma@redhat.com>
2026-07-03 12:48:23 +05:30
Kefu Chai
507f77f400
Merge pull request #69864 from tchaikov/wip-rpm-build-deps
ceph.spec.in: drop rhel 8 support and use pkgconfig() when possible

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2026-07-03 09:18:33 +08:00
Kefu Chai
56ae6e2a0b
Merge pull request #69795 from tchaikov/wip-qa-cephfs-escape-warnings
qa/cephfs: fix invalid escape sequence SyntaxWarnings

Reviewed-by: Edwin Rodriguez <edwin.rodriguez1@ibm.com>
2026-07-03 09:17:51 +08:00
Kefu Chai
9cb50428b3
Merge pull request #69908 from sunyuechi/crc32c-riscv-zbc-no-gp-tp
common/crc32c: stop using gp/tp as scratch in RISC-V Zbc CRC32C

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-03 09:01:37 +08:00
Kefu Chai
64cc5407ae
Merge pull request #69652 from yzaken/bugfix_ceph_cli_watch_flag
ceph.in: reject -w/--watch flags when used with subcommands

Reviewed-by: Kobi Ginon <kginon@redhat.com>
Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-07-03 08:42:33 +08:00
mheler
ef7125e0f5
Merge pull request #69609 from mheler/wip-77538
mgr: filter root logger fallback
2026-07-02 17:16:59 -05:00
Afreen Misbah
42b3a7dfe8 mointoring: Add hardware alerts
Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-03 01:15:38 +05:30
Afreen Misbah
da21940d11 monitoring: add Ceph Hardware - Compression dashboard
Add comprehensive Grafana dashboard for monitoring hardware compression
metrics from FCM-enabled drives.

Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-03 01:12:20 +05:30
Afreen Misbah
b231e6f318 monitoring: add panel descriptions and fix tooltip labels
Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-03 00:48:41 +05:30
Afreen Misbah
4be833e11e monitoring: add PSU temperature graph to hardware dashboard
Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-03 00:48:41 +05:30
Edwin Rodriguez
c4472c0f9c cephfs-tool: wait for async callbacks before freeing contexts
bench_async_write_worker destroyed AsyncIOContext while the finisher
still ran async_io_callback after setting completed. Track in-flight
callbacks per worker and drain before tearing down the io_queue.

Signed-off-by: Edwin Rodriguez <edwin.rodriguez1@ibm.com>
2026-07-02 14:40:22 -04:00
Ilya Dryomov
a9da57265b
Merge pull request #69901 from tchaikov/wip-vstart-without-mgr
vstart.sh: create cephfs with fs new instead of fs volume create

Reviewed-by: Dhairya Parmar <dparmar@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2026-07-02 19:22:45 +02:00
Ronen Friedman
e4fa5048e0 crimson/os/alienstore: make CnLog::_flush non-blocking
CnLog::_flush() calls alien::submit_to(...).wait(), which blocks the
log flusher thread until the seastar reactor finishes outputting the
log entries. When the reactor writes to stderr and the pipe buffer is
full (e.g. two OSDs on the same host both dumping at debug level 20),
the reactor itself blocks on the write() syscall. Meanwhile BlueStore
threads filling the log buffer past m_max_new block in submit_entry()
waiting for the flusher - which is waiting for the reactor - which is
blocked on I/O. The kv thread cannot fire commit callbacks while it is
blocked on logging, so store transactions (including osdmap persistence)
hang indefinitely.

Fix by moving the entries into the lambda capture so _flush() can
return immediately without waiting.

This is a corrected version of the approach in commit 511af83e27
("crimson/os/alienstore/alien_log: _flush concurrently", PR #55262),
which was reverted in 4d52d1d22b because its lambda captured the
EntryVector by reference - the caller would clear/reuse the vector
before the lambda finished, causing a "pure virtual method called"
crash (https://tracker.ceph.com/issues/64140). The fix here avoids
that by swapping the entries out of the caller's vector and moving
them into the lambda's capture, so the lambda owns them for their
full lifetime.

Fixes: https://tracker.ceph.com/issues/77826
Fixes (failure of the 1st attempt): https://tracker.ceph.com/issues/64140
Assisted-by: Claude <claude.ai>

Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2026-07-02 16:33:05 +00:00
Shweta Sodani
bab1d6b39a mgr/cephadm: Add documentation for RGW keyring capability dependencies
This serves as a reminder for maintainers to verify whether capability
changes in the RGW service should also be applied to SMB and NFS
services to maintain consistency across services that use librgw.

Signed-off-by: Shweta Sodani <ssodani@redhat.com>
2026-07-02 21:52:11 +05:30
Afreen Misbah
8a39773f58
Merge pull request #69750 from rhcs-dashboard/atollon-dash
mgr/dashboard:  Add hardware monitoring dashboard using node proxy metrics

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Aashish Sharma <aasharma@redhat.com>
Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>
2026-07-02 21:24:09 +05:30
Guillaume Abrioux
4288984016 node-proxy: fix NodeProxySpec deserialization from JSON
NodeProxySpec.__init__() only accepts service_type and placement,
but ServiceSpec.from_json() passes all stored fields (ssl,
certificate_source, unmanaged, ...)

This breaks 'ceph orch ls' after the mgr reloads a node-proxy spec.

This commit forwards the standard ServiceSpec parameters to the
parent.

Fixes: https://tracker.ceph.com/issues/77874

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2026-07-02 17:46:56 +02:00
Imran Imtiaz
0b296db086 mgr/dashboard: support clone_by_snap_id flag in clone API for group snapshots
The clone API endpoint POST /api/block/image/{image_spec}/snap/{snapshot_name}/clone
only accepted a snapshot name, which uses rbd_clone3 (clone by name). This only works
for snapshots in the user namespace.

Group snapshots (with .group prefix, in RBD_SNAP_NAMESPACE_TYPE_GROUP namespace)
require rbd_clone4 (clone by snap ID). The rbd Python binding already supports this:
passing an int for p_snapshot triggers rbd_clone4.

Add an optional clone_by_snap_id boolean flag to the clone endpoint. When set, the
path parameter is treated as a numeric snap ID and passed directly as an int to
rbd.RBD().clone(). Clone format 2 is enforced because this flag is intended for
non-user namespace snapshots which cannot be protected (a prerequisite for clone
format 1).

Usage:
  POST /api/block/image/{image_spec}/snap/{snap_id}/clone
  {"clone_by_snap_id": true, ...}

Fixes: https://tracker.ceph.com/issues/77875
Signed-off-by: Imran Imtiaz <imran.imtiaz@uk.ibm.com>
Assisted-by: IBM Bob
2026-07-02 16:25:35 +01:00
John Mulligan
7c49869645
Merge pull request #69813 from phlogistonjohn/jjm-smb-rgw-doc-fixes
doc: fix some small errors and oversights for new rgw shares

Reviewed-by: Anoop C S <anoopcs@cryptolab.net>
Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
2026-07-02 10:05:00 -04:00
Kotresh HR
76df7a90b2
Merge pull request #69081 from karthik-us1/mirroring-checkpoints
tool/cephfs_mirror: Adding checkpoints for mirroring

Reviewed-by: Kotresh HR <khiremat@redhat.com>
2026-07-02 18:31:37 +05:30
Gabriel Benhanokh
a8e42b9ba0
Merge pull request #69434 from benhanokh/dedup_limits
rgw/dedup: extend shard scaling to support larger clusters
2026-07-02 15:30:33 +03:00
Ilya Dryomov
a81255e16f
Merge pull request #67968 from leonidc/prepare_import_migr_new_way
librbd/migration/NativeFormat: support specifying mon_host and key via spec

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2026-07-02 14:16:40 +02:00
Shweta Sodani
c1a672751d mgr/smb: Allow pool creation for empty RGW buckets
When an RGW bucket is empty, SMB clients cannot create directories,
files, or upload data because the default.rgw.buckets.data pool does
not yet exist. The previous CephX capabilities ('mon allow r') did not
permit pool creation during the first write operation.

The fix changes the mon capability from 'allow r' to 'allow *' to
enable pool creation on the first write to an empty RGW bucket. Once
the pool exists, subsequent operations succeed normally.

Fixes: https://tracker.ceph.com/issues/77764
Signed-off-by: Shweta Sodani <ssodani@redhat.com>
2026-07-02 17:44:35 +05:30
Devika Babrekar
4eb4e25e5e mgr/dashboard: Converting Create OSDs tearsheet type from Full to Wide
Fixes: https://tracker.ceph.com/issues/77132
Signed-off-by: Devika Babrekar <devika.babrekar@ibm.com>
2026-07-02 16:16:46 +05:30
Sun Yuechi
0338ed302a common/crc32c: stop using gp/tp as scratch in RISC-V Zbc CRC32C
crc32c_zbc's fold-by-four loop used gp (x3) and tp (x4) as scratch for
two folding temporaries. Save/restore around the loop is not enough:
they are ABI-reserved, so a signal delivered mid-loop runs its handler
with a corrupted tp, and the first TLS access there faults.

On riscv64 this reliably crashes the crimson/seastore unittests
unittest-transaction-manager and unittest-omap-manager, where seastar's
stall-detector timer fires often: the process dies with SIGSEGV in the
linker's TLS path with garbage in gp/tp.

Fixes: https://tracker.ceph.com/issues/77904
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-02 18:45:26 +08:00
Kefu Chai
bb9c1fde9f vstart.sh: only skip fs volume create when there's no mgr to run it
The mgr volumes module handles fs volume ls/create, not the monitor, so
without a mgr (CEPH_NUM_MGR=0) the "wait for volume module to load" loop
spun forever. Open-coding fs new unconditionally, my first attempt,
changed pool creation for every vstart user and dropped the wait, so
cephfs tests calling `fs volume ...` right after vstart.sh returns could
race a module still mid-load.

create_fs_volume() branches on CEPH_NUM_MGR instead. With a mgr it still
calls fs volume ls/create. Without one it runs the plain mon commands fs
volume create wraps: osd pool create cephfs.<name>.{meta,data}, then fs
new <name> <meta> <data>. The wait stays at its one call site above the
per-filesystem loop, so it runs once regardless of CEPH_NUM_FS, not once
per filesystem.

Verified against a local vstart: mgr present, fs volume ls polls once
then creates CEPH_NUM_FS=2 filesystems with no repeated poll;
CEPH_NUM_MGR=0, no poll, fs new creates 'a' directly, HEALTH_OK.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-02 18:40:59 +08:00
pujashahu
fb357cb4c4 mgr/dashboard: Implement NVMe-oF Gateway Group Resource Overview Page
Fixes: https://tracker.ceph.com/issues/77383

Signed-off-by: pujaoshahu <pshahu@redhat.com>
2026-07-02 15:49:38 +05:30
Ali Masarwa
d0607df424 rgw/posix: fix for posix versioning test
to expect a temporary 0 byte file instead of a dangling link

Signed-off-by: Ali Masarwa <amasarwa@redhat.com>
2026-07-02 13:04:30 +03:00
Ronen Friedman
c80f90324e crimson/osd: skip store flush for empty peering transactions
dispatch_context_transaction submits a real empty transaction to
AlienStore even when the peering context has no store ops. This
goes through the AlienCollection lock and thread pool slot machinery.
Under load from broadcast_map_to_pgs (many PGAdvanceMap operations
running concurrently with recovery), some of these empty transactions
get stuck inside AlienStore's pipeline and never reach BlueStore,
causing PGAdvanceMap to hang at complete_rctx indefinitely.

Fix by skipping the store round-trip for empty transactions. The
on_commit callbacks are still collected and fired, preserving the
fix from b0f5e1086a ("crimson/osd/pg: also trigger callbacks for
empty peering transactions"). The store flush that commit added was
incidental to the callback fix and is unnecessary here: the peering
pipeline's exclusive process stage already guarantees that prior
peering transactions on the same PG have completed.

Assisted-by: Claude <claude.ai>
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2026-07-02 09:37:39 +00:00
Karthik U S
fe7fa7b076 qa/test_mirroring: Add tests for mirroring checkpoints
Adding integration tests for validating the cephfs mirroring
checkpoints feature

Fixes: https://tracker.ceph.com/issues/73454
Signed-off-by: Karthik U S <karthik.u.s1@ibm.com>
2026-07-02 14:49:04 +05:30
Karthik U S
12bd34d69a doc: Add docs and pending release notes for mirroring checkpoints
Adding documentations and pending release notes for the mirroring
checkpoints feature.

Fixes: https://tracker.ceph.com/issues/73454
Signed-off-by: Karthik U S <karthik.u.s1@ibm.com>
2026-07-02 14:49:04 +05:30
Karthik U S
c64fe96365 mgr/mirroring,tools/cephfs_mirror: Handle checkpoint state transition
When a new checkpoint is being added or when the daemon gets restarted,
it will check whether the newly created checkpoint or any other old
checkpoints have already been mirrored onto the remote peer. If so, it
will transition to the correct state by checking for the highest snap id
present on the remote, and setting all the checkpoints which have snap id
lesser than or equal to that of the remote to COMPLETE. This is done by
sending an acquire notification from the mirroring module to the mirror
daemon, which is handled in the add_directory path, by adding the directory
to be checked for the state transition in the tick thread.
This path gets triggered:
a) when the daemon gets restarted
b) when the peer mapping changes
c) by sending the acquire notification from checkpoint add/now CLIs.
d) when mirroring module restarts

Fixes: https://tracker.ceph.com/issues/73454
Signed-off-by: Karthik U S <karthik.u.s1@ibm.com>
2026-07-02 14:49:04 +05:30
Karthik U S
37163436cc tools/cephfs_mirror: update checkpoint status during snapshot sync
When the sync completes or fails for a checkpointed snapshot, transition
the status of that checkpoint from CREATED/FAILED to COMPLETE/FAILED in
the do_sync_snaps() along with the timestamp of the event.

Fixes: https://tracker.ceph.com/issues/73454
Signed-off-by: Karthik U S <karthik.u.s1@ibm.com>
2026-07-02 14:49:03 +05:30
Karthik U S
0a394ef438 tools/cephfs_mirror: Helper functions for mirroring checkpoints
Implementation of helper functions and data structures for the
snapshot based mirroring checkpoints feature.

Fixes: https://tracker.ceph.com/issues/73454
Signed-off-by: Karthik U S <karthik.u.s1@ibm.com>
2026-07-02 14:49:03 +05:30
Karthik U S
959bc2eb07 mgr/mirroring: Add mirroring checkpoint CLIs
Add mgr checkpoint add/remove/ls/now commands that read and write
checkpoint metadata on the primary filesystem via do_snap_md_op.

Sample CLIs:
ceph fs snapshot mirror checkpoint add <vol-name> <dir-root> <snap-name>
ceph fs snapshot mirror checkpoint remove <vol-name> <dir-root> <snap-name>
ceph fs snapshot mirror checkpoint ls <vol-name> <dir-root>
ceph fs snapshot mirror checkpoint now <vol-name> <dir-root>

Fixes: https://tracker.ceph.com/issues/73454
Signed-off-by: Karthik U S <karthik.u.s1@ibm.com>
2026-07-02 14:49:03 +05:30
Afreen Misbah
aba8698e96 mgr/promethues: fixed tests and refactor module.py
- Update tests to check for queries in nested row panels
- fix tox tests
- add named tuple

Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-02 14:20:08 +05:30
Afreen Misbah
be324304db mgr/prometheus: refactor hardware metrics
- export `firmware_version` labels from `node_proxy_storage_capacity_bytes` metrics to be used in device firmware panel
- improve iterations for performance - dropping redundant node_proxy_firmware() RPC; firmware data is already
  present in the fullreport response, and removing unnecessary loops.
- replace health if/elif chain with HEALTH_STATUS_MAP dict lookup
- rename metrics to ceph_hardware_*, fix component labels and add tests
- added unit test cases
- add serial name, slot info to capacity metric
- fix health metrics showing ID instead of component name

Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-02 14:20:08 +05:30
Afreen Misbah
5dd6807e31 monitoring: improve hardware Grafana dashboard panels
- CPU/NVMe temp: change from stat to gauge panels with colored
  arc thresholds (green/yellow/red) and proper °C units
- Health panels: wrap queries in max() aggregation to show
  single worst-case value instead of overlapping series
- Pie charts: switch to donut style with visible legends
- Fan RPM: use locale unit for comma formatting instead of
  short which auto-scaled to "K"
- Fix temperature panels units
- Add Device List and Platform Firmware table panels

Fixes https://tracker.ceph.com/issues/77723

Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-02 14:20:08 +05:30
Afreen Misbah
ae0567e98a mgr/prometheus: Rename node_proxy_memory_capacity_mib to node_proxy_memory_capacity_bytes
Prometheus naming conventions require base units (bytes not MiB)

Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-02 14:20:08 +05:30
Afreen Misbah
04207ca6dc monitoring: fix hardware Grafana dashboard and health metrics
- Fix fan repeating panels: set multi=true on fan_speeds template
  variable so Grafana generates one panel per fan instead of one
- Remove TACH-only regex filter on fan_speeds template and AVG
  Cooling query so all system fans are visible regardless of naming
- Replace duplicate Power Control panel with Network health panel
- Fix NVMe drive count to use storage_capacity_bytes{protocol=NVMe}
  instead of counting temperature sensors (inaccurate proxy)
- Normalize all hostname filters to regex match (=~) for consistency
- Register hardware.libsonnet in dashboards.libsonnet so the
  dashboard JSON is generated during ceph-mixin builds
- Add temperatures category to prometheus health metrics loop

Signed-off-by: Afreen Misbah <afreen@ibm.com>
Assisted-by: Claude
Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-02 14:20:08 +05:30
Afreen Misbah
6bb07c6e53 mgr/prometheus: use orchestrator API for node-proxy hardware metrics
The hardware metrics exporter was reading cephadm's private KV store
directly via get_store_prefix('mgr/cephadm/node_proxy/data'). This
had two problems:

1. get_store_prefix() is module-scoped, so from the prometheus module
   it searched under prometheus's own namespace instead of cephadm's,
   resulting in zero metrics being exported despite metric definitions
   appearing at /metrics.

2. The firmware key was accessed as 'firmwares' (plural) but the
   stored field is 'firmware' (singular), causing all firmware version
   metrics to be silently empty.

Use node_proxy_fullreport() and node_proxy_firmware() via the
OrchestratorClientMixin instead. This routes through cephadm's
NodeProxyCache which handles KV access and firmware key compat
correctly. Follows the same pattern as set_cephadm_daemon_status_metrics()
and get_smb_metadata().

Signed-off-by: Afreen Misbah <afreen@ibm.com>
Assisted-by: Claude
Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-02 14:20:08 +05:30
Afreen Misbah
0514014687 monitoring: add hardware metrics Grafana dashboard
Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-02 14:20:08 +05:30
Afreen Misbah
aa8f05400d mgr/prometheus: add node-proxy hardware metrics exporter
Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-07-02 14:20:08 +05:30
Sun Yuechi
c22bb76ef5 src/xxHash: bump xxHash submodule to enable RISC-V RVV optimizations
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-02 01:05:52 -07:00
Ilya Dryomov
3b9dfd04eb
Merge pull request #69894 from tchaikov/wip-fix-nvmeof-spec
mgr/dashboard: fix stale nvmeof spec tests from a semantic merge conflict

Reviewed-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>
2026-07-02 10:01:15 +02:00
Vallari Agrawal
277b362439
mgr/dashboard: Fix daemon_name for NVMeoFClient
Reset self.daemon_name for matched gateway, to ensure
correct daemon_name is set when server_address param
is passed to the NVMeoFClient object.

Right now, NVMeoFClient().daemon_name shows same daemon_name
for all gateways within the group.

Also add unit test for NVMeoFClient in
src/pybind/mgr/dashboard/tests/test_nvmeof_client.py

Fixes: https://tracker.ceph.com/issues/77469

Signed-off-by: Vallari Agrawal <vallari.agrawal@ibm.com>
2026-07-02 12:44:38 +05:30
bluikko
897471b632
Merge pull request #69868 from bluikko/wip-doc-radosgw-sts-cleanup
doc/radosgw: improve style slightly in STS.rst
2026-07-02 12:57:31 +07:00
Shweta Bhosale
76f7f46dbb mgr/cephadm: remove spec.config options when a service is removed
For each key in spec.config, validate the option name and run
config rm on the service's config section. Call this from
remove_service() before spec_store.rm() schedules daemon removal.

Fixes: https://tracker.ceph.com/issues/77891
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-07-02 10:28:35 +05:30
Kefu Chai
7edbd7c61d mgr/dashboard: fix stale nvmeof spec tests from a semantic merge conflict
mgr-dashboard-frontend-unittests has been failing on main since 2026-07-01,
unrelated to whatever PR triggers it. Bisected to cfe12305fb, the merge
commit for PR #68180.

6182e67ea4 ("mgr/dashboard: NVMe Onboarding Setup Cards") moved gateway
group selection out of NvmeofSubsystemsComponent and
NvmeofNamespacesListComponent into a new NvmeofGatewayGroupFilterComponent,
dropping gwGroups, updateGroupSelectionState, handleGatewayGroupsError,
onGroupClear, and getSubsystems from both. Clean on its own.

PR #68180 forked from main in April, before that refactor existed, and
added three new tests against the old API. Also clean on its own, and never
rebased. Both PRs only touch lines the other doesn't, so when #68180 merged
34 minutes after 6182e67ea4, git's 3-way merge combined them without a
conflict. The merge is the only place the bug exists: new tests calling
methods the sibling parent had already deleted. TypeError at runtime,
TS2339/TS2551 from tsc, every run since. CI never caught it: GitHub was
showing #68180's make check as green from a run a full day older than
6182e67ea4.

nvmeof-namespaces-list.component.spec.ts: the dedup test is real, current
logic, it just called the old listNamespaces() instead of fetchData().
Fixed the call. The other two duplicate NvmeofGatewayGroupFilterComponent's
own spec; removed.

nvmeof-subsystems.component.spec.ts: same story, all three duplicate the
filter component's spec; removed.

Both removed error tests also checked preventDefault() on the error object,
which had no coverage elsewhere. Added that to
NvmeofGatewayGroupFilterComponent's spec, and hit a second bug while writing
it: the file's throwError(() => err) is the RxJS 7 factory overload, this
project is on RxJS 6.6.3, so it was emitting the arrow function itself as
the error. The existing error test never noticed since it only checks side
effects, not the error's identity. Used throwError(err) instead.

Verified: jest passes on all three spec files (was 6 failing), tsc -p
tsconfig.spec.json --noEmit is clean (was 17 errors), full block/nvmeof
sweep is 29/29 suites, 216/216 tests.

Fixes: cfe12305fb
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-02 11:28:11 +08:00
Kefu Chai
b1345f2576 common/web_cache: use ceph::shared_mutex instead of std::shared_mutex
WebCache declared its cache lock as a raw std::shared_mutex, bypassing the
project's ceph::shared_mutex alias that every other rwlock user goes through
(OSD::map_lock, TrackedOp::lock, ...). Align it with that convention: use
ceph::make_shared_mutex() to construct the member, and drop the explicit
<std::shared_mutex> template argument from each std::shared_lock/lock_guard
so they deduce whichever type ceph::shared_mutex resolves to.

This is idiom alignment only. The preceding commit fixes the Windows CI
timeout; this one does not touch it. ceph::shared_mutex only differs from
std::shared_mutex in debug builds, where it becomes ceph::shared_mutex_debug
for lockdep tracking. On the mingw-llvm release build that times out
unittest_web_cache, it still resolves straight to libc++'s std::shared_mutex.
The O(N^2) writer-wakeup behavior under heavy exclusive-lock contention is
unchanged.

Verified locally: unittest_web_cache builds and passes, exercising the debug
ceph::shared_mutex_debug path.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-02 09:46:12 +08:00
Kefu Chai
7240994bf5 test/common: reduce WebCache stress-test thread count
WebCacheConcurrencyTest sized its thread pool as hardware_concurrency() * 100,
which is 3200 on the 32-vCPU CI hosts. Linux runs that in a few seconds, but
the Windows (mingw-llvm/libc++) build times out unittest_web_cache at the
1800s per-test limit on every PR.

BasicAddSame reuses one key, so after the first insert every add() is a
shared-lock hit: 3200 threads log hit=3200000 miss=1 and finish in ~4s.
BasicAddUnique inserts a distinct key each iteration, so every add() misses
and takes the exclusive lock. libc++'s std::shared_mutex wakes all waiting
writers on every unlock, so N contending writers cost O(N^2) wakeups. At
N=3200 that alone overruns the 1800s budget, which is where the failing run
hangs. StampedeSyncCallOnce and StampedeMutex are exclusive-lock bound the
same way.

The tests only need enough threads to race; 4x oversubscription is enough.
Cap the count via a shared helper.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-02 09:46:12 +08:00
Kefu Chai
8a336b6718 ceph.spec.in: drop the python3_pkgversion macro
python3_pkgversion picked the native python3 package name on RHEL7, where
python3 was python34/python36 before 7.6 (cea9d18c). RHEL7 and RHEL8 are
gone, so on every supported distro (el9, el10, fedora, suse, openEuler) it
is always 3.

Replace python%{python3_pkgversion} with python3 and drop the fallback
default. No supported distro overrides it to build against a non-default
python flavor, so that capability goes too.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-02 09:42:23 +08:00
Kefu Chai
e125c47466 ceph.spec.in: use python3dist() for Python BuildRequires
Many Python deps were split across %if suse / %if rhel branches only
because openSUSE spells them CamelCase (PrettyTable, PyYAML, Sphinx,
CherryPy, Routes, Jinja2) and fedora/rhel/openEuler lowercase. The
python3dist() provides resolve the same on el9, el10, openEuler and
openSUSE, so those splits collapse to single lines.

Spell the deps python%{python3_pkgversion}dist(), matching the spec's
existing python%{python3_pkgversion}-Foo convention. Version constraints
and feature guards are preserved.

The dist name is the PyPI project name, not always the rpm name minus
python3-: python3-dateutil provides python3dist(python-dateutil), and
python3-saml provides python3dist(python3-saml).

Left as package names: the bare interpreter; python3-devel and numpy-devel,
which have no python3dist() provide; the build toolchain (setuptools,
Cython, pip, wheel); and the ceph-internal python3-* subpackages, which are
versioned = release deps.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-02 09:42:23 +08:00
Kefu Chai
8664e52d4b ceph.spec.in: use pkgconfig() for library BuildRequires
Library BuildRequires used per-distro -devel package names, some split
across %if suse / %if rhel branches only because the names differ. The
pkgconfig() virtual provides resolve identically on every rpm distro; the
spec already relies on them for fuse3, systemd and udev.

Switch every library dependency that ships a .pc to pkgconfig(), preserving
version constraints and feature guards. This collapses the suse-vs-rhel
naming splits (nss, keyutils, openssl, ldap, ibverbs, rdmacm, lz4, thrift,
libcryptopp) and the dedicated split blocks (nlohmann_json, lttng-ust,
babeltrace, expat) into single lines, and folds the duplicated xmlsec1 devel
packages together.

Left as package names: libaio and xfsprogs ship no reliable .pc; gperftools
carries version and tcmalloc-variant logic; numa is unconditional on rhel
but crimson-only on suse; and the xmlsec1 runtime backend plugins are
runtime test deps, not build-time pkgconfig() deps.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-02 09:42:23 +08:00
Kefu Chai
eb595fbad5 ceph.spec.in: drop RHEL <= 8 support
RHEL 8 and CentOS 8 left the supported test matrix in 2023 (449db416,
7a1dce1e). RHEL 8 has also been unbuildable since CMakeLists.txt began
requiring Python >= 3.9: it ships python 3.6 by default, so cmake refuses
to configure. The spec still carried the el8 conditionals, so assume
rhel >= 9 and drop them:

- remove branches reachable only on rhel <= 8: the gperftools 2.6.1
  BuildRequires, the tracker-36508 tcmalloc-libs requirement, and the
  dataclasses backport, which also targeted that dead python 3.6
- collapse "rhel >= 8" and "rhel >= 9" to plain "rhel"; with the floor at
  9 this is a no-op for rhel 9/10, fedora, suse and openEuler
- keep the rhel 9-vs-10 checks (== 9, < 10, >= 10)

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-02 09:42:23 +08:00
Samarah Uriarte
64fe83f2f4 rgw/posix: Implement SQLite lifecycle in POSIXUserDB
Signed-off-by: Samarah Uriarte <samarah.uriarte@ibm.com>
2026-07-01 15:55:02 -05:00
Ilya Dryomov
c7fddec189 doc/rbd: elaborate on key-ref syntax (existing in S3Stream and new in NativeFormat)
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2026-07-01 21:53:53 +02:00
Ilya Dryomov
e246fad552 qa/suites/rbd: add mon_host + key[-ref] coverage to migration-external
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2026-07-01 21:53:53 +02:00
Ilya Dryomov
047bbbf863 qa/suites/rbd: use client.0 entity in migration-external tests
Currently client.admin is passed for client_name and that doesn't
exercise client_name handling much as client.admin is the default.

When deploying multiple clusters the ceph task distributes keyrings
with only the key for the initial monitor and client.admin key.  Other
keys (e.g. client.0) are present only on their respective clusters, so
client.0's key for cluster2 needs to be obtained on cluster1 explicitly
with "ceph auth get".

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2026-07-01 21:53:53 +02:00
Leonid Chernin
705d865ae7 librbd/migration/NativeFormat: support specifying mon_host and key via spec
migration:
           -take  secret_key from spec
           -take  mon_host from the spec
           -ignore source keyring, source ceph.conf - bypass them
           -added validations of the new way schema
           -get key from the KV DB if key in spec  is actually key-ref

Fixes: https://tracker.ceph.com/issues/68177
Signed-off-by: Leonid Chernin <leonidc@il.ibm.com>
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2026-07-01 21:53:53 +02:00
John Mulligan
708d79cd5c doc: fix some small errors and oversights for new rgw shares
Fix some field names differing from the implementation and
add a missing field.

Fixes: 847dc5cc8c
Signed-off-by: John Mulligan <jmulligan@redhat.com>
2026-07-01 15:21:12 -04:00
Oguzhan Ozmen
66b1546825
Merge pull request #69713 from BBoozmen/wip-oozmen-77658
rgw/beast: fix shutdown crash from concurrent socket close in ConnectionList::close()
2026-07-01 14:29:32 -04:00
Oguzhan Ozmen
1c1eacc327
Merge pull request #69591 from BBoozmen/wip-oozmen-77509
rgw: add RGWObjCategory::MultiPart to separate multipart parts from completed objects in bucket stats
2026-07-01 14:24:20 -04:00
Sun Yuechi
f988cd91e3 tools/rados: do not close stdout after 'export' to '-'
'export' sets file_fd to STDOUT_FILENO for '-', but cleanup guarded
close() on STDIN_FILENO (always true), closing stdout. Guard on
STDOUT_FILENO instead, matching the 'import' path.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-01 09:46:24 -07:00
Sun Yuechi
0ea4eb19b0 tools/rados: fix leaked/unflushed output stream in 'ls'
`if (!stdout)` tested the libc FILE* (never null) instead of the
use_stdout flag, so the ofstream was never deleted when writing to
a file, leaking it and leaving its buffer unflushed. Test use_stdout.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-01 09:46:24 -07:00
Sun Yuechi
ef0b9f339f tools/rados: do not close stdin after put/append from '-'
do_put()/do_append() set fd to STDIN_FILENO for '-', but cleanup
guarded close() on STDOUT_FILENO (always true), closing stdin.
Guard on STDIN_FILENO instead.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-01 09:46:24 -07:00
Christopher Hoffman
aa70e12532 qa: Use mount class in test_fscrypt to read/write and to compare trees
In TestFSCryptVolumes, make sure to use mount class to
compare directory trees and to read/write files.

Fixes: https://tracker.ceph.com/issues/74110
Signed-off-by: Christopher Hoffman <choffman@redhat.com>
2026-07-01 16:42:23 +00:00
Ali Masarwa
52fa867b68 rgw/posix: fix remove API calls
Signed-off-by: Ali Masarwa <amasarwa@redhat.com>
2026-07-01 18:51:59 +03:00
Shweta Bhosale
c20ba11040 mgr/cephadm: redeploy stopped keepalived when ingress deps change
Fixes: https://tracker.ceph.com/issues/76304
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-07-01 21:10:09 +05:30
bluikko
a0c37e5c72
Merge pull request #67009 from bluikko/wip-doc-radosgw-ref-links-split4
doc/radosgw: change all intra-docs links to use ref (5 of 6)
2026-07-01 22:00:16 +07:00
mohit84
c354ebfe7c
Merge pull request #69428 from mohit84/op_touch_temp
core: Handle OP_TOUCH_TEMP in BlueStore without any guard(WITH_CRIMSON)
2026-07-01 20:21:35 +05:30
Ville Ojamo
9da610b679 doc/radosgw: improve style slightly in STS.rst
Fix capitalization and remove unnecessary dashes in text.
Use double backticks for literals.

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>
2026-07-01 21:49:08 +07:00
Yuval Lifshitz
96aea14fef
Merge pull request #69264 from ShreeJejurikar/wip-bucket-logging-lifecycle
rgw: emit bucket logging records for lifecycle delete actions
2026-07-01 17:22:54 +03:00
Sun Yuechi
e9750006d3 test/common: add RISC-V CRC32C performance benchmark to unittest_crc32c
Signed-off-by: Yuechi Sun <sunyuechi@gmail.com>
2026-07-01 21:38:27 +08:00
Jaya Prakash
b883573faa os/bluestore: add fast-path allocation latency perf counter for Allocator
Fixes: https://tracker.ceph.com/issues/76936

Signed-off-by: Jaya Prakash <jayaprakash@ibm.com>
2026-07-01 13:29:37 +00:00
Casey Bodley
c63915350f Revert "Reapply "qa/rgw/crypt: disable failing kmip testing""
This reverts commit d27261d0c2.

Fixes: https://tracker.ceph.com/issues/77873

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2026-07-01 09:22:21 -04:00
Casey Bodley
57342f5373 qa: pykmip task defaults to ceph fork
point the pykmip task to the ceph fork of the PyKMIP repo, which carries
fixes for newer python

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2026-07-01 09:22:21 -04:00
Igor Fedotov
ae39d63a93 kv/RocksdbKVStore: track max latencies for perf counters
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
2026-07-01 16:19:11 +03:00
Igor Fedotov
1b443af34c os/bluesore,kv/rocksdbstore: dump KV txc ops when slow
Depending on bluestore_log_op_verbose parameter one can see either
short:
2026-05-20T19:45:05.792+0300 7f60980006c0  0
bluestore(/home/if/ceph.2/build/dev/osd0) log_latency_fn slow operation
observed for _txc_committed_kv, latency = 0.004759922s, txc =
0x55c357bde900, txc bytes = 2756989, txc ios = 43, txc cost = 31566989,
txc onodes = 1, DB ops = ' p:P, p:P, p:O, p:O, p:O, p:O, p:O, p:O, p:O,
p:O, p:O, p:L, m:b, m:b, m:b, m:b, m:b, m:b, m:b, m:b, m:b, m:b, m:b,
m:b', DB updates = 24, DB bytes = 8937, cost max = 95489052 on
2026-05-20T18:38:20.749901+0300, txc max = 100 on
2026-05-20T18:38:21.215016+0300

or verbose:
2026-05-20T18:42:01.551+0300 7f60980006c0  0
bluestore(/home/if/ceph.2/build/dev/osd0) log_latency_fn slow operation
observed for _txc_committe
d_kv, latency = 0.003735413s, txc = 0x55c356a35500, txc bytes = 2757061,
txc ios = 44, txc cost = 32237061, txc onodes = 1, DB ops = '
PutCF( prefix = P key =
0x0000000000000499'.0000000032.00000000000000000005' value size = 185)
PutCF( prefix = P key = 0x0000000000000499'._fastinfo' value size = 194)
PutCF( prefix = O key =
0x7F8000000000000002DB7D25F2'!c!='0xFFFFFFFFFFFFFFFEFFFFFFFFFFFFFFFF6F0000000078
value size = 453)
PutCF( prefix = O key =
0x7F8000000000000002DB7D25F2'!c!='0xFFFFFFFFFFFFFFFEFFFFFFFFFFFFFFFF6F0006000078
value size = 455)
PutCF( prefix = O key =
0x7F8000000000000002DB7D25F2'!c!='0xFFFFFFFFFFFFFFFEFFFFFFFFFFFFFFFF6F000C000078
value size = 455)
PutCF( prefix = O key =
0x7F8000000000000002DB7D25F2'!c!='0xFFFFFFFFFFFFFFFEFFFFFFFFFFFFFFFF6F0012000078
value size = 455)
PutCF( prefix = O key =
0x7F8000000000000002DB7D25F2'!c!='0xFFFFFFFFFFFFFFFEFFFFFFFFFFFFFFFF6F0018000078
value size = 455)
PutCF( prefix = O key =
0x7F8000000000000002DB7D25F2'!c!='0xFFFFFFFFFFFFFFFEFFFFFFFFFFFFFFFF6F001E000078
value size = 455)
PutCF( prefix = O key =
0x7F8000000000000002DB7D25F2'!c!='0xFFFFFFFFFFFFFFFEFFFFFFFFFFFFFFFF6F0024000078
value size = 455)
PutCF( prefix = O key =
0x7F8000000000000002DB7D25F2'!c!='0xFFFFFFFFFFFFFFFEFFFFFFFFFFFFFFFF6F002A000078
value size = 21)
PutCF( prefix = O key =
0x7F8000000000000002DB7D25F2'!c!='0xFFFFFFFFFFFFFFFEFFFFFFFFFFFFFFFF6F
value size = 388)
MergeCF( prefix = b key = 0x0000000019080000 value size = 16)
MergeCF( prefix = b key = 0x0000000019100000 value size = 16)
MergeCF( prefix = b key = 0x0000000019180000 value size = 16)
MergeCF( prefix = b key = 0x0000000019200000 value size = 16)
MergeCF( prefix = b key = 0x0000000019280000 value size = 16)
MergeCF( prefix = b key = 0x0000000019300000 value size = 16)
MergeCF( prefix = T key = 0x0000000000000002 value size = 40)', DB
updates = 18, DB bytes = 4668, cost max = 95489052 on
2026-05-20T18:38:20.74
9901+0300, txc max = 100 on 2026-05-20T18:38:21.215016+0300

operations list when BlueStore observes slow op on KV commit.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
2026-07-01 16:19:11 +03:00
Edwin Rodriguez
18179111ba cephfs-tool: Add support for --async-write-fsync and --shared-file options
Add --async-write-fsync and --shared-file benchmark options.

--async-write-fsync sets ceph_ll_io_info.fsync on each async write to model
Ganesha-style write+commit and skips the post-phase ceph_ll_fsync when enabled.
--shared-file directs all worker threads at one file to stress client_lock
contention on a single inode.  Both options are recorded in JSON/text output.

Signed-off-by: Edwin Rodriguez <edwin.rodriguez1@ibm.com>
2026-07-01 09:11:15 -04:00
Edwin Rodriguez
eb107111d6 cephfs-tool: add stdout output support for JSON benchmark results
Signed-off-by: Edwin Rodriguez <edwin.rodriguez1@ibm.com>
2026-07-01 09:11:15 -04:00
Edwin Rodriguez
9c0f43e7da doc/cephfs: document new benchmark options and features
Fixes: https://tracker.ceph.com/issues/76183
Signed-off-by: Edwin Rodriguez <edwin.rodriguez1@ibm.com>
2026-07-01 09:11:15 -04:00
Edwin Rodriguez
c8d115ee5d cephfs-tool: Add filesystem statistics reporting before and after benchmark runs
Added filesystem stats reporting to cephfs-tool.cc.

Helper functions
• fetch_statvfs() — uses ceph_ll_statfs on the root inode when --async-io
  is set, otherwise ceph_statfs on mount_root
• report_fs_stats() — prints and optionally JSON-encodes:
  • Total / free blocks
  • File count

When it runs
• Before the iteration loop (after the bench directory is created)
• After all iterations finish (before the final report and optional cleanup)

Console output example

Filesystem statistics (before iterations):
  Total blocks:      ...
  Free blocks:       ...
  Files:             ...

JSON output (when --json is used) adds a filesystem_stats section with before
and after objects containing the same fields.

Fixes: https://tracker.ceph.com/issues/76183
Signed-off-by: Edwin Rodriguez <edwin.rodriguez1@ibm.com>
2026-07-01 09:11:15 -04:00
Edwin Rodriguez
d9dbfde2b7 cephfs-tool: add explanatory comments for benchmark features
Add inline comments documenting each benchmark enhancement and its CLI
flag. Remove duplicate ceph_pthread_setname calls accidentally introduced
in sync read/write workers.

Fixes: https://tracker.ceph.com/issues/76183
Signed-off-by: Edwin Rodriguez <edwin.rodriguez1@ibm.com>
2026-07-01 09:11:15 -04:00
Edwin Rodriguez
d96e5b148f cephfs-tool: set thread names for read/write workers
Set pthread names on all benchmark worker threads (wr-worker-N, rd-worker-N)
via ceph_pthread_setname so they are visible in top, perf, and lockstat
output during profiling.

Fixes: https://tracker.ceph.com/issues/76183
Signed-off-by: Edwin Rodriguez <edwin.rodriguez1@ibm.com>
2026-07-01 09:11:15 -04:00
Edwin Rodriguez
51f9de8d34 cephfs-tool: add async I/O support with configurable queue depth
Add --async-io to use ceph_ll_nonblocking_readv_writev for read and write
workers instead of synchronous ceph_read/ceph_write. Add --queue-depth to
limit outstanding async I/Os per thread via a counting semaphore and
pre-allocated AsyncIOContext pool. Async write uses ceph_ll_create; async
read uses ceph_ll_walk + ceph_ll_open and defers inode release until all
async ops including readahead complete. Reject --async-io with
--per-thread-mount since async workers require a shared mount.

Fixes: https://tracker.ceph.com/issues/76183
Signed-off-by: Edwin Rodriguez <edwin.rodriguez1@ibm.com>
2026-07-01 09:11:15 -04:00
Edwin Rodriguez
47f7475545 cephfs-tool: add lockstat collection and dumping support
When built with CEPH_LOCKSTAT, add --lockstat-dump and --lockstat-threshold
to collect lock contention data during benchmark phases. Start collection
at benchmark begin, reset counters before each read/write phase, and dump
per-iteration/per-phase JSON files (e.g. base_iter1_write.json). Requires
a lockstat-instrumented libcephfs build.

Fixes: https://tracker.ceph.com/issues/76183
Signed-off-by: Edwin Rodriguez <edwin.rodriguez1@ibm.com>
2026-07-01 09:11:15 -04:00
Edwin Rodriguez
f6b2f5fee9 cephfs-tool: Add support for overriding 'ms_async_op_threads' configuration
Add --msgr-workers to override ms_async_op_threads before mount (1-24;
0 keeps the ceph.conf default). Controls async messenger I/O threads
independently of benchmark worker thread count. Validate the range at
startup and include the resolved value in console and JSON output.

Fixes: https://tracker.ceph.com/issues/76183
Signed-off-by: Edwin Rodriguez <edwin.rodriguez1@ibm.com>
2026-07-01 09:11:15 -04:00
Edwin Rodriguez
50fc43d006 cephfs-tool: Add progress tracking support for benchmarking phases
Add --progress and --progress-interval to display live bandwidth, IOPS,
and ETA during read/write phases. A background progress_reporter thread
aggregates per-thread stats every 100ms and prints a single updating
status line. Progress percentage is based on elapsed time when --duration
is set, otherwise on files completed. Make ThreadStats fields atomic so
workers and the reporter can access them concurrently.

Fixes: https://tracker.ceph.com/issues/76183
Signed-off-by: Edwin Rodriguez <edwin.rodriguez1@ibm.com>
2026-07-01 09:11:15 -04:00
Edwin Rodriguez
75b0084f52 cephfs-tool: Add support for overriding the 'client_oc' options
Add --client-oc and --client-oc-size to override libcephfs object cacher
settings before ceph_init. Enables benchmarking with the object cacher
enabled or disabled and with a configurable cache size. Resolved values
are shown in console and JSON output.

Fixes: https://tracker.ceph.com/issues/76183
Signed-off-by: Edwin Rodriguez <edwin.rodriguez1@ibm.com>
2026-07-01 09:11:15 -04:00
Edwin Rodriguez
e565a2b06d cephfs-tool: Add support for dumping performance counters
Add --perf-dump to write libcephfs client performance counters to a file
at the end of the benchmark. Uses ceph_get_perf_counters() while the
mount is still active so counters reflect the full benchmark run.

Fixes: https://tracker.ceph.com/issues/76183
Signed-off-by: Edwin Rodriguez <edwin.rodriguez1@ibm.com>
2026-07-01 09:11:15 -04:00
Edwin Rodriguez
71cd5806a8 cephfs-tool: Add duration-based phase limit for benchmarking
Add --duration to cap each read and write phase at N seconds instead of
running until all files are processed. Workers loop until the time limit
expires, cycling through their per-thread file set in duration mode.
Pass phase start time into worker threads for consistent cross-thread
duration checks. Only count each unique file once in stats even when
filenames are reused across cycles.

Fixes: https://tracker.ceph.com/issues/76183
Signed-off-by: Edwin Rodriguez <edwin.rodriguez1@ibm.com>
2026-07-01 09:11:15 -04:00
Edwin Rodriguez
c1ef8e868a cephfs-tool: Add JSON output support for benchmarking results
Add --json to write structured benchmark results (configuration,
per-iteration read/write metrics, and cross-iteration summary statistics)
to a file using JSONFormatter. Extend print_statistics() to optionally
mirror mean, stddev, min, and max into the JSON document. Improve
parse_size() to accept both SI (1000-based, e.g. "4MB") and binary
(1024-based, e.g. "4MiB") size suffixes.

Fixes: https://tracker.ceph.com/issues/76183
Signed-off-by: Edwin Rodriguez <edwin.rodriguez1@ibm.com>
2026-07-01 09:11:15 -04:00
Sun Yuechi
439534ab9b rgw/sts: fix inverted tokenCode length validation
validate_input() rejected the valid TOKEN_CODE_SIZE length and accepted
all others. Compare with != so only a wrong length is rejected.

Fixes: https://tracker.ceph.com/issues/77872
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-01 20:20:20 +08:00
Sun Yuechi
93febadb4f rgw/posix: fix inverted If-None-Match check allowing overwrite
If-None-Match: * must fail when the object exists, but the posix writer
did the opposite, disabling overwrite protection. Return
PRECONDITION_FAILED when the object exists.

Fixes: https://tracker.ceph.com/issues/77871
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-01 20:19:10 +08:00
Sun Yuechi
b703ec465f rgw: fix inverted MFA check in DeleteMultiObj
A versioned delete carries a non-empty version-id, but the MFA guard
flagged the empty case: versioned deletes bypassed MFA while plain
deletes wrongly required it. Test !instance.empty().

Fixes: https://tracker.ceph.com/issues/77869
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-01 20:17:09 +08:00
Theofilos Mouratidis
f9941e765e rgw: rgw_rest_account.cc: Fix quota variable types from int32_t to int64_t
The max_size and max_objects in the Admin-OPs REST interface were accidentally set to int32_t
This allowed to set only up to 2GB of quota for accounts

Signed-off-by: Theofilos Mouratidis <mtheofilos@gmail.com>
2026-07-01 13:59:23 +02:00
Theofilos Mouratidis
411a50acd8 rgw: rgw_rest_account.cc: remove trailing spaces
Signed-off-by: Theofilos Mouratidis <mtheofilos@gmail.com>
2026-07-01 13:58:58 +02:00
Sun Yuechi
11c48eb547 crimson/mon/MonClient: fix use-after-free in run_command
run_command() captured a reference to the just-emplaced mon_commands
element into the send_message() continuation. mon_commands is a
std::vector, so a concurrent run_command() could reallocate it and
invalidate the reference before the continuation ran, dereferencing
freed memory.

No caller issues concurrent commands today, but the pattern is unsafe.

Take the result future before issuing the message, then coroutinize
the method so it is awaited directly instead of being captured into a
continuation lambda.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-01 19:13:32 +08:00
Sun Yuechi
2601725fbf mds/Server: return after responding on error paths
handle_client_symlink (-ENAMETOOLONG), reclaim_session (-EPERM) and
handle_client_readdir_snapdiff (-EINVAL) each sent an error reply on a
failure path but fell through, replying to the same MDRequest a second
time. reclaim_session additionally went on to bind reclaiming_from,
bypassing the auth_name check. Return right after sending the reply.

Fixes: https://tracker.ceph.com/issues/77862
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-01 19:07:59 +08:00
Kefu Chai
cc1f41c8d2
Merge pull request #69156 from sunyuechi/wip-riscv64-jsonnet-bundler-v0.6.0
monitoring/ceph-mixin: bump jsonnet-bundler to v0.6.0

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-07-01 18:57:00 +08:00
Afreen Misbah
cfe12305fb
Merge pull request #68180 from rhcs-dashboard/bug-13712
mgr/dashboard: Add option should be enabled even if allow all host access is enabled in nvme/tcp

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Abhishek Desai <abhishek.desai1@ibm.com>
Reviewed-by: Dnyaneshwari Talwekar <dtalweka@redhat.com>
Reviewed-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-07-01 16:21:00 +05:30
Afreen Misbah
2538df0b13
Merge pull request #69241 from rhcs-dashboard/bug-3990-pr1
mgr/dashboard: NVMe Onboarding  Setup Cards
2026-07-01 15:47:44 +05:30
Ville Ojamo
276da74687 common/options: fix invalid command for STS key generation
Config rgw_sts_key expects a 16-character hexadecimal key. The option
long_desc has an example with openssl rand -hex 16 which generates
16 hexadecimal bytes that consist of 32 characters rendered in ASCII.
Modify the example openssl command to output 8 bytes that matches the
required length of 16 characters.

Remove a space after a slash.

Raised in the doc bugs pad.

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>
2026-07-01 16:19:15 +07:00
Sun Yuechi
db8d59ee4d osd: report -EINVAL to on_finish in asok_route_to_pg catch
The handler passed the stale locate result (0) while returning -EINVAL,
so callers saw success on a bad_cmd_get failure.

Only reachable via the "scrubdebug" asok command (the sole user of
asok_route_to_pg) when a parameter fails type parsing.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-01 16:37:35 +08:00
Venky Shankar
8b05b5da09
Merge pull request #69021 from salieri11/igolikov-bug-74625
client: keep dentry alive while unlinking trimmed cache entries

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2026-07-01 13:35:58 +05:30
Pablo de Lara
b160e2a924 isa-l: Update isa-l submodule from 2.32.0 to 2.32.1
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
2026-07-01 07:50:45 +00:00
bluikko
d821b2814f
Merge pull request #67007 from bluikko/wip-doc-radosgw-ref-links-split2
doc/radosgw: change all intra-docs links to use ref (3 of 6)
2026-07-01 13:07:28 +07:00
Venky Shankar
fc9f8642b2 Merge PR #66558 into main
* refs/pull/66558/head:
	qa/cephfs: minor fix in comment
	qa/cephfs: give more time to tests in test_clone_stats.py
	qa/cephfs: increase number of files to cloned in test_clone_stats.py
	volumes/stats_util: improve log messages

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Jos Collin <jcollin@redhat.com>
2026-07-01 11:14:38 +05:30
Igor Golikov
ad95bb5450 client: fix Dentry UAF by holding DentryRef across unlink
Hold a function-scoped DentryRef in Client::unlink() to keep the dentry
alive through the entire teardown sequence.

Without the O_DIRECTORY dentry pin (PR #60909, merged after v19.2.1),
opening a directory does not call _ll_get(), so the dentry stays at
ref=1 while inode->dir is still active. When trim_cache() evicts this
dentry, Dentry::unlink() calls put() for the dir pin, ref drops to 0,
and the dentry is freed while Client::unlink() still needs it for
detach/lru_remove.

A function-scoped DentryRef guarantees the dentry survives until after
it has been properly removed from the dir and LRU, regardless of pin
state.

Signed-off-by: Igor Golikov <igolikov@ibm.com>
Fixes: https://tracker.ceph.com/issues/74625
2026-07-01 11:04:38 +05:30
Igor Golikov
227b558f20 client: add assert to catch dentry ref drop during unlink
Add ceph_assert(dn->ref > 0) after Dentry::unlink() in Client::unlink()
to catch the case where internal put() calls drop the dentry reference
to zero before we proceed to detach/lru_remove.

This turns a silent use-after-free into a clean assertion failure with
a full stack trace, making it possible to catch the exact conditions
in a test environment.

Signed-off-by: Igor Golikov <igolikov@ibm.com>
Fixes: https://tracker.ceph.com/issues/74625
2026-07-01 11:04:38 +05:30
Kefu Chai
2b638869b2 crimson: keep seastar's sanitizers in lockstep with WITH_ASAN
Seastar_SANITIZE defaults to "DEFAULT", which turns on ASan and UBSan for
Debug and Sanitize builds regardless of ceph's WITH_ASAN. A Debug build
with WITH_ASAN=OFF then instruments seastar while ceph links tcmalloc, and
the seastar tests crash before main() because ASan calls the allocator via
dlsym before it is initialized.

Force Seastar_SANITIZE to follow WITH_ASAN, next to the other forced
Seastar_* cache variables, so seastar is sanitized only when ceph is. This
keeps the fix on the ceph side instead of carrying a seastar fork patch.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-07-01 13:02:57 +08:00
Sun Yuechi
9f6701aa7b crimson/osd/ec_backend: fix transate_transaction typo
Rename transate_transaction() to translate_transaction(). The function
translates an os::Transaction into a PGTransaction.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-01 12:46:22 +08:00
Sun Yuechi
508e6972fa crimson/osd/ec_backend: fix OP_OMAP_RMKEYRANGE translation
transate_transaction() handled OP_OMAP_RMKEYRANGE in the same case
block as OP_OMAP_SETHEADER, decoding a single bufferlist and calling
omap_setheader(). But OP_OMAP_RMKEYRANGE encodes first/last as two
strings, so decode_bl() misreads the stream and the wrong op is
applied.

Give OP_OMAP_RMKEYRANGE its own case: decode the two strings and pass
them to the two-string PGTransaction::omap_rmkeyrange() overload, which
encodes them into the string[2] bufferlist internally.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-01 12:46:22 +08:00
Sun Yuechi
853968f367 crimson/osd/ec_backend: add missing break in transaction translation
ECCrimsonOp::transate_transaction() lacked a break after OP_OMAP_CLEAR
and OP_CLONERANGE2, so both cases fell through to the next one.

OP_OMAP_CLEAR fell into OP_OMAP_SETKEYS, which decodes an attrset from
the transaction iterator that OP_OMAP_CLEAR never encoded, corrupting
the decode stream for every following op.

OP_CLONERANGE2 fell into OP_CLONE; clone_range() had already marked the
destination op non-empty, so the following clone() trips
ceph_assert(op.is_none() || op.is_delete()).

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-01 12:46:22 +08:00
Ronen Friedman
6f3fa9a289 crimson/os/seastore: documenting the LBA manager
documenting the LBA manager's internal structure and operations.

Assisted-by: Claude <claude.ai>
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2026-07-01 04:17:40 +00:00
Sun Yuechi
e6dee4aac7 osd: dump_osd_network min section now reports min not max
The 'min' section uses sitem.max, which looks like a copy-paste slip;
if so, the computed min values would never be emitted.

Fixes: https://tracker.ceph.com/issues/77851
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-01 11:01:59 +08:00
Sun Yuechi
29017afa1a common/pick_address: match against the current iface in pick_iface
The loop iterated over addr but always passed the list head *ifa
to matches_with_net, so it only ever tested the first interface
while returning the iterated interface's name. Match against
*addr.

Fixes: https://tracker.ceph.com/issues/77847
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-01 10:06:29 +08:00
Kefu Chai
94bf1d7087
Merge pull request #69851 from anthonymicmidd/foundation-members
doc/foundation: Removed members section

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2026-07-01 10:00:55 +08:00
Anthony M
901a4944bb doc/foundation: Removed members section
These updates will allow us to update a single page rather than two. The detailed member lists and categories have been removed. I've added a redirect for visitors to this page to https://ceph.io/en/foundation/members/

Signed-off-by: Anthony M <amiddleton@linuxfoundation.org>
2026-07-01 09:44:27 +08:00
Yuri Weinstein
36ebea8177
Merge pull request #69440 from jdurgin/wip-release-notes-script
scripts/ceph-release-notes: use raw strings for escaped characters

Reviewed-by: Yuri Weinstein <yweinste@redhat.com>
2026-06-30 15:36:42 -07:00
Jesse Williamson
9922bd1096
Merge pull request #65535 from ceph/jfw-wip-rgw-libfdb
Add libfdb: Experimental FoundationDB support
2026-06-30 15:07:58 -07:00
Ilya Dryomov
97128d4d3b
Merge pull request #69836 from sunyuechi/wip-librbd-trash-purge-uaf
librbd: fix use-after-free in trash purge on image open error

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2026-06-30 21:34:36 +02:00
Adam King
7272236d10 mgr/cephadm: clear daemon_config_deps entry for removed daemons
This covers both removing them actively from now on when removing
daemons from the HostCache and also not loading entries like this
back in next time they are loaded when a user upgrades to a
version with this patch

Fixes: https://tracker.ceph.com/issues/77839

Signed-off-by: Adam King <adking@redhat.com>
2026-06-30 15:31:20 -04:00
Sun Yuechi
f18247e32b librbd: fix use-after-free in trash purge on image open error
When ImageState::open() fails with an error other than -ENOENT, it
asynchronously deletes the ImageCtx before returning. The trash purge
loop only skipped the -ENOENT case and fell through on every other
error, dereferencing the already-freed ictx in diff_iterate() and
state->close(). Add the missing continue so a failed open is skipped.

The fall-through dates back to 504f4e78, which split `if (r < 0)
continue` to log non-ENOENT errors but dropped the continue.

Fixes: https://tracker.ceph.com/issues/77836
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-07-01 00:13:42 +08:00
Jesse F. Williamson
132c77376f Add FoundationDB client library "libfdb".
Assisted-by: Codex:GPT-5

Signed-off-by: Jesse F. Williamson <jfw@ibm.com>
2026-06-30 09:06:20 -07:00
Matthew N. Heler
84d7550fc8 rgw/kms: null-check perfcounter before using it
Guard the KMS key fetch so we only touch perfcounter when it exists.
The fetch latency PerfGuard now sits in an optional and the error
counters are wrapped in an if, so both are skipped when perfcounter
is null.

This keeps key reconstitution working from radosgw-admin (like lc
process), where the perfcounter global isn't set up the way it is
inside radosgw.

Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
2026-06-30 10:22:13 -05:00
Matthew N. Heler
15a0e8f172 rgw: drop dedup shared-manifest and blake3 attrs on object rewrites
Do not carry RGW_ATTR_SHARE_MANIFEST or RGW_ATTR_BLAKE3 when object data
or manifest ownership is rewritten through admin rewrite, remote fetch,
CopyObject, or lifecycle transition.

Both attributes describe RGW dedup state for an existing RADOS layout:
the shared-manifest marker and the blake3 digest of the object's on-disk
bytes. Once a path writes a new layout or creates a new destination head
with its own refcount tag, preserving them leaves stale dedup on the
destination.

Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
2026-06-30 10:22:13 -05:00
Matthew Heler
1720adf625 qa/rgw/lifecycle: split transition recompress into separate jobs
Without a facet marker, teuthology concatenated all three yamls into
one job, running plain, CBC, and GCM workunits back-to-back under
whichever rgw override won.

Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
2026-06-30 10:22:13 -05:00
Matthew N. Heler
75f81837ce rgw: extract shared recompress pipeline for CopyObject and LC transitions
Extract the decrypt/decompress/recompress/re-encrypt filter pipeline
into a shared RGWRecompressDPF base class. RGWTransitionDPF is added
for lifecycle transitions and RGWCopyObjDPF is rewritten on top of
the same base, both as thin subclasses providing key retrieval and
compression config.

Add rgw_prepare_decrypt_object() for decrypting from stored attrs
without an HTTP request context (handles SSE-KMS/-GCM, SSE-S3/-GCM,
RGW-AUTO/-GCM), and rgw_prepare_reencrypt_object() which regenerates
the GCM salt before re-deriving the per-object key — avoids AEAD
nonce reuse when LC re-encrypts modified plaintext.

Includes a QA workunit that drives transitions through three
storage classes with different compression configs and verifies
data integrity, including with SSE-KMS encrypted objects.

Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
2026-06-30 10:21:55 -05:00
Venky Shankar
1d7f80b43b
Merge pull request #69031 from neesingh-rh/wip-71795
qa: fixing failures in TestShellOpts

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2026-06-30 19:10:09 +05:30
Venky Shankar
084d84f3f9 Merge PR #69324 into main
* refs/pull/69324/head:
	libcephfs: increment extra version to indicate new flock functions
	libcephfs: add public api for ceph_ll_flock

Reviewed-by: Venky Shankar <vshankar@redhat.com>
Reviewed-by: Greg Farnum <gfarnum@redhat.com>
2026-06-30 19:09:27 +05:30
Venky Shankar
24a317bc3c
Merge pull request #66723 from rishabh-d-dave/snap-metadata-mutate
cephfs: allow snapshot metadata mutations

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2026-06-30 19:06:39 +05:30
Shweta Bhosale
6ec51414a9 mgr/cephadm: Scale down ingress service only when the backend NFS service is scaled down
When scaling ingress down with a ranked NFS backend that still has more
daemons than its placement target, remove ingress from hosts without NFS
first and skip removing co-located ingress on NFS hosts until the next
iteration after NFS scales down.

Fixes: https://tracker.ceph.com/issues/76304
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-30 18:22:11 +05:30
Shweta Bhosale
4ca10faf79 mgr/cephadm: Disable keepalived by default and start during serve loop if needed
If the ingress service is configured with a placement count lower than the number of hosts,
then when one node goes down, Keepalived is configured on the remaining hosts.
When the failed host comes back up, if it was previously configured as the master,
the VIP may become active on two hosts. To prevent this, Keepalived is disabled by default
and started within the service loop.

Fixes: https://tracker.ceph.com/issues/76304
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-30 18:22:11 +05:30
Shweta Bhosale
e33edc8890 mgr/cephadm: Fixed rank calculation logic for potential candidates when NFS daemon colocation is enabled
When colocation of NFS daemons is allowed, the existing rank calculation incorrectly derives the
rank based solely on the number of hosts. This leads to inaccurate rank assignment because the logic
does not account for the full set of potential candidates, especially in scenarios where multiple
daemons can reside on the same host.

Fixes: https://tracker.ceph.com/issues/73442
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-30 18:22:11 +05:30
Shweta Bhosale
4e20140ecf mgr/cephadm: Fixed cache update for updating daemon user_stopped status
Fixes: https://tracker.ceph.com/issues/73442

Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-30 18:22:11 +05:30
Shweta Bhosale
51504dbf2a mgr/cephadm: Only start ranked daemon with highest rank gen, if many daemon exists with same rank
If the removal of an extra NFS daemon fails for any reason, that daemon should not be started, as the NFS daemon with the highest rank generation is already running.

Fixes: https://tracker.ceph.com/issues/73442
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-30 18:22:11 +05:30
Shweta Bhosale
918adfa512 mgr/cephadm: Stop NFS service/daemon from starting automatically after reboot, cephadm to manage startup
Fixes: https://tracker.ceph.com/issues/73442
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-30 18:22:10 +05:30
Igor Fedotov
641038fb0e test/osd-bluefs-volume-ops: minor cleanup
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
2026-06-30 15:46:56 +03:00
Igor Fedotov
f88927d283 qa/osd-bluefs-volume-ops: add more bluestore expansion test cases.
Reproduces: https://tracker.ceph.com/issues/75852
Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
2026-06-30 15:46:54 +03:00
Igor Fedotov
d6e021d76b os/bluestore: do not add expanded space to allocator after allocmap
recovery.

The recovery procedure performs that itself by design.
Fixes: https://tracker.ceph.com/issues/75852

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
2026-06-30 15:42:41 +03:00
Igor Fedotov
4b74d0af8f os/bluestore: make 'bluestore_debug_enforce_settings' control
rotational/non-rotational OSD behavior for DB/WAL devices.

Prior to this commit this parameter determined what device class
settings to apply for BlueStore only. OSD had partially applied
actual journal/db/main device class for some operations, e.g. benchmark
on init or allocmap persistence, irrespective to this setting value.
This commit allows 'bluestore_debug_enforce_settings" to override the
actual device class completely.

Signed-off-by: Igor Fedotov <igor.fedotov@croit.io>
2026-06-30 15:42:41 +03:00
Casey Bodley
283108e66b
Merge pull request #69371 from cbodley/wip-76961
qa/dnsmasq: use managed dnsmasq instead of editing resolv.conf

Reviewed-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-30 08:30:26 -04:00
Igor Fedotov
a65fd3baf3
Merge pull request #66344 from JoshuaGabriel/68797-bdev-expand-admin
os/bluestore: asok command to expand block device and Bluestore while OSD is running

Revewed-by: Igor Fedotov <igor.fedotov@croit.io>
2026-06-30 15:13:45 +03:00
Redouane Kachach
965ded5698
Merge pull request #69808 from rkachach/fix_issue_bump_python_version
mgr/cephadm: bump Python version path mapped in kcli dev env

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Kobi Ginon <kginon@redhat.com>
2026-06-30 14:09:48 +02:00
Igor Fedotov
0f23c68bf6
Merge pull request #69820 from sunyuechi/wip-blk-aio-fix-paren
blk/aio: fix mismatched parenthesis in POSIX AIO submit path

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
Reviewed-by: Igor Fedotov <igor.fedotov@croit.io>
2026-06-30 15:06:20 +03:00
pujashahu
6182e67ea4 mgr/dashboard: NVMe Onboarding Setup Cards
Fixes: https://tracker.ceph.com/issues/77067

Signed-off-by: pujaoshahu <pshahu@redhat.com>
2026-06-30 17:32:06 +05:30
Kobi Ginon
e6679c37ac qa/mgr: fix influx module_selftest health warning noise
Skip influx selftests when the Python influxdb client is not installed.
Remove the ineffective testhost workaround and ignore the expected
MGR_INFLUX_NO_SERVER warning when the module runs without a server.

Fixes: https://tracker.ceph.com/issues/77250
Signed-off-by: Kobi Ginon <kginon@redhat.com>
2026-06-30 14:58:26 +03:00
Ronen Friedman
5e9da92c17
Merge pull request #69711 from ronen-fr/wip-rf-mold3
cmake: implement WITH_MOLD option for Mold linker support

Reviewed-by: Mark Kogan <mkogan@redhat.com>
2026-06-30 14:49:36 +03:00
Redouane Kachach
32dccfab54
Merge pull request #66051 from ShwetaBhosale1/fix_issue_73633_haproxy_service_should_not_be_restart_on_nfs_updates
haproxy service should not be restart on nfs updates

Reviewed-by: Redouane Kachach <rkachach@ibm.com>
2026-06-30 13:43:52 +02:00
Redouane Kachach
b9fdd67b58
Merge pull request #66663 from Shubhaj1810/fix-issue-2384421
mgr/nfs: include placement details and active/passive roles in cluster info

Reviewed-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-30 13:42:16 +02:00
Redouane Kachach
9eb9e306d5
Merge pull request #67195 from kginonredhat/fix-ipv6-lo-routes
cephadm: allow lo routes on loopback interface, allow bgp-routes for list-networks

Reviewed-by: Redouane Kachach <rkachach@ibm.com>
2026-06-30 13:40:29 +02:00
Redouane Kachach
cc3f8a69b3
Merge pull request #69043 from yaelazulay-redhat/issue_76565_keep_mgr_ports_in_sync_when_modules_are_enabled_disabled
cephadm: keep mgr ports in sync when modules are enabled/disabled

Reviewed-by: Redouane Kachach <rkachach@ibm.com>
2026-06-30 13:36:34 +02:00
Redouane Kachach
fbdf45a6e3
Merge pull request #69192 from kginonredhat/issue-76981-Rocky10-failure-cause-by-Cannot-register-NFS-V3-on-TCP
cephadm: start rpcbind before NFS-Ganesha for NFSv3 TCP

Reviewed-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-30 13:34:28 +02:00
Redouane Kachach
ebbb3e19f1
Merge pull request #69283 from Shubhaj1810/node-exporter
cephadm: report intentionally stopped daemons as stopped

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2026-06-30 13:31:24 +02:00
Ali Masarwa
720a3b7c3d rgw/posix: reading a detele marker returns -ENOENT
Signed-off-by: Ali Masarwa <amasarwa@redhat.com>
2026-06-30 13:48:31 +03:00
Nitzan Mordechai
2e46312e2e pybind/ceph_daemon: add 10s timeout to admin socket client
ceph daemon command can hangs indefinitely when the target
daemon's admin socket is unresponsive. do_sockio() creates socket with
no timeout, so sock.recv(4) blocks forever if the daemon is slow to respond,
where AdminSocket::init() creates the socket file before store->mount()
returns).

The AdminSocketClient already uses a 10-second SO_RCVTIMEO / SO_SNDTIMEO.
Align the Python client by calling sock.settimeout(timeout) before connecting.
The default is 10s to match the C++ side; callers can pass timeout=None
to restore the previous blocking behaviour.

Fixes: https://tracker.ceph.com/issues/77743
Signed-off-by: Nitzan Mordechai <nmordech@ibm.com>
2026-06-30 10:46:02 +00:00
pujaoshahu
fcac016ae4 mgr/dashboard: Add option should be enabled even if allow all host access is enabled in nvme/tcp
Fixes: https://tracker.ceph.com/issues/75509
Signed-off-by: pujaoshahu <pshahu@redhat.com>
2026-06-30 15:18:13 +05:30
Mohit Agrawal
68dc3eb81e core: Handle OP_TOUCH_TEMP in BlueStore without any guard(WITH_CRIMSON)
During backfill recovery on Crimson OSD replica, Bluestore
aborts with _txc_add_transaction error ENOENT on operation
due to OP_TOUCH_TEMP. OP_TOUCH_TEMP is guarded by #ifdef WITH_CRIMSON
but WITH_CRIMSON is not defined when BlueStore is compiled.

OP_TOUCH_TEMP is a crimson specific op to manage temporary
objects and the op is not used by classic osd. It is a plain
integer constant defined in the shared transaction layer so
adding it to the common code path in Transaction.h and BlueStore.cc
requires no additional includes or compile time guards.Handling
it unconditonally is therefore safe for all Objectstore backends.

Solution: Handle OP_TOUCH_TEMP without any guard

Fixes: https://tracker.ceph.com/issues/75957
Signed-off-by: Mohit Agrawal <moagrawa@redhat.com>
2026-06-30 15:01:12 +05:30
Kefu Chai
3201a3d130 crimson/seastore: skip LBA-leaf crc check for in-place delta-overwrite extents
on a cold read, check_full_extent_integrity() compares the crc of the read
content against the crc stored in the LBA leaf (pin_crc). this aborts on a
RANDOM_BLOCK data extent that was overwritten via delta-based overwrite:

  abort(extent checksum inconsistent)
   4# Cache::check_full_extent_integrity()
   5# Cache::_read_extent()::{lambda}
   7# Cache::replay_delta()  (or _read_pin on the live IO path)

the leaf crc is not a valid reference for these extents. delta overwrite
mutates the extent in place to avoid touching the LBA tree, so
update_lba_mappings() skips the leaf for is_exist_mutation_pending() extents
and the pre-mutation crc stays there. the new content lands on disk later and
out-of-band, when the cleaner rewrites the modified region in place
(RandomBlockOolWriter::do_write). that write is not atomic with the leaf, so a
cold read can pick up new content against an old leaf crc.

updating the leaf crc on overwrite is not the fix. it brings back the LBA-tree
mutation (and the txn conflicts and hot-path crc) that delta overwrite exists
to avoid, and it still does not make the leaf reliable: the data write and the
leaf update happen separately, so publishing leaf=new while disk still holds
old just moves the mismatch.

so stop trusting the leaf for these extents. their content is verified at
replay, where replay_delta() applies the deltas and
CircularBoundedJournal::replay() asserts last_committed_crc == delta.final_crc.
and the content on disk is always current: modified_region accumulates every
change via union_insert and is cleared only when the in-place rewrite commits,
so do_write writes the latest content; only the leaf crc lags.

drop the leaf-crc check on read for RANDOM_BLOCK in-place-rewritable (data)
extents, but only when delta-based overwrite is enabled. with it off (the
default), overwrites go through remapping, the leaf crc stays valid, and the
check still runs for every other extent. last_committed_crc is computed
regardless, so the delta-chain check is unaffected.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-30 15:48:14 +08:00
Leonid Chernin
4770952270 nvmeofgw: stretched cluster fix relocate
fixes: https://tracker.ceph.com/issues/77803

Signed-off-by: Leonid Chernin <leonidc@il.ibm.com>
2026-06-30 10:05:28 +03:00
Sun Yuechi
ceff64b3a9 blk/spdk: call spdk_env_opts_init() before setting pci options
spdk_env_opts_init() reinitializes the whole spdk_env_opts struct,
overwriting pci_allowed/pci_whitelist and num_pci_addr that were set
just above it. As a result the local PCI device restriction never took
effect. Initialize opts first, then fill in the PCI address fields.

Fixes: https://tracker.ceph.com/issues/77802
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-30 14:41:34 +08:00
Xuehan Xu
fd808e8453
Merge pull request #69301 from xxhdx1985126/wip-seastore-circular_bounded_journal-tail-fix
crimson/os/seastore/journal/circular_bounded_journal: find the real journal tail on startup

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Myoungwon Oh <ohmyoungwon@gmail.com>
2026-06-30 13:58:38 +08:00
Leonid Chernin
d5e254c631 nvmeofgw: arm beacon timeouts for GWs in Created state
Signed-off-by: Leonid Chernin <leonidc@il.ibm.com>
2026-06-30 08:50:39 +03:00
Sun Yuechi
832d6122e9 blk/aio: fix mismatched parenthesis in POSIX AIO submit path
The condition `if ((cur->n_aiocb == 1) {` has an unbalanced
parenthesis and fails to compile. This code is only built on the
HAVE_POSIXAIO (BSD) path, so the breakage is normally hidden.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-29 21:07:56 -07:00
Xuehan Xu
c2a69f1663
Merge pull request #69615 from xxhdx1985126/wip-77548
crimson/osd/pg_backend: mark the omaps of the object dirty when removing omap keys

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2026-06-30 10:38:19 +08:00
Brad Hubbard
54de613cdd
Merge pull request #69278 from badone/wip-badone-tracker-76473-dont-inline-and-log-mem0ry-leak
common: Log when we leak and disallow inlining

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2026-06-30 11:26:16 +10:00
Kefu Chai
239be57c06
Merge pull request #69437 from tchaikov/wip-crimson-osd-coroutinize
crimson/osd: coroutinize and cleanups

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2026-06-30 08:18:33 +08:00
Kefu Chai
5220368596
Merge pull request #69537 from tchaikov/wip-crimson-compile-time-fmt-check
crimson: fix malformed logging message and enable compile-time fmt check

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2026-06-30 08:18:18 +08:00
Afreen Misbah
44b78ff004
Merge pull request #69070 from rhcs-dashboard/73367-carbonize-administration-module
mgr/dashboard : Carbonize configuration form

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Devika Babrekar <devika.babrekar@ibm.com>
2026-06-30 01:35:54 +05:30
Alex Ainscow
d46aea099e test/osd: Add scrub to osd test fixtures
Add scrubbing functionality to the OSD test fixtures to enable testing
of scrub behavior on erasure-coded pools. This implementation uses as
much of the scrub backend as possible, but avoids using pg_scrubber or
the interaction with peering due to the complexity of the dependency on
PG.

Changes:
- Add scrub_object() method to PGBackendTestFixture for testing scrub
  functionality on EC pools
- Add create_random_buffer() utility for generating random test data
- Add corrupt_shard_data() method to intentionally corrupt shard data
  for testing scrub detection
- Add MockScrubBeListener, MockPgScrubBeListener, MockSnapMapReader, and
  MockLoggerSinkSet mock classes to support scrub testing
- Add three new test cases: ScrubClean, ScrubDetectsCorruption, and
  ScrubPartialWrite to verify scrub behavior with clean data, corrupted
  data, and partial writes

Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>

# Conflicts:
#	src/test/osd/TestECFailoverWithPeering.cc
2026-06-29 16:27:15 +01:00
Alex Ainscow
bcc26ce6f2 test/osd: Fix event loop context violations in test fixtures
Refactored test fixture code to properly execute object context operations
within event loop context, fixing ceph_assert(osd != -1) failures.

Key changes:
- Split functions calling get_object_context() into implementation and
  wrapper functions that schedule work on the primary OSD via event loop
- Added accessor functions (set_object_context, clear_object_contexts)
  to encapsulate per-OSD object_contexts map access
- Fixed lambda capture bug in PGBackendTestFixture::update_osdmap()
- Refactored run_parallel_recovery_and_verify_callbacks() to execute
  within proper event loop context

All object context operations now execute within the correct OSD context,
with get_object_context() as the single point querying the current OSD.

Added test ZeroSizeObjectWithAttributesRecovery to verify recovery of
zero-size objects with attributes after primary failover, ensuring
attributes are properly recovered on the failed shard.

Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>
Assisted-by: IBM-Bob:Claude-Sonnet

# Conflicts:
#	src/test/osd/ECPeeringTestFixture.cc
2026-06-29 16:20:13 +01:00
Sun Yuechi
feffdffb8e crimson/test: shrink messenger-thrash max_message_len to 1MB
The test keeps random_num payload buffers (each up to max_message_len)
resident and sends copies concurrently; at 4MB the peak working set
runs ~600MB and overflows the test's 256M --memory arena with bad_alloc.

--memory only binds when seastar's own allocator is compiled in
(RelWithDebInfo/Release); Debug/Sanitize define
SEASTAR_DEFAULT_ALLOCATOR and use the heap, so the limit never bites
there.

Shrink max_message_len to 1MB, cutting the peak below 160MB so it fits
the 256M arena instead of enlarging it.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-29 08:02:29 -07:00
Ali Masarwa
2e892f3c5c rgw/posix: fix delete markers
Signed-off-by: Ali Masarwa <amasarwa@redhat.com>
2026-06-29 16:09:31 +03:00
Nithya Balachandran
ce787df56b rgw/posix: fix delete markers
Creates 0 byte files for delete markers for versioned buckets.

Signed-off-by: Nithya Balachandran <nithya.balachandran@ibm.com>
2026-06-29 16:09:31 +03:00
Nizamudeen A
7163b736cd
Merge pull request #69487 from rhcs-dashboard/fix-4878
mgr/dashboard : Implement dependent resource handling for gateway Deletion

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Sagar Gopale <sagar.gopale@ibm.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-06-29 18:03:12 +05:30
Redouane Kachach
0a528685d5
mgr/cephadm: bump Python version path mapped in kcli dev env
Bump Python version path mapped in kcli dev env from 3.9 to 3.12

Signed-off-by: Redouane Kachach <rkachach@ibm.com>
2026-06-29 14:20:22 +02:00
Naman Munet
c0e9ddabe5 mgr/dashboard: persist table selection across server side page changes
Fixes: https://tracker.ceph.com/issues/77651

Signed-off-by: Naman Munet <naman.munet@ibm.com>
2026-06-29 15:27:47 +05:30
Kefu Chai
b34429657e
Merge pull request #69794 from tchaikov/wip-crimson-test-thrash-teardown
crimson/test: stop the messenger-thrash config on every exit path

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2026-06-29 16:32:57 +08:00
Kefu Chai
66547e6c82
Merge pull request #69755 from tchaikov/wip-mgr-remote-pyunicode-leak
mgr: fix PyObject leak in the remote() pickle calls

Reviewed-by: Matthew N. Heler <matthew.heler@hotmail.com>
2026-06-29 16:29:57 +08:00
Nizamudeen A
748cbef07b
Merge pull request #69413 from rhcs-dashboard/fix-3958-autofetch
mgr/dashboard: Add auto-fetch listeners option in subsystem page

Reviewed-by: Sagar Gopale <sagar.gopale@ibm.com>
Reviewed-by: Naman Munet <nmunet@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-06-29 13:52:34 +05:30
Nizamudeen A
40a7245065
Merge pull request #69550 from rhcs-dashboard/fix-5059
mgr/dashboard: "Any" initiator entry appears in the Initiator table after selecting "All Hosts Allowed" during subsystem creation

Reviewed-by: Naman Munet <nmunet@redhat.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-06-29 13:37:10 +05:30
Nizamudeen A
4a7c4fb49c
Merge pull request #69424 from rhcs-dashboard/fix-4075
mgr/dashboard: NVME- Remove block pool from create gateway page

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Sagar Gopale <sagar.gopale@ibm.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2026-06-29 13:36:27 +05:30
NitzanMordhai
dd5cbcdf5c
Merge pull request #69675 from NitzanMordhai/wip-nitzan-config-trim-key-trail-lead-spaces
mon/config: trim whitespace in config target
2026-06-29 11:02:24 +03:00
Xuehan Xu
69cd81f147 crimson/os/seastore/journal/circular_bounded_journal: find the real
journal tail on startup

Recover the correct journal tail during replay by scanning
JOURNAL_TAIL records instead of relying solely on the on-disk
header, which may be stale if a crash occurs between record
write and header update.

The journal header is stored *separately* from the (append-only)
journal and is not updated atomically with journal writes,
so it may lag behind after a crash.
Treat the journal as the source of truth and rebuild the tail from persisted records.

Replay is reorganized into three passes:
1) discover latest journal tail
2) rebuild allocation map
3) replay deltas

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-06-29 10:24:57 +08:00
Kefu Chai
1ccb756182 qa/cephfs: fix invalid escape sequence SyntaxWarnings
python 3.12 warns on unrecognized escape sequences in string literals.
fuse_mount's admin-socket pyscript and mount's nft payload carry regex
and shell escapes (\., \d, \;) that are meant literally, so make those
strings raw. filesystem's get_mds_addr docstring had a stray \/ in the
example address, which is just a slash, so drop the backslash.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-29 09:36:12 +08:00
Kefu Chai
68ec2a1b3f crimson/test: stop the messenger-thrash config on every exit path
test_messenger_thrash put sharded_conf().stop() in the success
continuation, so an exception from the tests skipped it and left the
static sharded<ConfigProxy> holding live instances. its destructor then
asserts. a tester hit this when the run went OOM at --memory 256M:

  ERROR [shard 0:main] test - Test failed: got exception
    ceph::buffer::v15_2_0::bad_alloc (Bad allocation [buffer:1])
  /ceph/src/seastar/include/seastar/core/sharded.hh:573:
    seastar::sharded<Service>::~sharded()
    [with Service = crimson::common::ConfigProxy]:
    Assertion `_instances.empty()' failed.
  terminate called without an active exception

run the body in seastar::async and stop the config through
seastar::deferred_stop, as crimson/osd/main.cc already does. the guard
is constructed only after start() succeeds, and its destructor stops the
config on any exit path, so a failing test exits cleanly instead of
aborting.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-29 09:22:45 +08:00
Mykola Golub
a4b7a658c1 mon: allow getpoolstats for mgr modules
Fixes: https://tracker.ceph.com/issues/74490
Signed-off-by: Mykola Golub <mykola.golub@clyso.com>
2026-06-28 20:45:50 +03:00
Ronen Friedman
ebc18499af cmake: add WITH_MOLD option for Mold linker support
Add a WITH_MOLD cmake option that auto-configures the build for
the Mold linker. When enabled, cmake finds mold, sets CMAKE_LINKER,
injects -fuse-ld=mold into all linker flag variables, and sets
USING_MOLD_LINKER which gates the Mold-specific workarounds added
in the previous commit.

Usage: cmake -DWITH_MOLD=ON ...

Assisted-by: Claude <claude.ai>
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2026-06-28 14:53:54 +00:00
Ronen Friedman
e552441619 build: add Mold linker compatibility
Mold handles several linker features differently from ld.bfd:

- --exclude-libs,ALL hides .symver-aliased symbols (e.g. rados_*)
  even when the version script lists them as global. Detect Mold
  (via USING_MOLD_LINKER) and disable --exclude-libs; version
  scripts already control symbol visibility.

- Duplicate symbols across static archives are treated as errors.
  Merge crimson-alien-common into crimson-alienstore as a single
  archive. For crimson-osd, link crimson-alienstore before
  crimson-common with --allow-multiple-definition so the alien
  thread's non-crimson definitions win (Mold picks first definition).

- Python/Cython extension builds (via Distutils.cmake) invoked the
  system default linker. Pass -fuse-ld= via MOLD_FUSE_LD_FLAG so
  extensions link with the configured linker.

- Add rados_* and _rados_* to librados.map global section for Mold
  compatibility with .symver aliases.

Co-authored-by: Mark Kogan <mkogan@redhat.com>
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2026-06-28 14:53:54 +00:00
Yael Azulay
1966b0c917 cephadm: keep mgr ports in sync when modules are enabled/disabled
ceph orch ps  - was showing stale or incorrect port information for `mgr` daemons after enabling or disabling mgr modules (e.g., dashboard, prometheus).
This fix ensures the displayed ports always reflect the actual active module endpoints.

Code Fixes:
-----------
Changes in src/pybind/mgr/cephadm/services/cephadmservice.py:
- Extract and refactor _get_mgr_service_ports:
	The logic for reading active mgr module ports was inline inside prepare_create, using check_mon_command({'prefix': 'mgr services'}). It was extracted into a static helper method and refactored to read directly from mgr.get('mgr_map')['services'] instead -- which is faster and consistent with how get_dependencies reads the same data. The method is now shared by both prepare_create (existing use) and get_dependencies (new use).

- Extract `_get_mgr_service_ports`:
	The logic for reading active mgr module ports from `mgr_map['services']` was inline inside `prepare_create`. It was extracted into a static helper method so it can be reused by both `prepare_create` (existing use) and `get_dependencies` (new use). Returns the port numbers currently registered by active mgr modules (e.g., `[8443, 9283]`).

-  Override `get_dependencies` in `MgrService`:
	The base class returns `[]` for all Ceph daemons. The serve loop compares `get_dependencies()` output against `last_deps` (saved after the previous reconfig). If they differ, a reconfig is triggered. By overriding this method in `MgrService`, we make the serve loop aware of mgr port changes. Returns a sorted list such as `['port:8443', 'port:9283', 'sd_port:8765']`.

- Override `generate_config` in `MgrService`:
	After a reconfig completes, the result of `generate_config` is saved as `last_deps`. The inherited implementation returns `[]` as deps. If `[]` is saved but `get_dependencies` returns `['port:8443', 'sd_port:8765']`, they would always differ, causing an infinite reconfig loop. This override ensures that what is saved as `last_deps` matches what `get_dependencies` will return in the next iteration.

Changes in `src/pybind/mgr/cephadm/serve.py`:

- Update mgr cache on reconfig:
	The existing logic skipped the cache update for all Ceph daemon reconfigs. A new `elif` branch for mgr reconfigs fetches the existing cached `DaemonDescription`, updates only its `ports` field, and saves it back. This makes `ceph orch ps` reflect the correct ports immediately after a reconfig, without showing a misleading "starting" status.

Changes in `src/cephadm/cephadm.py`:

- Update `unit.meta` on reconfig:
	`unit.meta` is a per-daemon file on the host that stores metadata including ports.  It is read periodically by `cephadm ls` and fed back into the orchestrator cache. Previously it was only written on initial deployment, so every periodic cache refresh would overwrite the corrected ports with stale data from `unit.meta`. This change updates the `ports` field in `unit.meta` during reconfig, ensuring the cache refresh confirms the correct state rather than reverting it.

Changes to src/pybind/mgr/cephadm/tests/services/test_mgr.py:

- Updated _prepare helper:
	The original mocked check_mon_command to feed services data to prepare_create. Since our _get_mgr_service_ports now reads from mgr.get('mgr_map') instead of check_mon_command, the mock was replaced with mock_store_set('_ceph_get', 'mgr_map', ...) which sets the data in the right place.

- Added test_get_dependencies_changes_when_module_enabled:
   A new test that verifies get_dependencies returns different dep strings before and after a module is enabled. This directly tests the core sync mechanism -- the serve loop compares get_dependencies output against last_deps to detect port changes and trigger a reconfig.

to run new tests:
cd src/pybind/mgr
tox -e py3 -- cephadm/tests/services/test_mgr.py -v

Fixes: https://tracker.ceph.com/issues/76565
Signed-off-by: Yael Azulay <yazulay@redhat.com>
2026-06-28 15:03:03 +03:00
Nitzan Mordechai
0bac6af3c1 mgr/ClusterState: remove debug 30 memory spike
Remove the full pg_map JSON dump at dout(30) which was flooding the
log queue with multi-MB entries on every tick. The much smaller
incremental dump is retained for debugging.

Fixes: https://tracker.ceph.com/issues/64082
Signed-off-by: Nitzan Mordechai <nmordech@ibm.com>
2026-06-28 10:54:42 +00:00
Xuehan Xu
8e66072ec5 crimson/osd/pg_backend: mark the omaps of the object dirty when removing
omap keys

Fixes: https://tracker.ceph.com/issues/77548
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-06-28 16:33:11 +08:00
Xuehan Xu
1605266614
Merge pull request #69617 from xxhdx1985126/wip-77549
crimson/os/seastore/random_block_manager/avlallocator: avoid directly modifying avl_set elements

Reviewed-by: Myoungwon Oh <ohmyoungwon@gmail.com>
2026-06-28 14:33:03 +08:00
NitzanMordhai
af060383aa
Merge pull request #69456 from NitzanMordhai/wip-nitzan-mgr-signals-standby-handle
mgr: handle SIGTERM/SIGINT in standby mgr to avoid CEPHADM_FAILED_DAEMON
2026-06-28 09:12:01 +03:00
Kefu Chai
7f07e04922
Merge pull request #69618 from tchaikov/wip-mgr-exclude-tests-ci
cmake,deb,rpm: exclude CI and test directories from installed modules

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2026-06-28 07:04:48 +08:00
Kefu Chai
374bb8db28
Merge pull request #69453 from tchaikov/wip-dashboard-fix-e2e
mgr/dashboard: fix "welcome" page reference in add-host e2e feature

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-06-28 06:32:09 +08:00
Kefu Chai
336f93d450
Merge pull request #68836 from nh2/balancer-docs-average
docs: balancer: Explain PG shard count based balancing in more detail

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-06-27 21:24:38 +08:00
Sun Yuechi
03cf6f5f00 osd/scheduler: add missing breaks in PGRecoveryMsg::run
Without these breaks the switch fell through, recording a single
message's latency into every following per-type counter.

Fixes: https://tracker.ceph.com/issues/77751
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-27 21:08:40 +08:00
Niklas Hambüchen
8f83ec546a docs: balancer: Explain limitations of PG shard count balancing
Signed-off-by: Niklas Hambüchen <mail@nh2.me>
2026-06-27 11:50:36 +00:00
Niklas Hambüchen
a62a982aac docs: balancer: Elaborate what "average" means.
The wording "average" so far suggested a simple mean count, which it isn't.

It is important to document this, so that the user can follow
why the balancer is active or not.

Source for the logic (deviation computation and threshold check)
in `src/osd/OSDMap.cc`:

`calc_deviations()` - computes per-OSD deviation:

    float target = osd_weight.at(oid) * pgs_per_weight;
    float deviation = (float)opgs.size() - target;

Where `pgs_per_weight` is computed in `calc_pg_upmaps()`:

    float pgs_per_weight = total_pgs / osd_weight_total;

This also has the early threshold + early return:

    if (cur_max_deviation <= max_deviation) {
        ldout(cct, 10) << __func__ << " distribution is almost perfect" << dendl;
        return 0;
    }

`fill_overfull_underfull()` classifies OSDs using `max_deviation`:

    if (odev > max_deviation) {
        overfull.insert(oid);        // deviation > +5 -> source candidate
    }
    if (odev < -(int)max_deviation) {
        underfull.push_back(oid);    // deviation < -5 -> target candidate
    }

Signed-off-by: Niklas Hambüchen <mail@nh2.me>
2026-06-27 11:50:36 +00:00
Xuehan Xu
bfc7662a50
Merge pull request #69568 from xxhdx1985126/wip-77485
crimson/os/seastore/transaction_manager: refresh `indirect_cursor` after the `direct_cursor` is removed in `TM::_remove()`

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2026-06-27 15:44:20 +08:00
Sun Yuechi
cd8532314c test/osd: cover clear_omap tombstoning of cached keys
Add a regression test that processes an insert so the keys are cached
in key_map, then processes a later clear_omap and asserts the cached
keys are returned as tombstones.

This would have caught the by-value bind in process_entries, where
clear_omap mutated copies and left already-cached keys untouched.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-27 15:44:06 +08:00
Sun Yuechi
aa82faec86 osd/ECOmapJournal: bind by reference when clearing omap keys
The clear_omap loop bound each key_map entry by value, so
update_value() mutated a copy and cached keys were never marked
removed.

As a result, clear_omap had no effect on already-cached keys: cleared
omap keys retained stale values and were returned on read.

Introduced-by: 7289e4d2ec ("osd: Add ECOmapJournal class and relocate OmapUpdateType enum class")
Fixes: https://tracker.ceph.com/issues/77750
Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-27 15:44:06 +08:00
Kefu Chai
6bba648ac7
Merge pull request #68792 from MaxKellermann/nvmeof__missing_includes
nvmeof: add missing includes

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-06-27 14:13:23 +08:00
mheler
c0f5b1c9ff
Merge pull request #69528 from mheler/wip-bug-77278
rgw/lc: warn against changing rgw_lc_max_objs once lifecycle is in use
2026-06-27 00:50:33 -05:00
Kefu Chai
4e5fd9ae8b
Merge pull request #69699 from avanthakkar/fix-mgr-recoverdb
mgr: fix MgrModuleRecoverDB to honor its retry budget on db cleanup failure

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-06-27 08:44:47 +08:00
SrinivasaBharathKanta
ee4ac7fb8e
Merge pull request #67255 from Vivek-tgmc/wip-71716-setting_nodown_flag_reporting_issue_in_ceph_status_and_health
osd: Rephrase unclear OSD_FLAGS warning message
2026-06-27 05:05:28 +05:30
Kefu Chai
e59ab34666 qa/suites/rados/mgr: ignorelist MGR_INFLUX_DB_LIST_FAILED in module_selftest
module_selftest fails on a cluster log warning that is not ignorelisted:

  cluster [WRN] Health check failed: Failed to list/create InfluxDB database (MGR_INFLUX_DB_LIST_FAILED)

module_selftest is the only job that enables the influx module, and no suite
provisions an InfluxDB server, so once enabled the module cannot reach a backend
and raises a health warning: MGR_INFLUX_NO_SERVER when no hostname is set,
MGR_INFLUX_DB_LIST_FAILED from serve() once one is.  test_influx sets a hostname,
so it hits the latter.

This surfaced only after the test image started shipping the influxdb python
module.  Until then can_run() returned false, the module never enabled, and the
sole log line was "influxdb python module not found", which is already
ignorelisted.  self_test() cannot run without enabling the module, so ignorelist
the warning it raises.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-27 07:26:02 +08:00
Kefu Chai
b16654474e
Merge pull request #69745 from sunyuechi/wip-fio-riscv-upstream
cmake: upgrade to upstream fio

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-06-27 06:33:09 +08:00
Laura Flores
df4e012d0f
Merge pull request #69663 from aainscow/wip-truncate-write-io-sequence
osd/ECTransaction: fix truncate+write planning for EC shard sizes

Reviewed-by: Radoslaw Zarzynski <Radoslaw.Adam.Zarzynski@ibm.com>
2026-06-26 17:02:33 -05:00
Adam Emerson
7831477bff
Merge pull request #68912 from adamemerson/wip-cls-timeindex-build
build: Add `cls_timeindex_types` to `cls_timeindex` objclass

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
Reviewed-by: Jesse F. Williamson <jfw@ibm.com>
2026-06-26 17:52:44 -04:00
Matthew N. Heler
899457aeca rgw/http: skip pause/resume when the request already finished
req_data->mgr is null once the request finishes. Check it in
_set_write_paused and _set_read_paused before calling set_request_state,
otherwise a late streaming write segfaults.

Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
2026-06-26 15:05:36 -05:00
pujashahu
d83f5dcdb9 mgr/dashboard: "Any" initiator entry appears in the Initiator table after selecting "All Hosts Allowed" during subsystem creation
Fixes: https://tracker.ceph.com/issues/77467

Signed-off-by: pujaoshahu <pshahu@redhat.com>
2026-06-27 00:34:48 +05:30
pujashahu
d890779f75 mgr/dashboard: Add auto-fetch listeners option in subsystem page
Fixes: https://tracker.ceph.com/issues/77083

Signed-off-by: pujaoshahu <pshahu@redhat.com>
2026-06-27 00:32:30 +05:30
Afreen Misbah
4b250f2b05
Merge pull request #69426 from rhcs-dashboard/fix-77361
mgr/dashboard: NVMe-oF – Updated subtitle text on the Create Gateway page

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
2026-06-26 22:37:32 +05:30
Casey Bodley
fa4c6b2a9c
Merge pull request #69768 from MaxKellermann/rgw__missing_includes
rgw: add missing include

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2026-06-26 12:58:06 -04:00
Ilya Dryomov
9b26437ea3
Merge pull request #69769 from idryomov/wip-ptl-tool-multiple-source-prs
script/ptl-tool: fix redmine_linkage_correct case in _get_active_issues()

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2026-06-26 16:42:07 +02:00
Kefu Chai
6a8a0a51b2
Merge pull request #69660 from tchaikov/wip-cmake-boost-with-asan
cmake/boost: build libboost_context with BOOST_USE_ASAN

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2026-06-26 22:00:46 +08:00
John Mulligan
c43103e42d
Merge pull request #69103 from Sodani/shsodani_ceph_rgw_new_module
mgr/smb: add RGW-backed SMB share support

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2026-06-26 09:40:04 -04:00
Jaya Prakash
f3dcff54f5
Merge pull request #69598 from aclamk/aclamk-doc-fcm-plugin
doc/rados/bluestore: ExtBlkDev, FCM plugin

Reviewed-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
Reviewed-by: Jaya Prakash <jayaprakash@ibm.com>
2026-06-26 17:18:40 +05:30
Sun Yuechi
7c1e977365 osd: fix osd_reqid_t comparison strict weak ordering
The 'l.inc < r.inc' term in operator< lacked an 'l.name == r.name' guard,
breaking antisymmetry, so it was not a valid strict weak ordering.
Replace it with std::tie.

The only ordered container keyed on osd_reqid_t is
ECInject::write_failures0_reqid (QA/debug error injection), so no
production code path is affected and no backport is needed.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-26 19:42:06 +08:00
Kefu Chai
71943ead03
Merge pull request #69605 from sunyuechi/wip-readable-sliding-window
test/encoding: run readable.sh with a sliding window of jobs

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-06-26 19:26:10 +08:00
Ilya Dryomov
c82a83fc96 script/ptl-tool: fix redmine_linkage_correct case in _get_active_issues()
If redmine_linkage_correct is True, _get_active_issues() is supposed to
ignore issues with "Multiple Source PRs" category.  However that doesn't
happen because the category string is compared against (category, text)
pair.

Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
2026-06-26 13:07:31 +02:00
Max Kellermann
13e7fd758a rgw: add missing include
Signed-off-by: Max Kellermann <max.kellermann@ionos.com>
2026-06-26 12:28:45 +02:00
pujashahu
1c65a8afa5 mgr/dashboard: NVME- Remove block pool from create gateway page
Adds unit test for creating gateway service with encryption when enabled.

Fixes: https://tracker.ceph.com/issues/77359

Signed-off-by: pujashahu <pshahu@redhat.com>
2026-06-26 15:39:58 +05:30
pujashahu
4c790d72da mgr/dashboard : Implement dependent resource handling for gateway Deletion
Fixes: https://tracker.ceph.com/issues/77403

Signed-off-by: pujaoshahu <pshahu@redhat.com>
2026-06-26 15:26:37 +05:30
pujashahu
90e0541ec2 mgr/dashboard: NVMe-oF – Updated subtitle text on the Create Gateway page
Fixes :https://tracker.ceph.com/issues/77361

Signed-off-by: pujaoshahu <pshahu@redhat.com>
2026-06-26 14:50:05 +05:30
Ilya Dryomov
a9ab67306b
Merge pull request #69008 from MaxKellermann/tools__missing_includes
tools: add missing includes

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
Reviewed-by: Venky Shankar <vshankar@redhat.com>
2026-06-26 10:04:15 +02:00
Gil Bregman
55bf46e661
Merge pull request #69736 from gbregman/main
nvmeof: Change the NVMEOF image version to 1.9
2026-06-26 10:18:04 +03:00
Sun Yuechi
216d468a1a cmake: build fio from upstream 3.42
fio-3.42 is the latest stable release. We switched to the ceph/fio fork in
10baab3fc8 to pick up a clang build fix; that
fix has since landed upstream, so switch back. This also gives riscv64
proper arch support (arch-riscv64.h, added in fio-3.36), which the fork's
fio-3.27-cxx lacks.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-26 15:17:17 +08:00
Kefu Chai
9b7d761e84
Merge pull request #69510 from tchaikov/wip-cryptotools-sans-pyopenssl
python-common/cryptotools: reimplement on top of cryptography

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2026-06-26 14:29:38 +08:00
Kefu Chai
8ae44f6a37 mgr: fix PyObject leak in the remote() pickle calls
dispatch_remote() and ceph_dispatch_remote() invoke pickle.loads/dumps via
PyObject_CallMethodObjArgs(pmodule, PyUnicode_FromString("loads"), ...). The
method-name string created by PyUnicode_FromString is a new reference that
CallMethodObjArgs does not steal, and it was never released, so every cross
-module remote() call leaked three PyUnicode objects on the active mgr.

Use PyObject_CallMethod(pmodule, "loads", "(O)", arg), which takes the method
name as a C string and manages it internally, so there is nothing to leak.
The "(O)" format wraps the argument in a one-tuple so it reaches the callee as
a single object. A bare "O" passes the argument through directly, and when it
is a tuple (as remote()'s *args always is, including the empty tuple from a
no-argument call) PyObject_CallMethod unpacks it and calls dumps() with the
wrong arguments, e.g. dumps() with none.

Fixes: https://tracker.ceph.com/issues/77724
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-26 12:29:21 +08:00
Max Kellermann
b992ef63d1 tools: add missing includes
Signed-off-by: Max Kellermann <max.kellermann@ionos.com>
2026-06-25 21:42:29 +02:00
Ilya Dryomov
96e268b198
Merge pull request #69620 from tchaikov/wip-rbd-fix-uaf
librbd: fix use-after-free releasing object map locks in deep copy

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2026-06-25 20:08:52 +02:00
Yuri Weinstein
906cf6a7a6
Merge pull request #69140 from MaxKellermann/CompatSet_optimizations
CompatSet optimizations

Reviewed-by: Kefu Chai <tchaikov@gmail.com>
2026-06-25 10:24:00 -07:00
Yuri Weinstein
67c1027017
Merge pull request #68738 from aainscow/dout-hacks
test/osd: Improve test logging with OSD identification

Reviewed-by: Bill Scales <bill_scales@uk.ibm.com>
2026-06-25 10:23:20 -07:00
Avan Thakkar
8916aa85f7 mgr/smb: add RGW credentials resource and store in priv_stor
Refactored RGW credential management to use dedicated RGWCredential
resources instead of embedding credentials directly in Share resources.
This will help to reuse the same credential across multiple shares using
same user.

RGW credentials must not appear in the public RADOS store, which any
client with pool caps can read. Instead, write a config:merge stub
containing only the credential fields to the private mon config-key
store, and pass its URI to the container alongside the primary config
URI via extra_config_uris.

- Public config: ceph_rgw:access_key / secret_access_key are empty strings
- Private stub: stored under smb/config/<cluster_id>/config.smb.rgw,
  contains only the real credential values via config:merge
- extra_config_uris: new Config field carrying supplementary URIs
  appended to SAMBACC_CONFIG after the primary config_uri

Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Signed-off-by: Shweta Sodani <ssodani@redhat.com>
2026-06-25 22:27:52 +05:30
Shweta Sodani
847dc5cc8c doc/mgr/smb: document new options for smb share create rgw
Signed-off-by: Shweta Sodani <ssodani@redhat.com>
2026-06-25 21:49:45 +05:30
Shweta Sodani
9a80169005 mgr/smb: Add test coverage for RGW share functionality
Add test cases for RGW-backed SMB shares

Key changes:
- RGWStorage creation, validation, and serialization
- Edge cases: None values, mutual exclusion

Signed-off-by: Shweta Sodani <ssodani@redhat.com>
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2026-06-25 21:49:45 +05:30
Shweta Sodani
d317e31859 mgr/smb: Update handler to process RGW shares, added validation and error handling
- Add _generate_rgw_share() function for RGW share configuration
- Skip CephFS authorization for RGW shares in cluster assembly
- Configure Samba VFS with RGW-specific parameters:
  - rgw_bucket, rgw_user_id, rgw_access_key, rgw_secret_key
- Validate bucket accessibility using radosgw-admin
- Ensure proper error handling in share creation flow

This enables the handler to generate proper Samba configuration for
RGW-backed shares along with validation and error handling.

Signed-off-by: Shweta Sodani <ssodani@redhat.com>
2026-06-25 21:49:45 +05:30
Shweta Sodani
6a3f3103a8 mgr/smb: Add CLI command for RGW share creation
- Add 'smb share create rgw' CLI command
- Import rgw module for credential management
- Fetch RGW credentials using radosgw-admin
- Create Share resource with RGWStorage backend
- Handle errors from credential fetching

This enables users to create SMB shares backed by RGW buckets,
similar to 'nfs export create rgw' functionality.

Example usage:
  ceph smb share create rgw mysmb share1 mybucket

Signed-off-by: Shweta Sodani <ssodani@redhat.com>
2026-06-25 21:49:45 +05:30
Shweta Sodani
9b4fbb3f22 mgr/smb: Add RGW credential management utilities
- Add rgw.py module for RGW integration
- Implement fetch_rgw_credentials() to get user credentials from radosgw-admin
- Implement validate_rgw_bucket() to check bucket existence
- Handle bucket owner lookup when user_id not provided
- Add proper error handling and logging

This provides the foundation for RGW authentication in SMB shares,
similar to the NFS RGW export implementation.

Signed-off-by: Shweta Sodani <ssodani@redhat.com>
2026-06-25 21:49:45 +05:30
Shweta Sodani
6da208dec1 mgr/smb: Add RGWStorage resource class and update Share to support RGW
- Add RGWStorage component class for RGW bucket storage backend
- Include fields: bucket, user_id, access_key_id, secret_access_key
- Add password conversion support for credentials
- Update Share class to accept optional rgw field and added convert
  defination
- Update Share validation to support both cephfs and rgw backends
- Ensure exactly one storage backend (cephfs or rgw) is specified

This is the first step in implementing RGW storage support for SMB
shares, similar to the existing NFS RGW export functionality.

Signed-off-by: Shweta Sodani <ssodani@redhat.com>
2026-06-25 21:49:45 +05:30
Adam Emerson
eaf9d8c881
Merge pull request #57167 from mkogan1/wip-rgw-thread-pool-size
rgw: reduce default thread pool size

Reviewed-by: Mark Nelson <mark.a.nelson@gmail.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2026-06-25 12:08:16 -04:00
Casey Bodley
1e020ca1c0
Merge pull request #69314 from smanjara/wip-76725-test
common/async: wake async_cond waiters on their associated executor

Reviewed-by: Mark Kogan <mkogan@redhat.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2026-06-25 10:51:28 -04:00
Gil Bregman
1874a4410c nvmeof: Change the NVMEOF image version to 1.9
Fixes: https://tracker.ceph.com/issues/77709

Signed-off-by: Gil Bregman <gbregman@il.ibm.com>
2026-06-25 17:33:21 +03:00
J. Eric Ivancich
6e552527fc
Merge pull request #68675 from smanjara/wip-archive-bucket-timestamp
rgw/multisite: record the bucket creation time at archive zone

Reviewed-by: Matt Benjamin <mbenjamin@redhat.com>
2026-06-25 10:27:29 -04:00
J. Eric Ivancich
f25ead132b
Merge pull request #69084 from linuxbox2/wip-76790
rgwlc: fix a likely null dereference in LCObjsLister::get_obj()

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2026-06-25 10:09:27 -04:00
Nitzan Mordechai
948de224df mgr: fix dispatch throttle bottleneck causing OSD connection timeouts
The mgr dispatch throttle (ms_dispatch_throttle_bytes, default 100MB)
was far smaller than the OSD throttle (mgr_osd_bytes, 512MB),
making the dispatch queue the effective bottleneck on large clusters.

When the dispatch queue fills, OSD connections stall waiting for
throttle space. These connections time out and reconnect, causing
connection churn and heap growth from repeated connection setup/teardown.
The MGR also falls behind on PG stats processing, leading to unknown
and inactive PGs.

Add mgr_dispatch_throttle_bytes (default 1GB) and a
Messenger::set_dispatch_throttle_size() API to configure it at init time.
The new default is sized to accommodate the sum of all per-type policy
throttles, ensuring the dispatch queue is never the bottleneck.

Fixes: https://tracker.ceph.com/issues/66310
Signed-off-by: Nitzan Mordechai <nmordech@ibm.com>
2026-06-25 11:23:08 +00:00
Kefu Chai
b8098d8844
Merge pull request #69449 from sunyuechi/wip-protobuf-config-mode
cmake: find Protobuf via cmake config before module mode

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-06-25 19:05:47 +08:00
Jaya Prakash
1004a24eca os/bluestore: measure Bitmap allocator lock wait and processing latency
Fixes: https://tracker.ceph.com/issues/76936

Signed-off-by: Jaya Prakash <jayaprakash@ibm.com>
2026-06-25 10:42:33 +00:00
Jaya Prakash
3c1dacfb5a os/bluestore: measure allocator lock wait and processing latency
Fixes: https://tracker.ceph.com/issues/76936

Signed-off-by: Jaya Prakash <jayaprakash@ibm.com>
2026-06-25 10:42:33 +00:00
Jaya Prakash
1a0ee9b11d os/bluestore: add per-instance new allocator perf counters
Fixes: https://tracker.ceph.com/issues/76936

Signed-off-by: Jaya Prakash <jayaprakash@ibm.com>
2026-06-25 10:42:25 +00:00
Ilya Dryomov
35eb60117b
Merge pull request #68500 from abitdrag/wip-non-prim-demote-img-snap-removal
rbd-mirror: Remove old non-primary demoted image snapshots on the local cluster

Reviewed-by: VinayBhaskar-V <vvarada@redhat.com>
Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2026-06-25 12:33:51 +02:00
Kefu Chai
14deb05b8e
Merge pull request #69658 from sunyuechi/test_encoding
test/encoding: disable LeakSanitizer in readable.sh and check-generated.sh

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-06-25 16:46:19 +08:00
Kefu Chai
86c262941b
Merge pull request #69674 from sunyuechi/wip-bufferlist-asan
test/bufferlist: fix BenchDeref overrun and shrink benchmarks under ASan

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-06-25 16:11:11 +08:00
Xuehan Xu
52735b825f
Merge pull request #69279 from xxhdx1985126/wip-seastore-pessimistic-cc-mutation-pending-crc-fix
crimson/os/seastore/cached_extent: renew mutation_pending extents' last_committed_crc when committing rewrite transactions

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2026-06-25 15:31:28 +08:00
Xuehan Xu
3d1ab305ef
Merge pull request #69431 from xxhdx1985126/wip-seastore-fix-update_copied_lba_keys
crimson/os/seastore/transaction: clear copied_lba_keys on reset

Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2026-06-25 15:30:30 +08:00
Xuehan Xu
bdc5168683
Merge pull request #69447 from xxhdx1985126/wip-seastore-update-sync-fix
crimson/os/seastore/lba: don't update the lba mapping if its child is an initial pending one.

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
Reviewed-by: Ronen Friedman <rfriedma@redhat.com>
2026-06-25 15:29:57 +08:00
Nitzan Mordechai
cab98604fa mon/config: trim whitespace in config target
When set\get\rm config values with leading or trailing spaces,
those spaces need to be removed before we are trying to get\set\rm
those keys.
add trim code before each approch and also for exist keys in config.

Fixes: https://tracker.ceph.com/issues/77598
Signed-off-by: Nitzan Mordechai <nmordech@ibm.com>
2026-06-25 06:52:47 +00:00
Guillaume Abrioux
9e1e62070b
Merge pull request #69691 from guits/fix-staggered-upgrade
cephadm: fix staggered upgrade
2026-06-25 08:37:46 +02:00
Kefu Chai
ed1cdc5478
Merge pull request #69687 from tchaikov/wip-cpatch-esc-seq
script: fix invalid escape sequences

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2026-06-25 14:37:33 +08:00
Ronen Friedman
858baee959
Merge pull request #69451 from ronen-fr/wip-rf-getxa
crimson/osd: return ENOENT from EC getxattr for non-existent objects

Reviewed-by: Jose J Palacios-Perez <perezjos@uk.ibm.com>
2026-06-25 08:00:41 +03:00
Oguzhan Ozmen
19748f1767 test/rgw/beast: add unit test for ConnectionList shutdown behavior
Tests: https://tracker.ceph.com/issues/77658
Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-24 19:02:09 +00:00
Oguzhan Ozmen
179b873e9b rgw/beast: (non-functional change) add size method to ConnectionList
This is meant to be used in unittesting.

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-24 19:02:09 +00:00
Oguzhan Ozmen
15915c85e1 rgw/beast: fix shutdown crash in ConnectionList::close()
ConnectionList::close() is called from the main thread during
AsioFrontend::stop(), but io_context pool threads may still have
pending async_write operations on those sockets. The old code called
socket.close() which is a boost::asio thread-safety violation:

  - it NULLifies descriptor_data in the epoll reactor which causes
    concurrent start_op calls to dereference NULL (SIGSEGV), or
  - it deregisters the fd from epoll, orphaning pending operations so
    their completion handlers would never fire (hang)

Fix by using socket.cancel() + native ::shutdown():
- cancel() dequeues pending operations from the reactor's
  per-descriptor queue and adds completions with
  operation_aborted. This should handle writes blocked in epoll.
- shutdown(fd, SHUT_RDWR) disables the transport so any
  strand-queued writes that haven't entered epoll yet will fail
  with EPIPE on the next send() attempt

Also remove connections.clear() since coroutines exited via I/O error
and their RAII Guard removes them from the list on destruction.

This follows the same principle as the fix for
https://tracker.ceph.com/issues/58670 which handles the same scenario
in timeout_handler path.

Bottomlime, socket operations must not be performed from outside the owning
strand.

Fixes: https://tracker.ceph.com/issues/77658
Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-24 19:02:09 +00:00
Oguzhan Ozmen
222a39366d rgw/beast: (non-functional) refactor Connection and ConnectionList to header
Just like rgw_asio_frontend_timer.h, factor out the Connection struct and
ConnectionList class from the anonymous namespace in rgw_asio_frontend.cc
to a new header rgw_asio_frontend_connection.h (under namespace rgw::asio).

This is especially needed to get better coverage thru unittesting.

This is non-functional: no logic changes, the types, methods, and behavior
are identical as before. The frontend .cc uses type aliases to maintain the
same unqualified names.

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-24 19:01:47 +00:00
Miki Patel
5a77c4fd35 rbd-mirror: Remove old non-primary demoted image snapshots on the local cluster
When an image is demoted on primary cluster and later promoted again,
a non-primary demoted image snapshot is created on the peer (secondary)
cluster. Each such promote/demote cycle on same cluster results in an
additional non-primary demoted snapshot being created on peer.
As a result, repeated promote/demote cycles on the same cluster can
lead to accumulation of stale non-primary demoted snapshots on the
secondary cluster. These snapshots are not removed immediately because
cleanup is only triggered when peer (secondary) is promoted and then
demoted.

The proposed changes ensures that such snapshots are proactively identified
and removed, preventing unbounded buildup. Also added test coverage to
verify cleanup behavior.

Fixes: https://tracker.ceph.com/issues/76155

Signed-off-by: Miki Patel <miki.patel132@gmail.com>
2026-06-24 23:11:46 +05:30
Adam Kupczyk
9061bc4a44 doc/rados/bluestore: ExtBlkDev, FCM plugin
Add new page about 2 topics:
- EXTBLKDEV
- FCM plugin

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
2026-06-24 19:16:54 +02:00
Oguzhan Ozmen
78b5f05d00 doc: add PendingReleaseNotes entry for the new rgw multipart category
Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-24 16:53:29 +00:00
Oguzhan Ozmen
c429d2d15d rgw/s3-tests: test HEAD bucket object count excludes incomplete multipart parts
Add test_head_bucket_usage_multipart_incomplete to verify that
X-RGW-Object-Count (returned by HEAD Bucket with ?read-stats=true)
does not count in-progress multipart upload parts as completed
objects.

After uploading 2 parts without completing, HEAD bucket should
report 0 objects and ListObjectsV2 should return 0 keys.

Tests: https://tracker.ceph.com/issues/77509
Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-24 16:53:29 +00:00
Oguzhan Ozmen
29099f910c qa/rgw: add integration test for multipart part category in bucket stats
Test that multipart upload part heads are tracked under the
rgw.multipart category in radosgw-admin bucket stats, separate from
completed objects (rgw.main).

- Incomplete upload: rgw.main=0, rgw.multipart=X, rgw.multimeta=1
- CompleteMultipart: parts cleaned up, assembled object in rgw.main
- AbortMultipart: parts cleaned up, rgw.multipart returns to 0
- ListObjects returns 0 during incomplete upload

Also add quota enforcement testcase for multipart part category

- Verify that multipart upload parts tracked under RGWObjCategory::MultiPart
  still count against both bucket and user quotas.

Tests: https://tracker.ceph.com/issues/77509
Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-24 16:51:57 +00:00
Oguzhan Ozmen
93e763cee9 rgw: add RGWObjCategory::MultiPart for multipart upload part heads
Multipart upload part head objects are bucket-indexed with
RGWObjCategory::Main, making them indistinguishable from completed
objects in bucket stats. This is causing radosgw-admin bucket stats and
the RGW-extension X-RGW-Object-Count header on HEAD Bucket
(?read-stats=true) to report inflated object counts when in-progress
or abandoned multipart uploads exists.

So, this commit adds a new Object Category: RGWObjCategory::MultiPart
(value 5) and assigns it in MultipartObjectProcessor::complete() so
part heads are now tracked under a separate "rgw.multipart" category.

The X-RGW-Object-Count header, which reads only RGWObjCategory::Main,
now excludes in-progress parts. This is more correct as AWS S3 HEAD Bucket
does not expose any object count for incomplete uploads (the header itself
seems to be an RGW extension, not part of the S3 specification).

Quota enforcement, stats aggregation, and part cleanup are all
category-agnostic so should require no changes.

Also, updated the test_cls_rgw_stats simulator to use
the new category for multipart part entries.

Fixes: https://tracker.ceph.com/issues/77509
Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-24 16:46:21 +00:00
Oguzhan Ozmen
06eeede4cb cls/rgw: clarify RGWObjCategory::Shadow comment
The Shadow enum value (2) is never assigned anywhere in the codebase.
Update the comment to clearly state it is unused and reserved for
backward compatibility, since the value is serialized as a raw uint8_t
in bucket index entries and may exist in on-disk data.

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-24 16:46:20 +00:00
Ronen Friedman
e9bf02b72b crimson/osd: return ENOENT from EC getxattr for non-existent objects
PGBackend::getxattr() for EC pools reads from the attr_cache and
returns ENODATA when an attr is not found, without distinguishing
between "object exists but attr not set" and "object doesn't exist".
This causes cls methods like cls_refcount_get() to proceed as if the
object exists, enter the EC RMW pipeline, and hang.

Add an os.exists check in the EC path so non-existent objects return
ENOENT, matching the behavior of the replicated path which goes to
the object store and gets ENOENT directly.

Fixes: https://tracker.ceph.com/issues/77335
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2026-06-24 15:55:39 +00:00
Zack Cerza
41c9db3444
Merge pull request #69462 from sunyuechi/wip-sy-run-make-sccache-stats
build/script: print sccache stats alongside ccache
2026-06-24 09:46:19 -06:00
Afreen Misbah
c47da65d66
Merge pull request #69574 from rhcs-dashboard/dash-binding
mgr/dashboard: fix bind address regression from CherryPy isolation

Reviewed-by: Nizamudeen A <nia@redhat.com>
2026-06-24 21:13:56 +05:30
Afreen Misbah
98f04401bb
Merge pull request #69390 from rhcs-dashboard/fix-feedback-module
mgr/dashoard : Fix feedback module enablement


Reviewed-by: Naman Munet <nmunet@redhat.com>
2026-06-24 21:04:45 +05:30
Aashish Sharma
ae6c498b7e
Merge pull request #69298 from rhcs-dashboard/fix-77129-main
mgr/dashboard: Fix username validation for special characters by URL-encoding user lookup requests

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-06-24 20:55:47 +05:30
Kamoltat (Junior) Sirivadhna
093426d120
Merge pull request #69497 from kamoltat/wip-ksirivad-DFBUGS-7044
mon: fix ConnectionTracker::notify_rank_removed
Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-06-24 11:02:15 -04:00
Casey Bodley
69b89f2274
Merge pull request #65724 from adamemerson/wip-include-s3tests
Include s3-tests in ceph repo

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2026-06-24 10:12:54 -04:00
Redouane Kachach
0157ecd37e
Merge pull request #69438 from kginonredhat/issue-77374-orchestrator_cli-test_mds_rm-fails-missingforce-_delete_data-kwarg
mgr/test_orchestrator: accept force_delete_data in remove_daemons

Reviewed-by: Redouane Kachach <rkachach@ibm.com>
2026-06-24 15:43:58 +02:00
Guillaume Abrioux
6c15395e7f
Merge pull request #69477 from guits/node-proxy-atollon
node-proxy: atollon hardware monitoring (FCM stats, temperatures, fan speed..)
2026-06-24 14:58:59 +02:00
Avan Thakkar
0077cb990d mgr: fix MgrModuleRecoverDB to honor its retry budget on db cleanup failure
Both close_db() and open_db() during retry were unguarded, so a failure
in either escaped the decorator immediately instead of retrying up to
MAX_DBCLEANUP_RETRIES.

Wrapped both in the same try/except, and logged the
failed attempt.

Fixes: https://tracker.ceph.com/issues/77635
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2026-06-24 17:52:39 +05:30
Sun Yuechi
d843b01385 test/encoding: run readable.sh with a sliding window of jobs
The old scheduler spawned nproc jobs, then waited for the whole batch to
finish before starting the next, so cores that finished early sat idle
waiting on the slowest type in the batch. Keep a sliding window instead:
once the window is full, reap a single finished job before spawning the
next, which keeps the cores busy when per-type runtimes vary widely.

While here, pass vdir/arversion into test_object explicitly instead of
reading the loop globals, and give each job its own result file instead
of reusing slot-numbered files that the sliding window would otherwise
hand out to overlapping jobs.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-24 20:01:11 +08:00
Abhishek Desai
1262deedb2 mgr/dashboard : Carbonize configuration form
fixes : https://tracker.ceph.com/issues/76778
Signed-off-by: Abhishek Desai <abhishek.desai1@ibm.com>
2026-06-24 17:22:05 +05:30
Guillaume Abrioux
62ccfb2eb2
Merge pull request #69569 from guits/fix-77486
ceph-volume: skip internal raid mirror LVs in inventory
2026-06-24 13:43:44 +02:00
Matthew N Heler
5c6e98176d mgr: filter root logger fallback
Make the mgr root logger fallback follow debug_mgr and preserve the
original logger name, so third-party DEBUG logs are not emitted by
default or mislabeled as mgr logs.

Signed-off-by: Matthew N Heler <matthew.heler@hotmail.com>
2026-06-24 06:08:43 -05:00
Kefu Chai
d3df37a61b crimson/osd: mark ThrottleReleaser [[nodiscard]]
OperationThrottler::get_throttle() returns a ThrottleReleaser guard whose
destructor calls release_throttle(), the mClock RequestCompletion that
decrements in_progress.  The guard has to be held for the whole operation: a
caller that discards it at acquisition releases the slot in the same step, so
the operation never counts against max_in_progress and the throttle does
nothing.

Mark the type [[nodiscard]] so the compiler flags a discarded co_await of
get_throttle().

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-24 18:22:09 +08:00
Mark Kogan
d22252b6b5 rgw: reduce default thread pool size
Fixes: https://tracker.ceph.com/issues/65656

Signed-off-by: Mark Kogan <mkogan@redhat.com>
2026-06-24 09:45:47 +00:00
Xuehan Xu
78e74e8d71 crimson/os/seastore/lba: non-existing LBACursors are strictly forbidden
to refresh

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-06-24 16:24:38 +08:00
Xuehan Xu
cb2321f7fa osd/OSDMap: fix the pgs_per_weight calculation error when multiple pools
are being balanced

Before this commit, When multiple pools are being balanced,
pgs_per_weight only considers the last pool, which is wrong.

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-06-24 15:49:44 +08:00
Xuehan Xu
cfd916b1b4 osd: speed up upmap underfull fallback
According to our tests, this commit can speed up pg upmap calculations
by orders of magnitude

Performance analysis:
Suppose:
R = number of pg_upmap_items (the full candidate pool, after to_skip/only_pools filtering)
M = how many OSDs the deviation_osd loop visits before it breaks (less or equal to underfull count)
U = the number of distinct from OSDs in the upmap items. U is less or equal to number of OSDs, and should be less than R x k
k = mapping pairs per candidate (usually 1, small)

Time complexity of OSDMap::calc_pg_upmaps():
Before the commit: O(M * R)
After the commit: O(R * k + M * log(U)) ~= O(R + M *log(U))

Performant evaluation:
Existing upmap items    Underfull OSDs  Old path        New path        Speedup
50000                   1000            6.51s           0.047s          137x
100000                  1000            12.05s          0.086s          140x
200000                  2000            63.93s          0.281s          227x

Fixes: https://tracker.ceph.com/issues/77563
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
Assisted-by: Codex:GPT-5.5
2026-06-24 15:48:16 +08:00
Radoslaw Zarzynski
89ff2cbcef
Merge pull request #69526 from rzarzynski/wip-bug-77417
tests/neorados: fix ceph_test_neorados_completions being not installed

Reviewed-by: Kamoltat Sirivadhna <ksirivad@redhat.com>
Reviewed-by: Adarsha Dinda <adarshadinda@Adarshas-MacBook-Pro.local>
Reviewed-by: Guillaume Abrioux <gabrioux@ibm.com>
2026-06-24 09:41:29 +02:00
Abhishek Desai
bdbfa8a15a mgr/dashboard : Fix feedback module enablement
fixes : https://tracker.ceph.com/issues/75734
Signed-off-by: Abhishek Desai <abhishek.desai1@ibm.com>
2026-06-24 13:08:01 +05:30
Xuehan Xu
bf0e7afecb crimson/os/seastore/random_block_manager/avlallocator: avoid directly modifying avl_set elements
Fixes: https://tracker.ceph.com/issues/77549
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-06-24 15:08:41 +08:00
Xuehan Xu
aa9247a36c crimson/os/seastore/journal: change CircularBoundedJournal::replay() to
use coroutine

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-06-24 14:55:41 +08:00
root
bdc16ed02f os/bluestore: Rename allocator score output label
This commit changes the output label of the bluestore allocator
 score command from fragmentation_rating to fragmentation_score

Fixes: https://tracker.ceph.com/issues/77331

Signed-off-by: Prachi Lasurkar <lasurkarprachi@gmail.com>
2026-06-24 06:49:31 +00:00
Pablo de Lara
a273687273 crypto/isa-l: use new API added in v2.25
New API with parameter checking was added in v2.25,
deprecating the original API used in this module.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
2026-06-24 06:20:31 +00:00
Pablo de Lara
ff5970bb5b crypto/isa-l: update isa-l_crypto submodule to v2.26.1
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
2026-06-24 06:18:27 +00:00
Kefu Chai
b51d3cd190
Merge pull request #69517 from sunyuechi/fix-ctest-resource-online-cpus
run-make-check.sh: build ctest cpu resource pool from sched_getaffinity

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-06-24 14:10:32 +08:00
Kefu Chai
083ae56be7
Merge pull request #69157 from sunyuechi/wip-mds-quiesce-test-evaluate-await-idle
test/mds: don't drop await_idle_v under NDEBUG

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-06-24 14:01:41 +08:00
Kefu Chai
3a8737f3fd
Merge pull request #69162 from sunyuechi/wip-cephadm-tests-mock-find-program-lvcreate
cephadm/tests: mock find_program in test_container_engine

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2026-06-24 14:01:00 +08:00
Guillaume Abrioux
946c23055c
cephadm: fix staggered upgrade
Do not mark an upgrade complete until daemons in the filtered scope
match the target image. Limited (--limit) upgrades still complete
when remaining_count reaches zero.

Fixes: https://tracker.ceph.com/issues/62959

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2026-06-24 07:26:49 +02:00
Sun Yuechi
e16e6398b1 test/bufferlist: shrink benchmark iterations under ASan
The pure-timing *_bench/BenchAlloc loops repeat a stateless path millions of
times. Under ASan that adds no memory-error coverage but inflates the run from
minutes to hours, so cut the iteration counts when built with ASan.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-24 11:44:05 +08:00
Sun Yuechi
3a9a837ef7 test/bufferlist: avoid BenchDeref iterator overrun past end
Drive the deref loop by get_remaining() >= step instead of comparing to
end(), so a total length that is not a multiple of step no longer lets
`iter += step` run past the end and throw "End of buffer".

The original counts were always a multiple of step, so the overrun only
showed up once the following commit shrinks the rounds under ASan -- a
false alarm rather than a real regression, which is why it went unnoticed
until now.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-24 11:44:05 +08:00
Kotresh HR
a38b620ccd
Merge pull request #68827 from kotreshhr/mirror-metrics-new-mgr-interface
tools/cephfs_mirror: Mirror metrics via  new mgr interface

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2026-06-24 08:18:37 +05:30
Guillaume Abrioux
e982412a7a
node-proxy: expose FCM stats
Collect FCM stats locally from NVMe drives (vendor log page 0xCA)
and expose them via node-proxy, the cephadm agent, and
`ceph orch hardware status --category fcm`.

Introduce a node backend that aggregates Redfish data with
node-local collectors, since FCM metrics are not available
from the BMC.

Fixes: https://tracker.ceph.com/issues/77521

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2026-06-24 04:17:52 +02:00
Guillaume Abrioux
1cd4b32d72
node-proxy: rename firmwares to firmware with legacy aliases
Let's use 'firmware' as the standard name in node-proxy, cephadm,
and orch hardware status.

We still accept 'firmwares' as a deprecated alias and read legacy
cache payloads transparently for backward compatibility.

Fixes: https://tracker.ceph.com/issues/77410

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2026-06-24 04:17:39 +02:00
Guillaume Abrioux
9b3bfd171a
node-proxy: override with atollon specific
This adds AtollonSystem for AMI/Atollon BMCs:
- memory Id mapping,
- storage description fixes,
- StorageControllers based drive enrichment

It also wires vendor selection through cephadm (hw_monitoring_vendor)
and show slot/firmware in hardware status storage output.

Fixes: https://tracker.ceph.com/issues/77408

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2026-06-24 04:17:28 +02:00
Guillaume Abrioux
deabf21144
node-proxy: add temperatures and fan speed
This adds the temperatures category and fan speed information.

Fixes: https://tracker.ceph.com/issues/77408

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2026-06-24 04:17:12 +02:00
Kefu Chai
095cff07be crimson/common: use the value_map passed to dump_metric_value_map()
dump_metric_value_map() takes a value_map by const reference but ignored it
and iterated seastar::scollectd::get_value_map() instead.  Both callers already
fetch the map and pass it in, so the map was fetched twice per call and the
argument was dead.

Iterate vmap.  The output is unchanged, since the callers pass get_value_map(),
but the parameter now means what it says and the redundant fetch is gone.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-24 08:58:15 +08:00
Kefu Chai
2e2f5f7670
Merge pull request #69664 from tchaikov/wip-asan-options
cmake,ceph,qa: update and populate ASan options to ceph cli

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2026-06-24 08:53:07 +08:00
Kefu Chai
a33b19eb68 script: fix invalid escape sequences
Python 3.12 warns on invalid escape sequences in non-raw strings:
```
  ceph-release-notes:45: SyntaxWarning: invalid escape sequence '\d'
  cpatch.py:465: SyntaxWarning: invalid escape sequence '\;'
```

In `cpatch.py`, use raw strings for the two Dockerfile RUN lines so
their find -exec ... \; keeps the literal backslash-semicolon.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-24 07:11:26 +08:00
Kefu Chai
a1afaa6529 cmake/boost: build libboost_context with BOOST_USE_ASAN
The ceph-api job failed under ASan: radosgw-admin aborted at startup with a
heap-buffer-overflow in boost.context's fiber resume().

  ==1155842==ERROR: AddressSanitizer: heap-buffer-overflow ... READ of size 8
    #0 boost::context::detail::fiber_activation_record::resume() fiber_ucontext.hpp:153
    #10 RGWSI_Notify::do_start(optional_yield, DoutPrefixProvider const*) svc_notify.cc:261
  0x... is located 8 bytes after 1048-byte region allocated by
    boost::context::detail::fiber_activation_record_initializer()

fiber_activation_record carries three extra members (fake_stack, stack_bottom,
stack_size) only when BOOST_USE_ASAN is defined.  Under WITH_ASAN we build
boost.context with context-impl=ucontext and give consumers BOOST_USE_ASAN and
BOOST_USE_UCONTEXT through Boost::context's INTERFACE_COMPILE_DEFINITIONS, but
we never pass BOOST_USE_ASAN to b2.  So libboost_context, which compiles
fiber_activation_record_initializer() and allocates the record, uses the
smaller no-ASan layout, while consumers that include the header (here rgw's
RGWSI_Notify::do_start via boost::asio::spawn) compile resume() with the larger
layout and read stack_bottom/stack_size past the allocation.

Pass define=BOOST_USE_ASAN to b2 so the library and its consumers agree on the
struct layout.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-24 06:44:18 +08:00
Laura Flores
cf695e6c81 qa/suites/upgrade: add POOL_FULL variations to ignorelist
Fixes: https://tracker.ceph.com/issues/75414
Signed-off-by: Laura Flores <lflores@ibm.com>
2026-06-23 17:07:45 -05:00
Laura Flores
6158e8efcb
Merge pull request #69416 from NitzanMordhai/wip-nitzan-test-osd-recovery-prio-race
osd-recovery-prio: race condition fix in TEST_recovery_pool_priority

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Kamoltat Sirivadhna <ksirivad@redhat.com>
2026-06-23 16:27:28 -05:00
Laura Flores
5475faaddf qa/tasks: change collectl source to download.ceph.com
Getting it from our own service avoids exposing us to
supply chain attacks in our QA infrastructure.

Fixes: https://tracker.ceph.com/issues/76927
Signed-off-by: Laura Flores <lflores@ibm.com>
2026-06-23 15:13:33 -05:00
Afreen Misbah
f45a1bfc93 mgr/dashboard: fix bind address regression from CherryPy isolation
The CherryPy isolation refactor (PR #67227) accidentally changed the
dashboard bind address from wildcard (*:8443) to mon_ip:8443. The
get_mgr_ip() replacement was originally only for URI generation, but
the refactor passed the mutated address to CherryPyMgr.mount() as the
actual socket bind address.

This breaks the management gateway when its VIP is not on the same
interface as mon_ip, as the dashboard becomes unreachable on other
interfaces.

Preserve the original wildcard address for binding and only use
get_mgr_ip() for the advertised URI. Add regression test to prevent
future confusion between bind_addr and server_addr.

Fixes: https://tracker.ceph.com/issues/77491

Signed-off-by: Afreen Misbah <afreen@ibm.com>
2026-06-23 22:29:12 +05:30
eameh-LF
a12a619bba
Merge pull request #69361 from eameh-LF/wip-doc-77183
doc/rados/configuration: Fix FileStore contradiction in journal-ref.rst
2026-06-23 17:25:57 +01:00
eameh-LF
5d3232a3cd
Merge pull request #69362 from eameh-LF/wip-doc-77184
doc/rados/operations: Strengthen cache-tiering deprecation notice
2026-06-23 17:25:34 +01:00
eameh-LF
de698b6c11
Merge pull request #69669 from eameh-LF/wip-doc-77188
doc/cephfs: clarify deprecated commands and remove obsolete upgrade path
2026-06-23 17:13:32 +01:00
Ilya Dryomov
59af04847a
Merge pull request #69676 from tchaikov/wip-revert-asan
Revert "script/run-make: enable ASan"

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2026-06-23 18:04:56 +02:00
eameh-LF
d6e511a752
Merge pull request #69359 from eameh-LF/wip-doc-77192
doc/install: Update EOL version floor references in index.rst
2026-06-23 17:01:20 +01:00
mheler
14f892dc8f
Merge pull request #69336 from mheler/wip-rgw-sse-gcm-mpu-fix
rgw: fix AES-256-GCM key/IV reuse on multipart part re-upload
2026-06-23 10:52:28 -05:00
Kefu Chai
320a3a0529 qa/lsan.supp: suppress CPython 3.12/3.13 interpreter leaks
The python binaries on some CI images and dev boxes ship stripped, so the
allocator frames in an interpreter-shutdown leak come through unsymbolised
(/usr/bin/python3.13+0x...) and the function-name matches above cannot apply.
leak:python3.10 already handled this for 3.10, and a stale comment claimed 3.12
does not leak.

Add leak:python3.12 and leak:python3.13, mirroring the 3.10 entry, so the
interpreter globals are suppressed whatever CPython the build uses.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-23 23:02:05 +08:00
Kefu Chai
55ac10180c ceph.in: load asan/lsan suppressions on WITH_ASAN builds
bin/ceph from a WITH_ASAN build aborts at exit, with LeakSanitizer reporting
CPython and Cython module-init allocations as leaks:

  ==2577940==ERROR: LeakSanitizer: detected memory leaks
  Direct leak ... in PyObject_Malloc (/usr/bin/python3.12+...)
    #4 __pyx_pymod_exec_rados rados_processed.c
  SUMMARY: AddressSanitizer: 32113 byte(s) leaked in 30 allocation(s).

These are interpreter globals that live for the process lifetime.  qa/lsan.supp
already suppresses them, but bin/ceph never loaded it: vstart.sh sets
LSAN_OPTIONS for the daemons it spawns, while a bin/ceph invoked separately
(ceph-api runs ./bin/ceph fsid once vstart.sh returns) inherits none and exits
non-zero.  It stayed hidden until radosgw-admin stopped crashing in vstart and
the run reached that call.

ceph.in already re-execs with the ASan runtime preloaded under WITH_ASAN.  Set
ASAN_OPTIONS and LSAN_OPTIONS first, from the CEPH_ASAN_OPTIONS and
CEPH_LSAN_OPTIONS that CMake also feeds add_ceph_test(), so the re-exec'd
interpreter starts with the suppressions loaded.  Use setdefault so a value
from the caller still wins.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-23 23:02:05 +08:00
Kefu Chai
2c9b73894b cmake: factor the ASan/LSan test options into cache variables
add_ceph_test() spelled out the suppression-file paths and sanitizer flags
inline.  bin/ceph needs the same options, so lift them into CEPH_ASAN_OPTIONS
and CEPH_LSAN_OPTIONS and have add_ceph_test() consume those.  The environment
the tests run with is unchanged.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-23 23:02:05 +08:00
Nathan Hoad
59bf2dad24 rgw: For consistency with C++ core guidelines, make these out-params pass-by-reference instead of pointers.
Signed-off-by: Nathan Hoad <nhoad@bloomberg.net>
AI-Assisted: I used AI to make this change. I reviewed and tested it manually.
2026-06-23 10:35:11 -04:00
David Galloway
f586ee7b7e
Merge pull request #69681 from djgalloway/fix-backport-workflow
.github/workflows/releng-audit.yaml: Check main for upstream commit
2026-06-23 10:33:48 -04:00
Nathan Hoad
12efa485f5 rgw: Implement support for --shard-id in radosgw-admin's gc process and gc list commands
This allows operators to do targeted GC work against specific shards, instead of working on all of them.

Signed-off-by: Nathan Hoad <nhoad@bloomberg.net>
AI-Assisted: Used AI to generate the initial implementation.
2026-06-23 10:31:27 -04:00
David Galloway
212eb0bc04 .github/workflows/releng-audit.yaml: Check main for upstream commit
This check was failing to correctly identify whether backported commits
had actually been cherry-picked because it was checking the target branch
instead of the main branch.  This change checks out the main branch
to look for the upstream cherry-picked commit.

Signed-off-by: David Galloway <david.galloway@ibm.com>
2026-06-23 10:11:34 -04:00
Gil Bregman
e317e86c8e
Merge pull request #69632 from gbregman/main
mgr/cephadm: Add degraded namespace flag to NVMEoF spec file
2026-06-23 16:24:36 +03:00
mheler
c6bee66adc
Merge pull request #69175 from cbodley/wip-qa-rgw-multisite-gcm 2026-06-23 08:17:59 -05:00
Venky Shankar
9be97d7cde
Merge pull request #68482 from chrisphoffman/wip-73347
qa: Add direct_io test for fscrypt

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2026-06-23 18:34:39 +05:30
Kotresh HR
b80828fbad qa: Add mgr snapshot mirror status tests
Add teuthology coverage for `ceph fs snapshot mirror status`:
parity with asok peer_status, default idle metrics, stale omap
handling, error paths, filter scopes, daemon restart, and cache TTL.

Reuse existing peer_dir_status and metrics assertion helpers.
Adjust stale test wait for InstanceWatcher timeout and set a
short cache TTL in the cache test. Pass peer_uuid via --peer_uuid=
in the test helper for peer-only scope queries.

Fixes: https://tracker.ceph.com/issues/76686
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-23 18:12:26 +05:30
Kotresh HR
87638c8be4 doc/cephfs: Document fs snapshot mirror status mgr command
Describe the mirroring module command that reads persisted omap
metrics, including syntax, output layout, stale detection, caching,
and comparison with the admin socket peer status interface.

Document --peer_uuid as a named argument for peer-only and
directory/peer filtering.

Also update PendingReleaseNotes

Fixes: https://tracker.ceph.com/issues/76686
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-23 20:49:29 +05:30
Kotresh HR
01dd9506f2 mgr/mirroring: make snapshot mirror metrics cache optional
Add snapshot_mirror_metrics_cache_enabled (default true). When disabled,
metrics_status reads omap directly and skips complete and partial caches.
When enabled, behavior is unchanged.

Fixes: https://tracker.ceph.com/issues/76686
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-23 20:49:29 +05:30
Kotresh HR
e36faf8bcc mgr/mirroring: make snapshot mirror metrics cache TTL configurable
Add snapshot_mirror_metrics_cache_ttl as a runtime mgr/mirroring module
option (default 15 seconds) instead of a hard-coded CACHE_TTL_SECS.
Both complete and partial lru_cache_timeout wrappers read the value
when caching omap metrics so operators can tune cache freshness without
code changes.

Fixes: https://tracker.ceph.com/issues/76686
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-23 20:49:29 +05:30
Kotresh HR
a7325bafb9 mgr/mirroring: handle missing cephfs_mirror object in metrics status
When snapshot mirroring is not enabled for a filesystem, the
cephfs_mirror RADOS object does not exist and "ceph fs snapshot mirror
status" fails reading sync stat omap. Map rados ENOENT to a clear
MirrorException so the CLI reports that snapshot mirroring must be
enabled instead of a generic omap read failure or an uncaught exception.

Also catch unexpected errors in metrics_status like other mirror CLI
handlers, so the mgr module does not crash on failure. Return
-errno.EINVAL from the generic error path instead of the exception
message as exit code.

Fixes: https://tracker.ceph.com/issues/76686
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-23 20:49:29 +05:30
Kotresh HR
c24b8d3cc2 mgr/mirroring: Show default stats on newly added dir_root
When the directory is added for mirroring and the snapshot
is not taken yet, the peer_status show the following default
metrics.

{
  'state': 'idle',
  'snaps_synced': 0,
  'snaps_deleted': 0,
  'snaps_renamed': 0,
}

The mgr interface should also match that.

Fixes: https://tracker.ceph.com/issues/76686
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-23 20:49:29 +05:30
Kotresh HR
15e998b865 mgr/mirroring: detect stale snapshot mirror sync metrics in omap
Persisted metrics in the cephfs_mirror omap can outlive the writing
daemon when cephfs-mirror stops or a directory is reshuffled to another
instance; mgr would keep reporting stale progress until the owning
daemon writes again.

Extend format_and_order_sync_stat_for_display() to compare persisted
_instance_id against InstanceWatcher live instances (via
FSPolicy.get_live_instance_ids()) and the directory's tracked instance
(via Policy.get_tracked_instance_id()). Mark metrics stale when the
persisted writer is no longer live (any state), or when it does not
match the tracked instance while persisted state is not "idle". Show
state "stale" with current_syncing_snap omitted.

Pass policy and live instance ids through load_sync_stat_metrics() and
fetch_sync_stat_metrics(), and into sync_stat_complete_cache and
sync_stat_partial_cache loaders. Cache hits serve already-formatted
(stale-marked) entries until TTL expiry without re-checking instance
liveness.

Fixes: https://tracker.ceph.com/issues/76686
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-23 20:49:29 +05:30
Kotresh HR
8647c2ee79 mgr/mirroring: cache fs snapshot mirror status omap metrics
Add a short-lived in-memory cache for metrics returned by
"ceph fs snapshot mirror status", which reads persisted sync stats
from the cephfs_mirror object omap. Omap walks are relatively
expensive; caching reduces repeated reads when the CLI or multiple
clients poll at short intervals.

Two TTL-bucketed LRU caches (CACHE_TTL_SECS) are used instead of one
unified cache. The complete cache always holds every mirrored
directory and peer for a filesystem; the partial cache holds one
directory. A single directory-granularity cache cannot prove it has
all directories, so full-scan queries rely on the complete cache
contract.

Cache implementation (lru_cache_timeout decorator backed by
_TimedLRUCache, not functools.lru_cache):

- lru_cache has no TTL and no peek-on-hit API. Single-directory
  queries must read the complete cache without loading on miss;
  lru_cache.cache is also unavailable on Python 3.14+.

- complete (sync_stat_complete_cache): full omap prefix scan via
  load_sync_stat_metrics. Cache key: (time_token, filesystem).
  COMPLETE_CACHE_MAX limits filesystem entries per TTL window.

- partial (sync_stat_partial_cache): per-directory omap key load via
  fetch_sync_stat_metrics. Cache key: (time_token, filesystem,
  dir_path, peer_ids). PARTIAL_CACHE_MAX limits directory entries.

Bind cache_peek/cache_info/cache_clear via a CachedMethod descriptor
so TTL lookups receive (self, ...); otherwise single-dir status fails
with 'str' object has no attribute 'mgr'.

Serve logic (under the existing lock):

- status <fs> [--peer_uuid=<uuid>]: load complete cache on miss;
  filter by peer when requested.

- status <fs> <dir>: peek complete cache (no load on miss); if the
  entry contains the directory and peers, serve from complete.
  Otherwise load partial cache (omap on miss).

Fixes: https://tracker.ceph.com/issues/76686
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-23 20:49:29 +05:30
Kotresh HR
4c19167719 mgr/mirroring: Reorder and format metrics output
Reorder and format status output to match the output of asok
interface peer_status command. Return metrics under
metrics/<dir>/peer/<uuid> to match the asok peer_status layout,
including {"metrics": {}} when no peers are configured.

Fixes: https://tracker.ceph.com/issues/76686
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-23 20:49:29 +05:30
Kotresh HR
33c572242b mgr/mirroring: Add new interface to expose mirroring metrics
Add the following new interface to expose mirroring metrics

ceph fs snapshot mirror status <fsname> [<mirrored_dir_path>] [--peer_uuid=<peer_uuid>]

The cmd loads the persisted directory sync metrics from the
cephfs_mirror object's omap. Metrics are grouped by mirrored
directory and peer.

When --peer_uuid is specified, only metrics for that peer are
returned. peer_uuid is a named CLI argument (_end_positional_) so
peer-only filtering does not require a mirrored directory path.

Fixes: https://tracker.ceph.com/issues/76686
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-23 20:49:29 +05:30
Kotresh HR
51a8348dfd tools/cephfs_mirror: Remove persisted dir stats
When a directory is removed from mirroring, the persisted directory
stats need to be removed.  This patch handles the cleanup.

Omap keys must not be removed when mirrored directories are reshuffled
across cephfs-mirror daemons.  The mgr release notify now carries a
purging flag (set only during permanent removal, not reshuffle), and
the daemon removes persisted stats only when purging is true.  On
reshuffle with an in-progress sync, clear live current_syncing_snap
state and persist idle metrics so the acquiring daemon does not inherit
stale syncing omap entries.

Fixes: https://tracker.ceph.com/issues/76686
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-23 20:49:29 +05:30
Kotresh HR
4b907eaf08 tools/cephfs_mirror: Load persisted mirror metrics from omap
Load last_synced_snap metadata from the cephfs_mirror object omap on
PeerReplayer initialization and when a mirrored directory is added.
Live current_syncing_snap metrics are not restored; they are rebuilt
when synchronization starts.

When the daemon restarts after a snapshot was synced on the remote but
metrics were not yet written to omap, loaded metadata may belong to an
older snapshot.  Add reconcile_last_synced_snap() to compare against
the remote snap map, clear stale last-sync fields, and update
last_synced_snap id/name in memory.

Treat snaps_synced, snaps_deleted, and snaps_renamed as per-session
counters.  Do not load them from omap; they start at zero for each
daemon session and are still reported via the admin socket.  Persist
omap metrics unconditionally after reconcile so the mgr picks up the
new instance id and cleared session counters on restart.

Fixes: https://tracker.ceph.com/issues/76686
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-23 20:49:25 +05:30
Kotresh HR
c5dd509670 tools/cephfs_mirror: Persist metrics to omap
Persist snapshot mirroring metrics to the cephfs_mirror object omap
for the mgr/mirroring module to support status command.

The tick thread only keeps omap up to date for in-progress syncs on
registered directories. Persist explicitly when stats change so omap
is updated before a directory unregisters:

 - snap delete/rename propagation (inc_deleted_snap / inc_renamed_snap)
 - after each successful snap sync (following set_last_synced_stat)
 - after sync_snaps failure (following _inc_failed_count)
 - after sync_perms failure (following _inc_failed_count)

Without the explicit call sites, omap can keep stale live or idle state
after sync completes or fails because directories leave m_registered
before the next tick.

Fixes: https://tracker.ceph.com/issues/76686
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-23 20:49:19 +05:30
Kotresh HR
9f59332b35 tools/cephfs_mirror: Adds capability to persist metrics
Adds the capability to persist mirroring metrics.
The metrics are persisted in the omap of the cephfs-mirror
object. Metrics are persisted asynchronously.

Each mirrored directory path stores the corresponding
metrics as the value of a unique omap key representing
the mirrored directory. The omap key is as below.

sync_stat/<fsname>/<peer_uuid>/<mirrored_dir_path>

Fixes: https://tracker.ceph.com/issues/76686
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-23 20:49:19 +05:30
Kefu Chai
d0bf1d4987 Revert "script/run-make: enable ASan"
in 8ac962c6, we enabled ASan in script/run-make, which is part of
our CI workflow to build the tree. The goal was to enable us to identify
memory related issues early, but this introduced two kinds of problems:

- it's observed that some tests take around 4x time to complete in
  comparison to the test time without ASan enabled
- api and dashboard e2e tests are failing because the ASan supppression
  rules are not populated to them

This reverts commit 8ac962c698.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-23 20:32:05 +08:00
Matthew N. Heler
3a81a12608 rgw: restore constant-time GCM tag comparison in ISA-L path
a8ed43bfc0 replaced ct_memeq with memcmp in the ISA-L GCM accelerator,
making tag verification and the key-cache compare non-constant-time.
Restore ct_memeq for both; the OpenSSL and EVP paths already compare in
constant time.

Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
2026-06-23 07:02:05 -05:00
Matthew Heler
57e0250f57 rgw: fix AES-256-GCM key/IV reuse on multipart part re-upload
Re-uploading the same part number in a GCM multipart upload encrypted the new
data under the same key and IV as the first upload, since the IV is
part_number||chunk_index and the part key came from the part number alone. GCM
requires a unique IV per key; reusing one to encrypt different data weakens its
confidentiality and integrity guarantees.

Generate a random 16-byte salt on each UploadPart and fold it into the part key,
HMAC(ObjectKey, BE32(part) || salt), so every upload gets a fresh key. The salt
rides RGWUploadPartInfo, and complete stores the selected part's salt in
RGW_ATTR_CRYPT_PART_NUMS, which now holds (part, salt) pairs. GET reads it back
to re-derive the key, and an empty salt reproduces the old derivation so unsalted
parts still decrypt.

Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
2026-06-23 07:02:05 -05:00
Gil Bregman
77764dbe66 mgr/cephadm: Add degraded namespace flag to NVMEoF spec file
Fixes: https://tracker.ceph.com/issues/77556

Signed-off-by: Gil Bregman <gbregman@il.ibm.com>
2026-06-23 13:49:34 +03:00
Emmanuel Ameh
b2ecec6b3a doc/cephfs: clarify deprecated commands and remove obsolete upgrade path
Several CephFS pages described deprecated commands and features without
clear status or replacements, and one page documented an upgrade path that
predates every supported release.

* cephfs-mirroring: correct the peer_add deprecation note to reference the
  actual replacement command, peer_bootstrap create, and link the Bootstrap
  Peers section
* kernel-features, experimental-features: state that inline data has been
  deprecated since the Octopus release and that enabling it triggers a
  health warning
* upgrading: remove the "Upgrading pre-Firefly file systems past Jewel"
  section; the tmap_upgrade command it referenced was removed in Kraken

Fixes: https://tracker.ceph.com/issues/77188
Signed-off-by: Emmanuel Ameh <eameh@contractor.linuxfoundation.org>
2026-06-23 11:23:14 +01:00
Ilya Dryomov
b4f29b38df
Merge pull request #69659 from tchaikov/wip-dashboard-prettytable
mgr/dashboard: skip the table when an nvmeof cli result has no columns

Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Tomer Haskalovitch <tomer.haska@ibm.com>
2026-06-23 12:12:38 +02:00
Emmanuel Ameh
72265c8800 doc/rados/operations: refine cache-tiering deprecation note per review
Reduce the note to point existing users to the removal procedure, add a
ref target for the removal section, and rephrase the dm-cache mention to
reflect community adoption without implying official endorsement.
2026-06-23 10:34:33 +01:00
Emmanuel Ameh
6062ff880b doc/install: restore cephadm Octopus version floor note
Reverts the removal of the minimum version bullet per review feedback.
The Rook version line and ceph-ansible block removals are kept as-is.
2026-06-23 10:18:29 +01:00
eameh-LF
d96d022259
Merge pull request #69357 from eameh-LF/wip-doc-77197
doc/install: Update mirrors.rst to use https and current release
2026-06-23 10:05:43 +01:00
Alex Ainscow
51d8c5c489 osd/ECTransaction: fix truncate+write planning for EC shard sizes
There are multiple problems fixed here, which are caused by operations
which perform a truncate-then-write in a single transaction.

NOTE: The only known scenario for these operations are CLS-sparsify operations.
I recommend backporting these changes to Tentacle, however, as they are
regressions in the RADOS API which can lead to data corruption.

PROBLEM: Cache not invalidated correctly:
If projected_size >= orig_size, the invalidate_cache flag is not set in the plan
This means there is potentially data in the RMW extent cache, which
may cause data corruption in subsequent writes (although not the write
being made). No actual use case for this operation is known, so it may not be as serious as
it sounds.

FIX: Invalidate cache on any truncate or "delete_first"

PROBLEM: Parity writes permitted with truncates:
Currently parity delta writes do not support operations which may have
truncates. However, if the object ended up the same size as pre-truncate, a
parity delta write may have been attemtped. This may lead to data corruption.

FIX: Block parity writes in this case.
NOTE: It is not worth the development effort to support PDW in this scenario, as
performance benefit would be minimal overall.

PROBLEM: RMW Reads can occur beyond first truncate point.
Such reads reflect the pre-truncate data (which is incorrect) and be
preserved, even if the invalidate cache flag is set (since the cache is
invalidated before the reads). This can lead to similar corruption as
the earlier "cache not invalidated"

FIX: trim the read to the lower truncate size.

PROBLEM:
When performing a truncate, the parity shards may need to be updated to
reflect truncates on other shards. These truncate writes in the plan were
overriding, rather than adding to, other writes in the op.  The result was
a potentially truncated coding shard.  This leads to assertions on reads.

FIX: Replace = with insert()

PROBLEM: Some shards not set to correct size on truncate-then-write
If projected_size is smaller or equal to the original size, then the
code which attempts to correctly size a shard will not run.  However if
an operation performs a truncate then a partial write and does not
write to a particular shard AND the shard ends up smaller, then this
can lead to an incorrectly sized shard. This leads to assertion on reads.

FIX: Execute the shard-resize code on all truncates.

AI assistance was mainly used to write unit test. However, I cannot rule out a
contribution to the simple fixes found in this commit, so out of caution,
I place the Assisted-by tag.

Fixes: https://tracker.ceph.com/issues/77276

Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>
Assisted-by: IBM-Bob:ClaudeSonnet/GPT
2026-06-23 09:06:35 +01:00
Kefu Chai
9dd7fd0b0a mgr/dashboard: skip the table when an nvmeof cli result has no columns
The dashboard leaves prettytable unpinned.  prettytable commit 2574492 ("Apply
some Pylint rules (PLR)", #436) rewrote _stringify_row()'s row_height as
`max(_get_size(c)[1] for c in row)`, which raises ValueError("max() iterable
argument is empty") on a row with no cells.  The change is undocumented and
shipped in 3.18.0; get_string() trips on it when a table has a row but no
columns.

AnnotatedDataTextOutputFormatter builds such a table for an empty result, or
one whose only field is status or error_message, so NvmeofCLICommand.call()
returns -EINVAL and the command fails.  This broke run-tox-mgr-dashboard-py3
once the tox virtualenv picked up prettytable 3.18.0.

Return an empty string when there are no columns instead of formatting a
degenerate table.

Fixes: https://tracker.ceph.com/issues/77589
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-23 15:45:41 +08:00
anrao19
18fd73a5cd
Merge pull request #68194 from adamemerson/wip-rgw-multi-reshard-generation-recovery
rgw/multi: Recover from future generation sync
2026-06-23 10:43:45 +05:30
Kautilya Tripathi
9595f380aa
Merge pull request #69344 from knrt10/ceph-perf-pr
crimson/cbt: use yaml.safe_load in t2c and add unit tests
2026-06-23 10:30:10 +05:30
Sun Yuechi
640bea831a test/encoding: disable LeakSanitizer in readable.sh and check-generated.sh
readable.sh forks two short-lived ceph-dencoder processes per corpus
object(tens of thousands of processes overall). With ASAN's
LeakSanitizer enabled each process runs a whole-heap leak check at exit
that dominates runtime (~10s vs ~1s per process, measured on riscv64).
This test only validates encode/decode behaviour, not leaks, so disable
leak detection while keeping ASAN's other checks active.

check-generated.sh forks ceph-dencoder the same way, so apply the same
fix there.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-23 09:41:25 +08:00
Kefu Chai
32be6d184a
Merge pull request #69596 from tchaikov/wip-rocksdb
rocksdb: update submodule to fix FTBFS due to missing <cstdint>

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2026-06-23 08:14:08 +08:00
Kefu Chai
1a946cf36b
Merge pull request #69629 from tchaikov/wip-doc-min-compat
doc/rados/operations: document the kernel-client min-compat holdout

Reviewed-by: Anthony D'Atri <anthony.datri@gmail.com>
2026-06-23 08:11:40 +08:00
Kefu Chai
ca44481cdf
Merge pull request #69622 from sunyuechi/boost
cmake: define BOOST_USE_UCONTEXT tree-wide under ASan

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2026-06-23 07:52:23 +08:00
Adam C. Emerson
5bc34f8ed8
qa/rgw: Remove 'force-branch' from s3tests configs
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2026-06-22 19:33:47 -04:00
Adam C. Emerson
d0c88cb53c
qa/rgw: Run s3-tests from within the Ceph repo
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2026-06-22 19:33:47 -04:00
Adam C. Emerson
868d20182a
test/rgw: Include s3-tests in Ceph repo
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2026-06-22 19:33:47 -04:00
Alexander Indenbaum
048d2534a6 cephadm: disable/enable NVMe-oF gateways during host maintenance
Automatically run nvme-gw disable before stopping daemons on maintenance
enter and nvme-gw enable after daemons restart on maintenance exit so
gateways fail over in a controlled way.

Signed-off-by: Alexander Indenbaum <aindenba@redhat.com>
2026-06-22 22:04:10 +03:00
Yonatan Zaken
1651ec9a0f ceph.in: reject -w/--watch flags when used with subcommands
parse_known_args() silently consumes -w and all --watch-* flags
before subcommand arguments reach the command validator. When a
user ran "ceph orch ps -w", the watch handler entered cluster log
streaming mode while discarding the subcommand entirely, with no
error or indication that the orch ps command was ignored.

Add a guard at the entry of the watch handler: if any watch flag
is active and childargs is non-empty, print "Invalid command:
unused arguments: [...]" and return EINVAL. This matches the
behaviour already seen when -s/--status is combined with a
subcommand.

Assisted-by: Claude:claude-4.6-sonnet
Fixes: https://tracker.ceph.com/issues/77121
Signed-off-by: Yonatan Zaken <yzaken@redhat.com>
2026-06-22 20:02:39 +03:00
Ilya Dryomov
d66d996ea8
Merge pull request #69641 from ceph/ljflores-patch-1
doc: update email address for Laura Flores

Reviewed-by: Gregory Farnum <gfarnum@redhat.com>
Reviewed-by: Dan van der Ster <dan.vanderster@clyso.com>
Reviewed-by: Casey Bodley <cbodley@redhat.com>
2026-06-22 18:58:47 +02:00
lizhipeng
aed96f7a2f rgw:adding bucket index Indexless check when bi list
fixes:https://tracker.ceph.com/issues/73861

Signed-off-by: lizhipeng <qiuxinyidian@gmail.com>
Co-authored-by: lizhipeng <lizhipeng@kylinos.cn>
2026-06-23 00:00:07 +08:00
Lumir Sliva
960b330d87 rgw/lc: report x-amz-expiration only for the current version
The x-amz-expiration response header describes the current-version
Expiration lifecycle action for an object. s3_expiration_header() instead
selected between the Expiration and the NoncurrentVersionExpiration rule
based on whether the object key carried an instance id:

  const LCExpiration& rule_expiration =
    (obj_key.instance.empty()) ? expiration : noncur_expiration;

This was wrong in two ways. OLH resolution writes the current version's
instance id into the key before the header is computed, so even a plain
GET/HEAD of the current version (no versionId) was treated as a
non-current request and reported a NoncurrentVersionExpiration rule. And
x-amz-expiration is not meant to surface NoncurrentVersionExpiration at
all. A versioned bucket whose only rule was NoncurrentVersionExpiration
therefore returned a bogus expiry-date and rule-id on the current object,
for both a plain request and a versionId request.

Compute the header from the current-version Expiration rule only, and
return no header when the request targets a specific version. The header
helper now passes the client request key (s->object_key) rather than the
OLH-resolved key, so a plain GET/HEAD of the current version is no longer
misclassified as a versioned request.

Add a unit test covering a noncurrent-only policy (no header for either a
plain or a versioned request) and a combined policy (the current
Expiration rule for the current version, no header for a versionId
request).

Tracker: https://tracker.ceph.com/issues/77564
Reported-by: Saravanan Palanisamy
Signed-off-by: Lumir Sliva <61183145+lumir-sliva@users.noreply.github.com>
2026-06-22 17:28:45 +02:00
Kefu Chai
c900556800
Merge pull request #69621 from sunyuechi/wip-asan-ci-rebase
build,test: fix issues surfaced by tests after enabling ASan

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-06-22 23:25:36 +08:00
Laura Flores
af88cf4531
doc: update email address for Laura Flores
Changed Red Hat email to IBM.

Signed-off-by: Laura Flores <lflores@ibm.com>
2026-06-22 10:18:50 -05:00
Casey Bodley
591ce1234b qa/dnsmasq: remove unused backup/replace_resolv()
Signed-off-by: Casey Bodley <cbodley@redhat.com>
2026-06-22 10:34:50 -04:00
Casey Bodley
617f553913 qa/dnsmasq: use managed dnsmasq instead of editing resolv.conf
this task was manually editing /etc/resolv.conf to configure dns names.
on ubuntu 24, these changes were getting clobbered by NetworkManager or
systemd service restarts

instead, manage dnsmasq depending on host distro. rpm-based distros use
NetworkManager's dnsmasq plugin, while deb-based distros use
systemd-resolved to point at the local dnsmasq

Fixes: https://tracker.ceph.com/issues/76961

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2026-06-22 10:34:50 -04:00
Sagar Gopale
70a122f21b mgr/dashboard: fix-user-creation-validation
Fixes: https://tracker.ceph.com/issues/77024

Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-06-22 19:51:57 +05:30
Ali Masarwa
2866fcf20c
Merge pull request #69190 from RaminNietzsche/fix/rgw-cors-aws-rule-matching
RGW: match CORS rules like Amazon S3

reviewed-by: Ali Masarwa <amasarwa@redhat.com>
2026-06-22 17:16:25 +03:00
Nathan Hoad
9cce928885
Merge pull request #68713 from nhoad/remove-dead-gc-code
rgw: Remove GC deferred entries options and code
2026-06-22 10:00:11 -04:00
Sridhar Seshasayee
4d29f4e402 osd/PeeringState: add perf counters for PG rebuild times
Track per-OSD PG rebuild duration in prepare_stats_for_publish()
(primary OSD only). Three new counters are added to the
recoverystate_perf collection:

 - pg_rebuild_duration: LONGRUNAVG time counter (sum+count pair);
   This counter internally maintains 'avgcount' that tracks the
   cumulative number of rebuild events.
 - pg_rebuild_max_secs: maximum rebuild duration observed in seconds
 - pg_rebuild_min_secs: minimum rebuild duration observed in seconds

The logic uses a per-PG in-memory latch (rebuild_start_time) to
capture the redundancy failure entry point. When a PG is first observed
to be in a vulnerable state (PG_STATE_DEGRADED,
PG_STATE_UNDERSIZED, or num_objects_misplaced/degraded > 0), it
latches info.stats.last_change as the start time, provided
last_change > last_clean, which ensures only genuine new failures
after the prior clean interval are tracked.

On recovery, the rebuild duration is computed as (now - rebuild_start_time)
and is only recorded if delta num_objects_recovered > 0 or the PG had
confirmed redundancy loss at latch time, filtering out spurious state
transitions. The latch is cleared after each recorded event.

The latch is also cleared in clear_primary_state() so that an interval
change or role transition (primary -> replica) does not carry a stale
start time or baseline recovered count into a future interval.

The new last_degraded is intentionally not used here to retain compatibility
with older Ceph branches where the field doesn't exist and requires encoding
changes. A future simplification can replace the latch with a direct
(last_clean - last_degraded) calculation once last_degraded is consistently
available.

This interim solution is a close approximation of the PG rebuild time.

These counters are scraped per-OSD by ceph-exporter and exposed to
Prometheus, enabling durability score calculations over user-defined
time windows.

Other Changes:
1. Add unit tests to TestPeeringState.cc that exercise the latch logic.
2. Add a standalone integration test to verify that the rebuild perf counters
   are incremented on the primary OSD after a recovery event.

Fixes: https://tracker.ceph.com/issues/77493
Signed-off-by: Sridhar Seshasayee <sridhar.seshasayee@ibm.com>
2026-06-22 19:19:34 +05:30
Xuehan Xu
01c0602b1d crimson/os/seastore/lba: coroutinize LBACursor::refresh()
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-06-22 20:53:24 +08:00
Xuehan Xu
8ab6f707ce crimson/os/seastore/transaction_manager: refresh the indirect_cursor
after removing the direct_cursor in _remove()

Fixes: https://tracker.ceph.com/issues/77485
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-06-22 20:53:24 +08:00
Kefu Chai
002d35cbcc
Merge pull request #69601 from sunyuechi/wip-test-gtest-parallel-cache
cmake: make GTEST_PARALLEL_COMMAND visible to all test directories

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-06-22 18:54:32 +08:00
Vallari Agrawal
d970052faf
Merge pull request #69556 from VallariAg/refresh-network-cli
mgr/dashboard: Add "gw refresh_network" cmd
2026-06-22 15:38:40 +05:30
Afreen Misbah
bedc4d3bb8
Merge pull request #69396 from rhcs-dashboard/77298-fixing-EC-profile-creation-scroll
mgr/dashboard: Fix for EC profile creation modal scrollbar

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Abhishek Desai <abhishek.desai1@ibm.com>
2026-06-22 14:48:51 +05:30
Ilya Dryomov
6b040893f2
Merge pull request #69161 from sunyuechi/wip-librbd-pwl-cancel-timer-before-perf-stop
librbd/cache/pwl: cancel periodic_stats timer before perf_stop()

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2026-06-22 10:11:59 +02:00
Kefu Chai
d123f74bb2 doc/rados/operations: document the kernel-client min-compat holdout
A kernel CephFS or RBD mount can keep ``set-require-min-compat-client
reef`` from succeeding: the kernel client does not advertise the
pg-upmap-primary feature, so ``ceph features`` reports it as a luminous
client even when the daemons and every userspace client are newer.
ceph-fuse, libcephfs, and librbd advertise the full feature set and are
not affected.

Document this, and add a section on finding the holdout with
``ceph features`` and switching it to a userspace client.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-22 15:39:55 +08:00
Sun Yuechi
51a82211ed qa/lsan.supp: suppress seastore reactor-teardown leaks
unittest-seastore runs the seastar reactor on a separate thread
(SeastarRunner) and stops it at exit without draining its pending
tasks, so a few cached extents those tasks still held are leaked at
shutdown. Suppress them by the three seastore subsystems they come from.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-22 14:52:57 +08:00
Sun Yuechi
e35cdb7123 qa/lsan.supp: suppress still-reachable third-party leaks
Several unittests fail LeakSanitizer on still-reachable allocations that
belong to third-party libraries, not to Ceph. These are boost.thread's
main-thread TLS, OpenSSL's one-time init (the ForkDeathTest children
_exit() before it is freed), and the cipher, DRBG and error-stack state
that OpenSSL and libcryptsetup keep behind the librbd encryption and
migration unittests. None get freed without OPENSSL_cleanup(), so suppress
them by their allocation entry points.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-22 14:52:22 +08:00
Sun Yuechi
0a4edc5213 vstart: load lsan/asan suppressions on WITH_ASAN builds
AddCephTest.cmake runs unittests with
ASAN_OPTIONS/LSAN_OPTIONS=suppressions=qa/{asan,lsan}.supp, but vstart.sh
does not, so on a WITH_ASAN build `ceph-mon --mkfs` aborts on a still-reachable
leak that those suppressions cover and fails the "ceph API tests" job. Export
the same options when WITH_ASAN=ON.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-22 13:41:04 +08:00
Sun Yuechi
5166ec6f06 cmake: define BOOST_USE_UCONTEXT tree-wide under ASan
Under WITH_ASAN Boost.Context is built ucontext-only, so consumers that
include its headers without linking Boost::context (e.g. libosd) were
still built for the fcontext backend and broke the link:

    mold: error: undefined symbol: boost::context::detail::make_fcontext

Define the backend tree-wide so every consumer agrees on it.

riscv64's ASan runtime mis-handles makecontext/swapcontext, so the
ucontext fiber backend reports false-positive heap-buffer-overflows on
fiber switch that can't be suppressed. So exclude it.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-22 13:26:12 +08:00
Kefu Chai
746019c6ed
Merge pull request #69611 from sunyuechi/wip-boost-asan-context-impl-feature
cmake/boost: load context Jamfile before passing context-impl to b2

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-06-22 13:24:08 +08:00
Sun Yuechi
3ba08f88ad test/encoding: probe setarch -R and drop the arch argument
readable.sh wraps ceph-dencoder with `setarch $(uname -m) -R` to disable
ASLR on ASan builds, but the arch-qualified form also sets the personality
to that arch, which fails where setarch can't (e.g. riscv64). Use bare
`setarch -R` to only clear ASLR, and probe it first so the script falls
back to running ceph-dencoder unwrapped.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-22 10:57:27 +08:00
Sun Yuechi
1089fa3a6f crimson: build seastar with the default allocator under ASan
With this build configured as RelWithDebInfo, seastar keeps its own
allocator instead of falling back to libc's. Under ASan that allocator
is called (via dlsym) before it is initialized and SIGSEGVs every
seastar/crimson unittest before main(). Define SEASTAR_DEFAULT_ALLOCATOR
under WITH_ASAN to keep seastar on the libc allocator.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-22 10:57:27 +08:00
Sun Yuechi
82162c04bb cmake: use the libc allocator for sanitizer builds
tcmalloc/jemalloc keep exporting the global operator new/delete even though
their malloc is shadowed by the sanitizer interceptor, so memory the sanitizer
allocated gets freed through tcmalloc and SIGSEGVs (e.g. seastar coroutine
frames). Force libc when WITH_ASAN is set.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-22 10:57:27 +08:00
Kefu Chai
ecd883a1b3 librbd: fix use-after-free releasing object map locks in deep copy
DeepCopyRequest::send_copy_object_map() holds the destination image's
owner_lock and image_lock (shared), issues an asynchronous
object_map->rollback(), and then releases the two locks through
m_dst_image_ctx.  The rollback callback drives the rest of the request to
completion (handle_copy_object_map() ... finish() -> put()) on other
threads.  That path needs image_lock, so it only proceeds once this method
releases it, and can then run to completion and free 'this' before the
following owner_lock.unlock_shared() executes, dereferencing the freed
DeepCopyRequest.

ASan reported this as a heap-use-after-free in
TestImageReplayer.StartReplayAndWrite: the object was freed on the finisher
thread (handle_copy_metadata() -> finish()) while the io_context thread was
still in send_copy_object_map().

The destination image ctx outlives the request, so operate through a local
reference to it and release the locks through that reference rather than
through 'this', as
operation/SnapshotRollbackRequest::send_rollback_object_map() already does.

Keep the explicit lock_shared()/unlock_shared() calls rather than scoped
std::shared_lock guards: unlike
operation/SnapshotRollbackRequest::send_rollback_object_map(),
send_copy_object_map() has four exit paths (send_copy_metadata(),
send_refresh_object_map(), finish(), and the object map rollback), each of
which must release the locks before running a different continuation, so
RAII guards would push that continuation out of the locked scope and read
less clearly than unlocking at each exit.

Fixes: https://tracker.ceph.com/issues/77551
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-22 08:25:53 +08:00
Kefu Chai
c5693b3cca
Merge pull request #69577 from tchaikov/wip-debian-copyright
debian/copyright: silence old-fsf-address-in-copyright-file

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2026-06-21 20:50:22 +08:00
Kefu Chai
9a7299c455 cmake,deb,rpm: exclude CI and test directories from installed modules
The mgr module install rule already excluded tests/*, but that glob only
dropped the files inside tests/ and left an empty tests/ directory in the
installed tree.  The ci/ directories, which hold Dockerfiles, e2e test
scripts, and cluster specs used only for upstream CI pipelines, were not
excluded at all and were shipped by ceph-mgr-dashboard and ceph-mgr-rook.

Change the install pattern from tests/* to tests so the whole directory
is pruned, and add a ci pattern.  Both patterns live in the exclude list
shared by every mgr module install rule (the module list, dashboard, and
rook), so this covers all of them in one place.

This supersedes the per-package excludes added in 8a2e0eb2fc and
d74b88500b, so drop them from ceph.spec.in and debian/rules.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-21 20:45:10 +08:00
Kefu Chai
fb5c994f92
Merge pull request #69579 from tchaikov/wip-build-no-ci-in-package
debian,rpm: exclude CI directories from mgr plugin packages 

Reviewed-by: John Mulligan <jmulligan@redhat.com>
2026-06-21 15:25:09 +08:00
benhanokh
d5f1477218 rgw/dedup: extend shard tiers, add overflow guard, and fix bucket filter
The dedup subsystem was limited to 512 MD5 shards with 10% headroom,
  capping effective operation at ~4 billion objects.
  - Raise MAX_MD5_SHARD from 512 to 2048 and add new shard tiers for
    1024 (2GB), 2048 (4GB) shards, supporting up to ~213 billion objects.
  - Increase headroom from 10% to 20% to improve hash table load factor.
  - Reject pools that exceed the maximum capacity with -EOVERFLOW
  - Apply bucket filter in collect_all_buckets_stats() so that filtered
    buckets do not inflate the object count used for shard sizing.
  - Update sizing table, comments, and documentation to reflect the new
    tiers and capacity limit.

Signed-off-by: benhanokh <gbenhano@redhat.com>
2026-06-21 05:48:03 +00:00
Dax Kelson
b11c0eaedc msg/async/rdma: don't abort on an invalid per-connection eventfd
The async RDMA messenger reports each connection's per-connection eventfd
(notify_fd) as its socket fd().  notify_fd can legitimately be -1 -- under
fd-table pressure eventfd() returns -1, and the base RDMAConnectedSocketImpl
constructor skips queue pair *and* eventfd creation entirely when
ms_async_rdma_cm=true (the cm path is only implemented by the iwarp socket
classes).  Either way an fd() == -1 socket reaches the success path of
Processor::accept and trips ceph_assert(socket.fd() >= 0) in
AsyncConnection::accept, or is fed to EventCenter::create_file_event() and
indexes file_events[-1] out of bounds.  Both abort the OSD under connection
load.

Make the messenger surface the failure instead of handing back a broken
socket:

- Initialize RDMAConnectedSocketImpl::qp to nullptr.  It was uninitialized and
  read via get_qp() when ms_async_rdma_cm=true skips its assignment.

- Create the notify eventfd before the queue pair in the base constructor, and
  adopt the cm id/channel before the eventfd in the iwarp constructor, so a
  failure leaks nothing and the two failure causes stay distinguishable.  Check
  the rdma_create_event_channel()/rdma_create_id() return values on the touched
  paths, and destroy the per-request cm id on the accept error paths (acking the
  event does not free it).

- At each accept/connect construction site, reject a socket with fd() < 0
  (-EMFILE) or a null queue pair (-EIO) instead of returning it.  Free the
  half-built socket on its owning worker thread via center.submit_to() so
  ~RDMAConnectedSocketImpl's center.in_thread() assertion holds and no RDMA/fd
  state leaks (this also fixes the pre-existing leak on the rdma_accept()
  failure path).

- Require ms_async_rdma_cm and ms_async_rdma_type=iwarp to be enabled together
  in RDMAWorker::listen()/connect() (-EOPNOTSUPP with a clear message).  The
  two are coupled -- only the iwarp classes implement rdma_cm and they always
  use it -- so any mismatch leaves a socket half-built and would otherwise
  crash-loop the daemon.

- Guard EventCenter::create_file_event() against a negative fd as defense in
  depth.

Fixes: https://tracker.ceph.com/issues/77547

Signed-off-by: Dax Kelson <daxkelson@gmail.com>
2026-06-20 21:03:25 -06:00
Sun Yuechi
61b21c04f4 mgr/MMgrReport: default-initialize PerfCounterType members
readable.sh can fail decoding pre-v3 corpus objects that predate the
unit field, e.g.:

  **** reencode of .../objects/PerfCounterType/... resulted in a different dump ****
  7c7
  <     "unit": 36
  ---
  >     "unit": 101

For struct_v < 3 unit is never decoded and had no in-class default, so
it read indeterminate memory and differed between the two ceph-dencoder
processes readable.sh compares. Default it to UNIT_NONE.

Also default-initialize type to PERFCOUNTER_NONE for consistency.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-21 09:21:13 +08:00
Kefu Chai
767e1b6514
Merge pull request #69316 from sunyuechi/wip-tests-venv-system-site-packages
test: optionally run test venvs with system site-packages

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-06-21 07:14:12 +08:00
Sun Yuechi
a764ecf720 monitoring/ceph-mixin: make jsonnet-bundler clone idempotent
A run that aborts before the cleanup fixture runs (timeout, OOM,
interrupt) leaves the clone dir behind, so the next build fails with
"destination path already exists". Remove it first.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-20 23:22:01 +08:00
Anthony D'Atri
fc400429fa
Merge pull request #69600 from anthonyeleven/improve-onode
doc/rados/bluestore: Improve fast-onode-scan.rst
2026-06-20 07:54:48 -04:00
Sun Yuechi
5d82294f3f cmake/boost: load context Jamfile before passing context-impl to b2
With WITH_ASAN, b2 runs as `b2 context-impl=ucontext headers stage` for the
build step and `b2 context-impl=ucontext install` for the install step. The
`context-impl` feature is declared in libs/context/build/Jamfile.v2, which
neither the headers/stage nor the install targets load, so b2 aborts with:

    error: unknown feature "<context-impl>"

Name the context project as a target in both commands so its Jamfile (and the
feature) loads before the build request is expanded.

This works around https://github.com/boostorg/context/issues/297, fixed
upstream in
12ac945158
and first released in Boost 1.88; drop it once the bundled Boost is bumped to
1.88 or newer.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-20 15:41:11 +08:00
Sun Yuechi
3652faa3fb test/cmake: drop dead env_vars_for_tox_tests set_property
Both `tox_tests` and `env_vars_for_tox_tests` have been undefined since
f0079a1030, so this expands to `set_property(TEST PROPERTY ENVIRONMENT)`
-- a no-op. The actual per-test environment for tox tests is set in
add_tox_test() (cmake/modules/AddCephTest.cmake). Remove the leftover.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-20 14:49:16 +08:00
Sun Yuechi
ba5da131ec mypy: skip follow_imports for prettytable
With test venvs on system site-packages, mypy picks up the system
prettytable (3.4.0+, typed). It flags mgr's add_row(tuple) against the
list[Any] signature (src/mypy.ini) and qa's float_format = str against
the dict[str, str] property (qa/mypy.ini). Skip follow_imports in both.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-20 14:49:16 +08:00
Sun Yuechi
ae6bc8220b test: optionally run test venvs with system site-packages
Add a CEPH_PYTHON_SYSTEM_SITE switch (off by default). When set:

  - setup-virtualenv.sh builds its venv with --system-site-packages;
  - run_tox.sh exports VIRTUALENV_SYSTEM_SITE_PACKAGES=true for tox's venvs.

This lets distro packages satisfy test dependencies instead of pip building
them from sdist, which helps where prebuilt wheels are missing (e.g. scipy and
numpy on riscv64) by avoiding a slow rebuild when the RPMs are installed.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-20 14:49:16 +08:00
Kefu Chai
3f2efbaab9
Merge pull request #69603 from sunyuechi/wip-mgr-tox-pytest-xdist
mgr/tox: run pytest in parallel

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-06-20 12:37:34 +08:00
Ronen Friedman
465318cf0e
Merge pull request #69458 from ronen-fr/wip-rf-inc1406-crimson
crimson,crimson/test: clean-up 'unused' warnings

Reviewed-by: Shraddha Agrawal <shraddhaag@ibm.com>
2026-06-20 07:03:12 +03:00
Kefu Chai
e42e43a18b
Merge pull request #56537 from tchaikov/wip-cmake-enable-sanitizers
script/run-make: enable ASan

Reviewed-by: Bill Scales <bill_scales@uk.ibm.com>
2026-06-20 09:24:42 +08:00
Anthony D'Atri
0e0d2c6b2b doc/rados/bluestore: Improve fast-onode-scan.rst
Signed-off-by: Anthony D'Atri <anthonyeleven@users.noreply.github.com>
2026-06-19 17:37:23 -04:00
Sun Yuechi
0705c6d60b cmake: make GTEST_PARALLEL_COMMAND visible to all test directories
GTEST_PARALLEL_COMMAND was set as an ordinary variable inside the
`if(NOT TARGET gtest-parallel_ext)` guard, so it only existed in the
first directory that include()s AddCephTest (src/common/options). Later
includes skip the guarded block and leave it empty, so PARALLEL
unittests under src/test silently ran serially.

Promote it to CACHE INTERNAL so it is visible across all directories
regardless of include order.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-19 23:09:22 +08:00
Sun Yuechi
2aabf864f3 mgr/tox: run pytest in parallel
The py3 and coverage tox environments run the full mgr pytest suite
serially, which makes run-tox-mgr the longest test in CI. Add
pytest-xdist and pass `-n auto` to both so the suite is distributed
across the available CPUs.

pytest-xdist is constrained to <2 to stay compatible with the pinned
pytest-cov. Running in parallel also surfaced a hard-coded port in
cephadm's test_node_proxy, which now allocates an ephemeral port per
process.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-19 23:08:37 +08:00
Casey Bodley
0bdeb61ac7
Merge pull request #69134 from cbodley/wip-rgw-multi-delete-op_ret
rgw: use local error code in handle_individual_object()

Reviewed-by: Shilpa Jagannath <smanjara@redhat.com>
Reviewed-by: Mark Kogan <mkogan@redhat.com>
2026-06-19 09:21:10 -04:00
John Mulligan
f6d797ba17
Merge pull request #68840 from rapanigrahi/certmgr_smb
mgr/cephadm: Support cephadm certmgr with SMB/TLS configuration

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
2026-06-19 08:50:37 -04:00
Kefu Chai
9df2719e63 rocksdb: update submodule to fix FTBFS due to missing <cstdint>
59afb3d6 bumped rocksdb submodule in hope to address the FTBFS failure
when building rocksdb with GCC 16, but the tree still failed to build:

```
In file included from /ceph/src/rocksdb/include/rocksdb/trace_record_result.h:14,
                 from /ceph/src/rocksdb/trace_replay/trace_record_result.cc:6:
/ceph/src/rocksdb/include/rocksdb/trace_record.h:55:32: error: expected ')' before 'timestamp'
   55 |   explicit TraceRecord(uint64_t timestamp);
      |                       ~        ^~~~~~~~~~
      |                                )
/ceph/src/rocksdb/include/rocksdb/trace_record.h:63:11: error: 'uint64_t' does not name a type
   63 |   virtual uint64_t GetTimestamp() const;
      |           ^~~~~~~~
/ceph/src/rocksdb/include/rocksdb/trace_record.h:1:1: note: 'uint64_t' is defined in header '<cstdint>'; this is probably fixable by adding '#include <cstdint>'
```

in this change, we cherry-pick upstream fix to address this build
failure.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-19 19:22:44 +08:00
Kefu Chai
8ac962c698 script/run-make: enable ASan
when performing tests, we should enable sanitizers for detecting
potential issues. so, in this change, we enable ASsan, TSan and
UBSan.

script/run-make.sh is used by our CI job for testing PRs, so
enabling these sanitizers helps us to identify issues as early as
possible. because ASan cannot be used along with TSan, we prefer
using ASan for capturing memory related issue in favor of
detecting the multi-threading issues.

also, because of https://bugs.llvm.org/show_bug.cgi?id=23272, we
cannot enable multiple sanitizers. but we should enable UBSan as well,
once we can use a higher version of Clang than Clang-14. with
Clang-14, when enabling UBSan, we'd have following FTBFS
```
error: Cannot represent a difference across sections
```
when compiling `src/tools/neorados.cc`

Signed-off-by: Kefu Chai <tchaikov@gmail.com>
2026-06-19 19:03:42 +08:00
Kefu Chai
1939c6dbc5
Merge pull request #69457 from tchaikov/wip-test-objectstore-fix-leaks
test/objectstore: hold split_blob cache shards in unique_ptr

Reviewed-by: Adam Kupczyk <akupczyk@ibm.com>
2026-06-19 19:03:14 +08:00
Soumya Koduri
5a3b87ac6f
Merge pull request #69373 from mheler/wip-rgw-restore-queue-shard-fix
rgw/restore: run shard hash through HASH_PRIME

Reviewed-by: Soumya Koduri <skoduri@redhat.com>
2026-06-19 16:27:00 +05:30
Ilya Dryomov
1ba7d25d0b
Merge pull request #68613 from tchaikov/wip-test-journal-fix-use-after-free
journal/ObjectPlayer: don't acquire locks in destructor

Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2026-06-19 11:42:42 +02:00
Ilya Dryomov
5b397048cc
Merge pull request #68161 from abitdrag/wip-miki-image-prim-snap-removal
rbd-mirror: prune obsolete primary mirror snapshots after relocation

Reviewed-by: Ramana Raja <rraja@redhat.com>
Reviewed-by: VinayBhaskar-V <vvarada@redhat.com>
Reviewed-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2026-06-19 11:38:14 +02:00
Guillaume Abrioux
daec7e125a
ceph-volume: skip internal raid mirror LVs in inventory
ceph-volume inventory started including all LVM mapper devices after
c06bee965f. On hosts with raid mirrored system volumes, that pulls in
hidden legs like var_rmeta_0 which have no /dev/vg/lv node and makes
cephadm's ceph-volume inventory call fail.

Skip those internal LVs in get_devices() and avoid rewriting the device
path to a missing lv_path in Device._parse().

Fixces: https://tracker.ceph.com/issues/77486

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2026-06-19 08:27:04 +02:00
Venky Shankar
da8d57b981
Merge pull request #69531 from xiubli/lazyio
common/options: mark client_force_lazyio as not runtime updatable

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2026-06-19 09:32:53 +05:30
naman munet
71f19d3f07
Merge pull request #69090 from rhcs-dashboard/76795-support-wildcard-sans
mgr/dashboard : Support wildcard sans and zonegroup hostnames

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Abhishek Desai <abhishek.desai1@ibm.com>
Reviewed-by: Naman Munet <nmunet@redhat.com>
2026-06-19 09:14:00 +05:30
Ilya Dryomov
dc130970dd
Merge pull request #69414 from Adarsha1999/teuthology-smoke-log-ignorelist
qa/smoke/basic: add log-ignorelist entries for expected smoke test warnings

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2026-06-19 00:30:09 +02:00
Casey Bodley
8f057a9ec2
Merge pull request #69474 from sunyuechi/wip-rgw-a-declare-schedulers-kmip-deps
rgw: declare rgw_a's dependency on rgw_schedulers and kmip

Reviewed-by: Casey Bodley <cbodley@redhat.com>
2026-06-18 13:34:29 -04:00
naman munet
2e767ce96d
Merge pull request #69393 from rhcs-dashboard/wip-rgw-roles-account-scoping
mgr/dashboard: align RGW role management with Carbon and fix API routing

Reviewed-by: Naman Munet <nmunet@redhat.com>
Reviewed-by: Nizamudeen A <nia@redhat.com>
Reviewed-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-06-18 22:38:25 +05:30
Adarsha Dinda
517c72f659 qa/smoke/basic: add log-ignorelist entries for expected smoke test warnings
Ignore FS_DEGRADED in libcephfs_interface_tests and MON_DOWN in
rbd_python_api_tests to avoid false failures from transient cluster health
messages.

Signed-off-by: Adarsha Dinda <adarshadinda@Adarshas-MacBook-Pro.local>
2026-06-18 22:29:30 +05:30
Abhishek Desai
eee6a15dcd mgr/dashboard : Support wildcard sans and zonegroup hostnames
fixes : https://tracker.ceph.com/issues/76795
Signed-off-by: Abhishek Desai <abhishek.desai1@ibm.com>
2026-06-18 21:29:39 +05:30
Rabinarayan Panigrahi
f02b985bd8 python-common/ceph/smb: Add smb features constants for SSL
Add smb constants to support for ca_certificate validation and string
alteration

Signed-off-by: Rabinarayan Panigrahi <rapanigr@redhat.com>
2026-06-18 21:09:36 +05:30
Rabinarayan Panigrahi
6f598d7513 mgr/cephadm: Registering smb features for ssl certificate with certmgr
Here we are registering smb features such remote_control and keybridge
with certificate manager and can be list with ceph orch certmgr bindings
ls

Signed-off-by: Avan Thakkar <athakkar@redhat.com>
Signed-off-by: Rabinarayan Panigrahi <rapanigr@redhat.com>
2026-06-18 21:09:36 +05:30
Rabinarayan Panigrahi
dbf3221479 mgr/cephadm/services: Add supported function for ssl certificates
Added function for get smb features TLSCredentials using _get_feature_certs

Added function _get_certificates_from_spec_ssl_certificates to get the
certificates for ssl_certificates

Updating features ssl certificates to default path of containers

Signed-off-by: Rabinarayan Panigrahi <rapanigr@redhat.com>
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2026-06-18 21:04:46 +05:30
Sun Yuechi
8a35d17c72 librbd/cache/pwl: cancel periodic_stats timer before perf_stop()
AbstractWriteLog::shut_down() calls perf_stop(), which deletes
m_perfcounter, but the 5s periodic_stats timer is only canceled later
in the destructor. If it fires in between, periodic_stats ->
update_image_cache_state dereferences the freed m_perfcounter. Cancel
the timer under m_timer_lock first.

Destroying an AbstractWriteLog that was init()'ed without a matching
shut_down() is illegal, so the cancel_event() in the destructor is now
redundant and is dropped.

Fixes: https://tracker.ceph.com/issues/77501

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-18 22:53:53 +08:00
David Galloway
847b84d955
Merge pull request #69555 from djgalloway/wip-pulp-container
Containerfile: Support pulp repo URLs
2026-06-18 10:19:59 -04:00
Kefu Chai
8a2e0eb2fc ceph.spec.in: exclude CI and test directories from mgr plugin packages
The ceph-mgr-dashboard and ceph-mgr-rook packages install their entire
mgr module directory, which includes a ci/ subdirectory containing
Dockerfiles, e2e test scripts, and cluster specs used only for upstream
CI pipelines.  ceph-mgr-cephadm similarly ships a tests/ directory with
Python unit tests.  These files have no runtime purpose on a deployed
system and should not be shipped in the binary packages.

Exclude mgr/cephadm/tests, mgr/dashboard/ci, and mgr/rook/ci via
%exclude directives.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-18 22:15:31 +08:00
Kefu Chai
d74b88500b debian/rules: exclude CI and test directories from mgr plugin packages
The ceph-mgr-dashboard and ceph-mgr-rook packages install their entire
mgr module directory, which includes a ci/ subdirectory containing
Dockerfiles, e2e test scripts, and cluster specs used only for upstream
CI pipelines.  ceph-mgr-cephadm similarly ships a tests/ directory with
Python unit tests.  These files have no runtime purpose on a deployed
system and should not be shipped in the binary packages.

Exclude mgr/cephadm/tests, mgr/dashboard/ci, and mgr/rook/ci via
dh_install --exclude.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-18 22:15:31 +08:00
Casey Bodley
c99131fa29
Merge pull request #69425 from cbodley/wip-76995
Reapply "qa/rgw/crypt: disable failing kmip testing"

Reviewed-by: J. Eric Ivancich <ivancich@redhat.com>
2026-06-18 10:10:59 -04:00
Rabinarayan Panigrahi
a3aa693987 mgr/cephadm: add ca_cert_required parameter to get_certificates mock in test
Add the ca_cert_required parameter to the lambda function mocking
CephadmService.get_certificates in test_prometheus_config_security_enabled
to match the updated method signature.

This ensures the test mock properly handles the new parameter that was
added to the get_certificates method.

Signed-off-by: Rabinarayan Panigrahi <rapanigr@redhat.com>
2026-06-18 19:20:17 +05:30
Rabinarayan Panigrahi
33c9e66cea doc/cephadm: Add SMB TLS/SSL configuration and examples
Add SMB TLS/SSL configuration with example for SMB features
remote_control and keybridge

Signed-off-by: Rabinarayan Panigrahi <rapanigr@redhat.com>
2026-06-18 19:20:17 +05:30
Rabinarayan Panigrahi
035b96985e mgr/cephadm: Add test case from smb spec and validating functionality
with test

Here we have added a test case to test_smb.py to validate functionality
of certificate manager after apply certificates.

Signed-off-by: Rabinarayan Panigrahi <rapanigr@redhat.com>
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2026-06-18 19:20:17 +05:30
Rabinarayan Panigrahi
38105f7015 mgr/cephadm: Test case are updated to validate for ssl certificate for
smb services

Signed-off-by: Rabinarayan Panigrahi <rapanigr@redhat.com>
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2026-06-18 19:20:17 +05:30
Rabinarayan Panigrahi
66fc26d43b mgr/smb: Add ssl_certificates buffer for smb features
Add ssl_certificates buffer for smb features like remote_control
and keybridge. when certificate applied it stores as a feature
name and SSLParameters as value where SSLParameters holds cert,
key and ca-cert.

Signed-off-by: Rabinarayan Panigrahi <rapanigr@redhat.com>
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2026-06-18 19:20:17 +05:30
Rabinarayan Panigrahi
362580714d mgr/smb: Add is_feature_enabled function to resource class
This funality added to find if feature like remote_control
and keybride is enable.

Signed-off-by: Rabinarayan Panigrahi <rapanigr@redhat.com>
2026-06-18 19:20:17 +05:30
Rabinarayan Panigrahi
ee6fd847b0 mgr/cephadm: Add function to get ssl certificate from ssl_certificates
A function is added _get_certificates_from_spec_ssl_certificates to get
certificates from ssl_certificates buffer. it returns TLSCredentials from
SSLParameters of ssl_certificates[feature]

Signed-off-by: Rabinarayan Panigrahi <rapanigr@redhat.com>
Signed-off-by: Avan Thakkar <athakkar@redhat.com>
2026-06-18 19:20:17 +05:30
Rabinarayan Panigrahi
8202dae834 ceph/deployment: Adding smb to REQUIRES_CERTIFICATES list
Signed-off-by: Rabinarayan Panigrahi <rapanigr@redhat.com>
2026-06-18 19:20:17 +05:30
Rabinarayan Panigrahi
16553bfa23 mgr/cephadm: Add function all to get ssl certificates
A function call added to get ssl certificates from ssl_certificates
buffer. it returns TLSCredentials:

Signed-off-by: Rabinarayan Panigrahi <rapanigr@redhat.com>
2026-06-18 19:20:17 +05:30
Rabinarayan Panigrahi
176de7b903 ceph/deployment: Adding class SSLParameters for storing cert, key
and ca-cert for ssl and other features

Signed-off-by: Rabinarayan Panigrahi <rapanigr@redhat.com>
2026-06-18 19:18:02 +05:30
Sagar Gopale
1d3e33140b mgr/dashboard: align RGW role management with Carbon and fix API routing
Fixes: https://tracker.ceph.com/issues/77328

Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-06-18 18:30:47 +05:30
Kefu Chai
7e8c2e062f debian/copyright: silence old-fsf-address-in-copyright-file
in order to silence the warnings like:

```
W: rbd-nbd: old-fsf-address-in-copyright-file
```

let's refere to the email address of FSF, instead of its old postal
address.

See https://lintian.debian.org/tags/old-fsf-address-in-copyright-file.html

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-18 20:34:28 +08:00
Nizamudeen A
99b840735c
Merge pull request #69573 from rhcs-dashboard/fixe2e
mgr/dashboard: fix daemon e2e 

Reviewed-by: Nizamudeen A <nia@redhat.com>
2026-06-18 17:31:46 +05:30
Kefu Chai
a35a92639f journal/ObjectPlayer: don't acquire locks in destructor
~ObjectPlayer took m_timer_lock only to assert two invariants. but that
lock is borrowed by reference from the caller's SafeTimer, and an
ObjectPlayer can outlive it: a C_Fetch/C_WatchFetch completion on the
librados finisher may hold the last reference and run ~ObjectPlayer after
the timer and its lock are already gone. re-taking the freed lock is a
heap-use-after-free, which unittest_journal hits on arm64 under ASan:

  ==ERROR: AddressSanitizer: heap-use-after-free
    journal::ObjectPlayer::~ObjectPlayer  ObjectPlayer.cc:63
    journal::ObjectPlayer::C_WatchFetch::~C_WatchFetch
    journal::ObjectPlayer::C_Fetch::finish

the lock isn't needed though: at refcount 0 the watch has been cancelled
(m_watch_ctx == nullptr, asserted below) so no timer task references us, and
no fetch is in flight since a pending fetch holds a reference. nothing else
can touch our state. Furthermore, we can also skip acquiring `m_lock` as
well, because, in the destructor, it shouldn't really matter -- if one
of these asserts fails because the execution of the destructor races
with some `ObjectPlayer` mthod, we would get what the `assert()` was
added for. They are here to catch bugs and such a race just being
possible is a bug in itself.

Fixes: https://tracker.ceph.com/issues/77496
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-18 19:42:49 +08:00
Shai Fultheim
d3cada2c85 test/crimson/seastore: assert omap extent retirement on split/merge/balance
There was no coverage for extent leaks or double-frees during structural
omap B-tree transitions. While investigating a suspected extent leak, these
assertions were added and confirmed the caller-layer (parent-node) already
retires nodes correctly — no leak and no double-free.

Add count_live_omap_extents() to omap_manager_test_t, which scans the LBA
mapping space via scan_mapped_space() to count live OMAP_INNER/OMAP_LEAF
extents, and assert against it in leafnode/innernode_split_merge_balancing:
the live count returns to baseline after contraction, and is exactly 0 after
clearing the tree. This pins the caller-layer (parent-node) retirement
contract in CI.

Signed-off-by: Shai Fultheim <shai.fultheim@gmail.com>
2026-06-18 13:55:49 +03:00
mheler
247c99ad4e
Merge pull request #68978 from mheler/mon-backup
mon: implement mon backup mechanism
2026-06-18 05:11:13 -05:00
Xuehan Xu
f08533c487 crimson/os/seastore/lba: don't update LBALeafNode::modifications when
updating the lba mapping for committing rewrite transactions

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-06-18 17:07:20 +08:00
Xuehan Xu
5d52c07590 crimson/os/seastore/lba: don't update the lba mapping if its child is an
initial pending one.

Because initial pending extents' paddrs are considered newer.

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-06-18 17:07:20 +08:00
Xuehan Xu
3d174f6b40 crimson/os/seastore/lba: no need to pass the reference to the
transaction when synchronously updating the lba mapping

Because the transaction can be captured by the functor itself

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-06-18 16:45:35 +08:00
Kefu Chai
e0efa39663
Merge pull request #69521 from sunyuechi/warnings-cephfs-int32
pybind/cephfs: drop redundant int32_t typedef

Reviewed-by: Rishabh Dave <ridave@redhat.com>
Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-06-18 16:17:10 +08:00
Naman Munet
268a857803 mgr/dashboard: fix daemon e2e
fixes: https://tracker.ceph.com/issues/77489

Signed-off-by: Naman Munet <naman.munet@ibm.com>
2026-06-18 13:40:34 +05:30
Vallari Agrawal
955d7e68ea
mgr/dashboard: Add "gw refresh_network" cmd
Fixes: https://tracker.ceph.com/issues/77442

Signed-off-by: Vallari Agrawal <vallari.agrawal@ibm.com>
2026-06-18 13:22:07 +05:30
Matan Breizman
4bef6e680a
Merge pull request #69076 from xxhdx1985126/wip-seastore-paddr-cross-coroutine-fix
crimson/os/seastore/lba, TM: prevent of the users of LBACursor/LBAMapping from pass the paddr/lba_map_val_t across the boundaries of coroutines

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2026-06-18 10:27:38 +03:00
Rishabh Dave
7f1f0e9eb2 PendingReleaseNotes: add note for mutability of CephFS snapshot metadata
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2026-06-18 11:57:45 +05:30
Rishabh Dave
df7abb7a7b test_cephfs.py: add tests for do_snap_md_op()
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2026-06-18 11:57:45 +05:30
Rishabh Dave
8d5cf70ba6 pybind/cephfs: add python binding for ceph_do_snap_md_op()
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2026-06-18 11:57:45 +05:30
Rishabh Dave
446bd6af11 tests/libcephfs: add tests for snap metadata mutations
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2026-06-18 11:57:45 +05:30
Rishabh Dave
5726edbcf3 libcephfs: provide API to mutate snapshot metadata
Fixes: https://tracker.ceph.com/issues/66293
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2026-06-18 11:57:45 +05:30
Rishabh Dave
a7617f354e mds: add MDS code to allow snap metadata mutations
Fixes: https://tracker.ceph.com/issues/66293
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2026-06-18 11:57:45 +05:30
Rishabh Dave
047f41aac1 mds: rename a finisher method in Server.cc since...
it will be used in a different context as well in the upcoming commit.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2026-06-18 11:57:45 +05:30
Rishabh Dave
2aed10be29 mds: enable logging for snap.cc
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2026-06-18 11:57:45 +05:30
Rishabh Dave
bf5080c08c client: add client code to allow snap metadata mutations
Fixes: https://tracker.ceph.com/issues/66293
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2026-06-18 11:57:39 +05:30
Xiubo Li
a2f175f677 common/options: mark client_force_lazyio as not runtime updatable
The client_force_lazyio option is currently marked as supporting
runtime updates in the configuration schema, but this is misleading.

The value is read once during each file open/create and stored in
the file flags. There is no config observer registered to handle
dynamic updates, and there is no logic to propagate changes to the
already opened file handles.

This patch adds the NO_RUNTIME flag to the option definition to
correctly reflect reality.

Fixes: https://tracker.ceph.com/issues/77451
Signed-off-by: Xiubo Li <xiubo.li@clyso.com>
2026-06-18 14:19:10 +08:00
Kotresh HR
2fca3c3538
Merge pull request #69074 from kotreshhr/mirror-metrics-counter-dump
tools/cephfs_mirror: Expose Mirror metrics via asok counter dump

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2026-06-18 09:40:55 +05:30
Matthew N. Heler
83189af8e6 rgw/lc: Warn against changing rgw_lc_max_objs once lifecycle is in use
Each bucket's lifecycle entry is hashed onto a shard by this count, so
changing it remaps existing entries and can leave duplicates behind.

Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
2026-06-17 21:19:08 -05:00
Matthew N. Heler
3a9ae41e2a mon: add monitor RocksDB backup and restore
Implements an opt-in backup mechanism for the monitor using
rocksdb::BackupEngine. Backups run on a schedule when
mon_backup_interval is set, or are triggered manually via
`ceph tell mon.* backup`. Cleanup keeps the last N, hourly,
and daily snapshots, with a free-space guard. Off by default.

Restore is offline: stop the mon and run
  ceph-mon --restore-backup <dir> --yes-i-really-mean-it
optionally with --backup-version (BackupEngine logical version,
as shown by --list-backups). The mon keyring is stashed alongside
the RocksDB backup so a wiped mon_data is recovered end-to-end,
and kv_backend is stamped back when missing.

Co-authored-by: Daniel Poelzleithner <poelzleithner@b1-systems.de>
Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
2026-06-17 19:24:52 -05:00
David Galloway
a803d372ad Containerfile: Support pulp repo URLs
Signed-off-by: David Galloway <david.galloway@ibm.com>
2026-06-17 14:25:04 -04:00
Ronen Friedman
e5c20fa6a8 crimson,crimson/test: clean-up 'unused' warnings
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2026-06-17 17:22:22 +00:00
Kefu Chai
d814088ce7 python-common/cryptotools: reimplement on top of cryptography
the last change dropped X509Req, but InternalCryptoCaller still uses
pyOpenSSL, which keeps deprecating and removing its OpenSSL.crypto and
OpenSSL.SSL APIs. pyOpenSSL's own README [1] says:

  If you are using pyOpenSSL for anything other than making a TLS
  connection you should move to cryptography and drop your pyOpenSSL
  dependency.

none of this module makes a TLS connection, and we already depend on
cryptography and use it in cephadm/ssl_cert_utils.py.

so reimplement the caller with cryptography:

- create_private_key(): rsa.generate_private_key() dumped as PKCS8 PEM,
  the same format pyOpenSSL produced.
- create_self_signed_cert(): x509.CertificateBuilder, with the dname
  fields mapped to their NameOIDs.
- _load_cert() and get_cert_issuer_info(): load_pem_x509_certificate()
  and cert.issuer.
- certificate_days_to_expire(): cert.not_valid_after.
- verify_tls(): compare the cert's and the key's public keys instead of
  loading them into an SSL.Context.

password_hash() and verify_password() already used bcrypt, so they are
left as is. the output formats and error behavior are unchanged, hence
the existing tls tests pass without changes.

[1] https://github.com/pyca/pyopenssl/blob/main/README.rst

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-18 00:07:18 +08:00
Vallari Agrawal
e4cf2f0c84
Merge pull request #69511 from VallariAg/wip-update-submodule-182
mgr/dashboard: bump nvmeof submodule to 1.8.2
2026-06-17 21:14:50 +05:30
Ilya Dryomov
1b51e11d25
Merge pull request #69356 from eameh-LF/wip-doc-77200
doc: Replace Python 2 package names with Python 3 equivalents

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
2026-06-17 16:21:28 +02:00
Matan Breizman
c6741c101d
Merge pull request #69275 from fultheim/seastore-io-wakeup
crimson/os/seastore: wake blocked IO on BackgroundProcess wakeup

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2026-06-17 16:56:01 +03:00
Oguzhan Ozmen
7d3cb2c34d
Merge pull request #67141 from BBoozmen/wip-oozmen-74677
rgw/multisite: Balance sync traffic across DNS-resolved backends using CURLOPT_CONNECT_TO
2026-06-17 09:52:19 -04:00
Emmanuel Ameh
a3a347e6d1 doc/rados/operations: Strengthen cache-tiering deprecation notice
The existing warning hedged with "this does not mean it will certainly
be removed", softening the message for a deprecated feature. Operators
scanning the page may begin a how-to setup without reading past the
introductory prose.

- Remove the hedging language; state plainly that cache tiering may be
  removed without further notice
- Add a bold "Do not deploy new cache tiers" directive
- Add a note immediately before the how-to content scoping the page
  to existing deployments only, with a pointer to preferred alternatives

Fixes: https://tracker.ceph.com/issues/77184

Signed-off-by: Emmanuel Ameh <eameh@contractor.linuxfoundation.org>
2026-06-17 14:35:05 +01:00
Emmanuel Ameh
be52c92ace doc/rados/configuration: Fix FileStore contradiction in journal-ref.rst
The page carried a deprecation warning on line 4 stating FileStore
"is no longer supported" but line 10 said "Filestore is preferred for
new deployments" -- a direct contradiction. Fix by:

- Expanding the warning to explicitly state FileStore must not be used
  for new deployments and to use BlueStore instead
- Updating the intro paragraph to reflect that BlueStore is the
  default and recommended back end, and that this page is for
  pre-existing Filestore OSDs pending migration only

Fixes: https://tracker.ceph.com/issues/77183

Signed-off-by: Emmanuel Ameh <eameh@contractor.linuxfoundation.org>
2026-06-17 14:34:36 +01:00
eameh-LF
c0be03258d
Merge pull request #69355 from eameh-LF/wip-doc-77208
doc: Remove empty README.md and relocate nvmeof HA design doc
2026-06-17 14:25:50 +01:00
eameh-LF
0c899ca274
Merge pull request #69354 from eameh-LF/wip-doc-77206
doc/security: Add Security Lead and Working Group pages to toctree
2026-06-17 14:25:27 +01:00
Emmanuel Ameh
0531ac8950 doc/install: Update EOL version floor references in index.rst
The install overview cited Octopus (EOL 2022) as the cephadm minimum
and Nautilus (EOL 2021) as the Rook minimum. Update to Reef and Pacific
respectively, which are the currently supported release floor values.
Remove the repeated Nautilus/Octopus codename references in the
ceph-ansible paragraph, as these are historical release names not
needed to convey the message.

Fixes: https://tracker.ceph.com/issues/77192

Signed-off-by: Emmanuel Ameh <eameh@contractor.linuxfoundation.org>
2026-06-17 14:24:10 +01:00
Emmanuel Ameh
3966415dab doc/install: Update mirrors.rst to use https and current release
All mirror URLs used insecure http://. Update to https://. The example
repository paths used debian-hammer and rpm-hammer (Hammer was EOL in
2016); update to the current stable release (tentacle). Also update
the GitHub mirroring link from the stale master branch to main.

Fixes: https://tracker.ceph.com/issues/77197

Signed-off-by: Emmanuel Ameh <eameh@contractor.linuxfoundation.org>
2026-06-17 14:20:01 +01:00
Emmanuel Ameh
b9b725cab1 doc: Replace Python 2 package names with Python 3 equivalents
librados-intro.rst referenced ``python-rados`` for CentOS/RHEL.
rbd-openstack.rst referenced ``python-rbd`` for both apt and yum.
Python 2 reached end-of-life in January 2020; these package names
install the Python 2 bindings (or fail entirely) on current distros.
Replace with the correct Python 3 package names: python3-rados and
python3-rbd.

Fixes: https://tracker.ceph.com/issues/77200

Signed-off-by: Emmanuel Ameh <eameh@contractor.linuxfoundation.org>
2026-06-17 14:16:31 +01:00
Samarah Uriarte
e7d1a1b843
Merge pull request #65286 from pritha-srivastava/wip-rgw-d4n-mem-leak
rgw/d4n: deleting LFUDAEntry and LFUDAObjEntry instances

Reviewed-by: Mark Kogan <mkogan@redhat.com>
Reviewed-by: Gal Salomon <gsalomon@redhat.com>
2026-06-17 08:05:20 -05:00
Aishwarya Mathuria
34cf71d143 mon/OSDMonitor: only log PG merge backoff for crimson pools
This warning was added as part of Crimson pg merging and is supposed to be
meant just for Crimson pools. Correcting the if condition to reflect the same.

Fixes: https://tracker.ceph.com/issues/77459
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
2026-06-17 16:55:56 +05:30
Kefu Chai
ded0d4c146 crimson/osd: coroutinize OSD methods
this change migrate most of the OSD implementation to C++20 coroutine
when appropriate to improve the readability and maintainability.

there are two patterns in this migration:

- mechanic replacing future-promise then() chain with co_await/co_return
- replace `return <ready future>` with a `co_return` statement
- replace `seastar::do_for_each()` with a regular for loop, and
  optionally restructure the code so it's more compacted.
- replace handle_exception() with regular try-catch block

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-17 18:54:08 +08:00
Kefu Chai
17c6e03f17 crimson/osd: use std::unique_lock when appropriate
instead of lock and unlock a mutex manually, let's use the RAII
helper. simpler this way.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-17 18:54:08 +08:00
Kefu Chai
15be0e2d3b crimson/osd: avoid unnecessary co_await make_ready_future call
no need to create a future then co_await it. let's just co_return
it.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-17 18:54:08 +08:00
Kefu Chai
8e7fc84fef crimson/osd: remove extraneous co_return
no need to use co_return at the end of a coroutine which returns
future<>. despite that this does not incur performance penalty,
because:

co_await func() // func returns future<>

returns void

co_return is called with void operand, this statement is lowered to

promise.return_void()

and a bare "co_await func()" also calls return_void(). so they are
equivalent at runtime. but the extraneous co_return can be dispensed.

also take this opportunity to remove the bare co_return at the end
of a coroutine. it's unnecessary just like a `return` at the end of
a function returning `void`.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-17 18:54:08 +08:00
Miki Patel
a10f58df8c rbd-mirror: prune obsolete primary mirror snapshots after relocation
Previously, obsolete primary and demoted primary snapshots on the
secondary cluster were not cleaned up immediately after relocation.
Instead, old primary snapshots remained until a subsequent promote
operation triggered their cleanup, while old demoted primary snapshots
persisted until a later demote operation removed them.

Adding changes for proactive cleanup of obsolete primary and demoted
primary snapshots that are no longer required after relocation.

Also adding test coverage to validate the cleanup behavior.

Fixes: https://tracker.ceph.com/issues/76154

Signed-off-by: Miki Patel <miki.patel132@gmail.com>
2026-06-17 16:17:27 +05:30
Nitzan Mordechai
90b925d824 osd-recovery-prio: race condition fix in TEST_recovery_pool_priority
Use norecover to pause recovery ops while inspecting reservation
state. Without it, recovery can complete before the admin-socket
dump is captured, causing intermittent test failures

Fixes: https://tracker.ceph.com/issues/77113
Signed-off-by: Nitzan Mordechai <nmordech@ibm.com>
2026-06-17 10:34:58 +00:00
Shweta Bhosale
488c7fdf09 mgr/nfs: read full conf object when removing export URL
remove_obj() called ioctx.read() without a length, so only the first
8192 bytes of conf-nfs.* were read before write_full(). Removing exports
from a large cluster truncated and corrupted the shared config object,
leaving malformed entries such as '%url "rados:' while export objects
were deleted.

Fixes: https://tracker.ceph.com/issues/77463
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-17 15:49:08 +05:30
Nizamudeen A
4fdb541ea6
Merge pull request #69378 from rhcs-dashboard/custom-filter
mgr/dashboard: add custom filtering rules to the table

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Naman Munet <nmunet@redhat.com>
2026-06-17 14:30:15 +05:30
Kefu Chai
1d18b19997 cmake: enable compile-time fmt format string checking
seastar checks log format strings at compile time when
Seastar_LOGGER_COMPILE_TIME_FMT is on; the option is gated on
fmt_VERSION >= 9.0.0. ceph builds fmt via add_subdirectory(), which
leaves the version in fmt's own FMT_VERSION and never sets the lowercase
fmt_VERSION the option reads, so the check was silently off for crimson.
read the version off the fmt target and expose it so the option turns
on.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-17 16:54:51 +08:00
Kefu Chai
863f65fdee crimson/osd, seastore: fix log argument mismatches in error paths
two log calls in errorator continuations were missing an argument. the
compile-time fmt check surfaced these as a confusing "call to immediate
function ... is not a constant expression" deep in the errorator/composer
machinery rather than a plain "too few arguments", because the consteval
format failure escalates through the continuation templates:

- shard_services.cc handle_pg_create_info(): "ignore pgid {}" never
  passed pgid.
- node.cc: the read_extent error handler logged "{} -- addr={},
  is_level_tail={}" without the leading error; pass the std::error_code
  and include <fmt/std.h> for its formatter.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-17 16:54:51 +08:00
Kefu Chai
846f9294c6 crimson/net: log io_state_t out of line in shard_states_t
try_enter_out_dispatching()'s impossible default case logs io_state_t,
which would instantiate fmt::formatter<io_state_t> inside IOHandler,
before the formatter (declarable only after the class, since io_state_t
is nested) is visible. move just that abort path to io_handler.cc, where
the formatter is in scope, and keep the hot switch inline.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-17 16:54:51 +08:00
Kefu Chai
d5d2ff2e49 crimson, osd: declare fmt formatters before they are used
the seastar logger checks log format strings with a consteval call at
the use site, which instantiates fmt::formatter<T> there. a
specialization declared at the bottom of a header, after inline methods
that already log T, is then "explicit specialization after
instantiation".

move the offending specializations ahead of the type: PG, LBALeafNode,
DummyNodeExtent and WatchTimeoutRequest get a forward declaration plus
the formatter before the class; overwrite_range_t, data_t and edge_t in
object_data_handler.cc move ahead of the functions that log them. while
here, trim_data_reservation() formatted a laddr_t base with 0x{:x}, but
laddr_t's formatter takes no spec, so use {}.

osd_types_fmt.h includes msg/msg_fmt.h: formatter<osd_reqid_t> formats
req_id.name (an entity_name_t), so it depends on that type's formatter
and must pull it in. otherwise logging a reqid instantiates
formatter<entity_name_t> before msg_fmt.h is seen.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-17 16:54:51 +08:00
Kefu Chai
cb5ea45c28 crimson/osd: fix malformed debug log format strings
a few debug() calls had format strings that did not match their
arguments. with the seastar logger formatting at runtime these only
threw an fmt error at the matching log level, or went unnoticed:

- pg_backend.cc: "{}: object does not exist: {}" had two placeholders
  but one argument; drop the stray leading "{}: ".
- pg.cc do_recover_missing(): the "need to wait for recovery ... version
  {}" message never passed the eversion_t; add it.
- client_request.cc: a DEBUGDPP() carried an extra leading "{}: " on top
  of the prefix the macro already injects (also fix the "rwoedered"
  typo).

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-17 16:54:51 +08:00
Kefu Chai
19d5e52d8b
Merge pull request #69476 from ronen-fr/wip-rf-fmtxx-crimson
crimson/os/seastore: fix laddr_t formatter and its use

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-06-17 16:52:07 +08:00
Xuehan Xu
af137a0d86 crimson/os/seastore/lba: avoid paddr from crossing coroutines
Previously, LBAManager::remap_mappings() works as follows:

1. get the mapping's val;
2. remove the mapping;
3. insert the remapped mappings.

With pessimistic cc in place, during the above step 2 and 3, a rewrite
transaction that modifies the same mapping might be committed and miss
the pending lba leaf nodes because it doesn't contain the mapping at the
time.

This commit change the above workflow as follows:

1. replace the mapping with the first remapped mapping;
2. insert the remaining remapped mappings.

Note that all the remaining mapped mappings' paddrs are calculated based
on the mapping before it in the same coroutine as the insertion, which
means it'll always see the modification of a background rewrite
transaction.

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-06-17 15:45:49 +08:00
Xuehan Xu
0419e34f8d crimson/os/seastore/transaction_manager: prevent paddr from
being passed across coroutines.

This is because rewrite transactions no longer invalidates other
transactions, so if paddr get passed across coroutines, it may
become outdated.

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-06-17 15:45:49 +08:00
Vallari Agrawal
3c5295cccb
Merge pull request #69297 from VallariAg/wip-prometheus-rados-ns-fix
monitoring: fix NVMeoFMultipleNamespacesOfRBDImage for different rados_namespace_name
2026-06-17 12:43:44 +05:30
Nizamudeen A
c7c62383d8 mgr/dashboard: add custom filtering rules to the table
```
<cd-table #table
                id="pool-list"
                [data]="pools"
                [columns]="columns"
                selectionType="single"
                [hasDetails]="true"
                [status]="tableStatus"
                [autoReload]="-1"
                (fetchData)="taskListService.fetch()"
                (setExpandedRow)="setExpandedRow($event)"
                (updateSelection)="updateSelection($event)"
                [customFilter]="true" # set this to true
                (customFilterChange)="onCustomFilterChange($event)" #
get the new rules from here>
```

Fixes: https://tracker.ceph.com/issues/77290
Signed-off-by: Nizamudeen A <nia@redhat.com>
2026-06-17 11:05:01 +05:30
Kefu Chai
4cd080d6f3
Merge pull request #69515 from tchaikov/wip-rocksdb-fix-ftbfs-gcc-16
rocksdb: update submodule to fix FTBFS due to missing <cstdint>

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2026-06-17 11:17:35 +08:00
Kefu Chai
1999afb8e5
Merge pull request #69509 from sunyuechi/wip-fix-unused-warnings
crimson,mgr,test: fix unused variable/function warnings

Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-06-17 10:07:27 +08:00
John Mulligan
1f70d69341
Merge pull request #69419 from Sodani/shsodani_mac_support
mgr/smb: Added a Mac client support for samba cluster

Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Sachin Prabhu <sp@spui.uk>
2026-06-16 18:50:31 -04:00
Hezko
38ef9bc6b9
Merge pull request #69339 from Hezko/clis-alignment
mgr/dashboard: align nvmeof cli with missing parameters and commands from the old nvmeof cli
2026-06-17 00:11:23 +03:00
Oguzhan Ozmen
fe6af7fc85 doc: add PendingReleaseNotes entry for rgw multisite DNS endpoint resolution
Documents the new rgw_rest_conn_connect_to_resolved_ips feature that
enables RGW to resolve HTTP endpoints for RGW services such as multisite,
into all IP addresses and distribute requests across them using
round-robin with per-IP health tracking, supporting DNS service
discovery deployments without external load balancers.

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-16 19:59:06 +00:00
Oguzhan Ozmen
29e0d6104f
Merge pull request #69132 from BBoozmen/wip-sync-err-log-76950
rgw/multisite: do not log transient per-object EBUSY/EAGAIN errors in sync error log
2026-06-16 15:11:24 -04:00
Patrick Donnelly
bfff14c222
Merge PR #69498 into main
* refs/pull/69498/head:
	doc/releases/tentacle: scope wording to package installs
	doc/releases/tentacle: remove bogus notable changes
	doc/releases/tentacle: add more v20.2.2 blocker fixes
	doc: tentacle 20.2.2 release notes

Reviewed-by: Adam King <adking@redhat.com>
2026-06-16 14:51:39 -04:00
Patrick Donnelly
1578756c74
doc/releases/tentacle: scope wording to package installs
Resolves: https://tracker.ceph.com/issues/77357
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-06-16 14:16:32 -04:00
Patrick Donnelly
f78b44aba3
doc/releases/tentacle: remove bogus notable changes
Resolves: https://tracker.ceph.com/issues/77357
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-06-16 14:16:32 -04:00
Kotresh HR
3fc6457a8f qa: Add tests for cephfs_mirror_directory perf counters
Verify counter dump registration, current-sync gauges while syncing
(full/delta), last-sync and summary counters after idle sync, and
extend mirror stats and remote-snap failure tests for per-directory
snaps_* and dir_state.

Fixes: https://tracker.ceph.com/issues/73457
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-16 23:11:16 +05:30
Oguzhan Ozmen
63b7f7430b
Merge pull request #68881 from jacquesh/fix-rgw-log-merging
rgw: Fix ops logs sometimes having several entries per line.
2026-06-16 13:37:14 -04:00
Shweta Sodani
23c910f52a doc/mgr/smb: document client compatibility mode
Add documentation for --client-compat parameter in 'cluster create'
command and new 'cluster update client-compat' command. This feature
enables macOS-specific SMB optimizations (fruit VFS, streams_xattr)
and can be set during cluster creation or updated for existing clusters.

Signed-off-by: Shweta Sodani <shsodani@redhat.com>
2026-06-16 22:20:37 +05:30
Kotresh HR
c5be81c79d doc/cephfs: Document cephfs_mirror_directory counter dump metrics
Describe per-directory labeled perf counters, labels, update behavior,
mapping to peer status, and counter reference tables. Document the
per-peer tick thread and cephfs_mirror_tick_interval, which refreshes
current-sync gauges on each tick. Add a PendingReleaseNotes entry for
the new cephfs_mirror_directory perf counter group.

Fixes: https://tracker.ceph.com/issues/73457
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-16 22:17:23 +05:30
Kotresh HR
13b0a36eec cephfs_mirror: update per-directory last sync and summary perf counters
Wire the remaining cephfs_mirror_directory labeled counters that were
registered in the prior commit but not yet refreshed. Live
current_syncing_snap gauges continue to be updated from the per-peer tick
thread; this commit updates last_synced_snap and per-directory snap
summary counters at the points where SnapSyncStat is actually modified.

Depends-on the cephfs_mirror_directory PerfCounters schema (dir_state,
current_*, last_*, snaps_*) added when each mirrored directory is
registered.

Prometheus / counter dump
-------------------------

These counters appear under "cephfs_mirror_directory" in "counter dump" with
the same labels as current-sync metrics (source_fscid, source_filesystem,
peer_uuid, peer_cluster_name, peer_cluster_filesystem, directory). ceph-exporter
exposes them as e.g. ceph_cephfs_mirror_directory_last_sync_bytes and
ceph_cephfs_mirror_directory_snaps_synced.

Unlike cephfs_mirror_peers, values are per (peer_uuid, directory) rather than
aggregated across all directories on the peer. Peer-level counters are
unchanged.

Fixes: https://tracker.ceph.com/issues/73457
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-16 22:17:23 +05:30
Kotresh HR
1600974187 tools/cephfs_mirror: Expose per-directory snap metrics via perf counters
Introduce a new labeled perf counter group, cephfs_mirror_directory, so
per-directory snapshot mirror progress can be scraped via "counter dump" and
exported to Prometheus by ceph-exporter (e.g.
ceph_cephfs_mirror_directory_current_sync_bytes).

Design
------

* One PerfCounters instance per mirrored directory on a peer, keyed in
  m_directory_perf_counters and registered on the daemon-wide
  PerfCountersCollection.
* Labels on each instance (flat counter dump array entries):
    - source_fscid, source_filesystem
    - peer_uuid, peer_cluster_name, peer_cluster_filesystem
    - directory (dir_root, e.g. "/parent/d1")
  The peer_uuid label disambiguates the same directory path mirrored to
  different peers.
* Counters are created in init() and add_directory(), removed in
  remove_directory() and the PeerReplayer destructor.
* Priority follows cephfs_mirror_perf_stats_prio (same as
  cephfs_mirror_peers).

Update path
-----------

Live / current_syncing_snap gauges are refreshed from
update_directory_current_sync_perf_counters(), called by
refresh_directory_current_sync_perf_counters() from the per-peer tick
thread (run_tick()). Each cephfs_mirror_tick_interval seconds (default 5)
the tick thread updates counters for each registered (actively syncing)
directory.

Counters registered (schema)
----------------------------

All of the following are added to the builder in this commit. Only the
"current sync" and dir_state fields listed under "Updated in this commit"
are written here; last_synced_snap and per-directory snap summary counters
are registered for a follow-up commit that updates them when stats change.

Directory state
  dir_state (gauge u64)
    0 = idle, 1 = syncing, 2 = failed
    Maps peer_status top-level "state" (numeric; no string values).

Current syncing snapshot (peer_status "current_syncing_snap")
  [Updated in this commit]
  current_snap_id          - snapshot id being synchronized
  current_sync_mode        - 0 = full, 1 = delta (snapdiff)
  current_read_bps         - bytes/sec read (raw, not formatted)
  current_write_bps        - bytes/sec written
  crawl_state              - 0 = N/A, 1 = in-progress, 2 = completed
  crawl_duration_seconds   - crawl duration; in-progress uses now - start
  datasync_wait_state      - 0 = none, 1 = waiting, 2 = complete
  datasync_wait_duration_seconds
  current_sync_bytes       - bytes synced so far for this snap
  current_total_bytes      - total bytes for this snap
  current_sync_bytes_percent - basis points (1745 = 17.45%)
  current_sync_files
  current_total_files
  current_sync_files_percent - basis points
  current_eta_valid        - 0 = calculating, 1 = ETA available
  current_eta_seconds      - ETA in seconds when valid

Per-directory snapshot summary (peer_status snaps_*)
  [Registered only; not updated in this commit]
  snaps_synced, snaps_deleted, snaps_renamed

Last synced snapshot (peer_status "last_synced_snap")
  [Registered only; not updated in this commit]
  last_snap_id
  last_crawl_duration_seconds
  last_datasync_wait_duration_seconds
  last_sync_duration_seconds
  last_sync_timestamp      - utime_t / seconds since epoch
  last_sync_bytes
  last_sync_files

When idle or failed, current_* counters are zeroed and dir_state reflects
0 or 2 respectively.

Fixes: https://tracker.ceph.com/issues/73457
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-16 22:17:23 +05:30
Kotresh HR
b8310f33c3 tools/cephfs_mirror: Add per-peer tick thread with configurable interval
Introduce a per-peer tick thread controlled by cephfs_mirror_tick_interval
(default 5 seconds). The interval is re-read each iteration so configuration
changes take effect without restarting the daemon. The thread provides a
generic hook for future periodic mirroring work.

Fixes: https://tracker.ceph.com/issues/76686
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-16 22:17:23 +05:30
Sridhar Seshasayee
2ccd861d04
Merge pull request #69366 from sseshasa/wip-fix-ok-to-upgrade-osd-sort-order
mgr/DaemonServer: Aggregate and globally sort OSDs for ok-to-upgrade

Reviewed-by: Radoslaw Zarzynski <rzarzynski@redhat.com>
Reviewed-by: Nitzan Mordechai <nmordech@ibm.com>
2026-06-16 21:46:13 +05:30
Radoslaw Zarzynski
c183e27dea tests/neorados: fix ceph_test_neorados_completions being not installed
Fixes: https://tracker.ceph.com/issues/77417
Signed-off-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
2026-06-16 16:02:25 +00:00
Ronen Friedman
620d9362f2 crimson/os/seastore: fix laddr_t formatter and its use
'laddr_t' existing formatter did not support a ':x' format specifier
(actually - the output was always hexadecomal).
Here we remove the ':x', but also refactor the custom formatter to
avoid using the streambuf mechanism.
Note - SEASTORE_LADDR_USE_BOOST_U128 is no longer supported by the formatter.

Fixes: https://tracker.ceph.com/issues/77399
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2026-06-16 15:13:34 +00:00
Sun Yuechi
d4fc8b65b7 pybind/cephfs: drop redundant int32_t typedef
int32_t is already provided by "from libc.stdint cimport *", so the extra
ctypedef in the platform_errno.h extern block redeclares it:

  warning: c_cephfs.pxd:29:4: 'int32_t' redeclared

Fixes: https://tracker.ceph.com/issues/77440

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-16 21:48:41 +08:00
Adam Kupczyk
daf1e89117
Merge pull request #69395 from aclamk/aclamk-doc-bs-fast-recovery
doc, bluestore: Documentation for Fast Onode Recovery feature
2026-06-16 15:03:31 +02:00
Sun Yuechi
82ab9cc2e3 crimson/cmake: restrict -Wno-non-virtual-dtor to C++ sources
The seastar target exports -Wno-non-virtual-dtor as a PUBLIC compile
option, so it leaks to every target that links seastar, including C
sources such as crimson-common's reverse.c, producing:

  cc1: warning: command-line option '-Wno-non-virtual-dtor' is valid for C++/ObjC++ but not for C

Guard the option with $<COMPILE_LANGUAGE:CXX> so that it only applies to
C++ compilations.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-16 19:25:22 +08:00
Sun Yuechi
2bb17b80aa run-make-check.sh: build ctest cpu resource pool from sched_getaffinity
gen_ctest_resource_file assumed online CPUs are a contiguous 0..nproc-1 range.
When a middle core is hot-offlined (e.g. for a hardware fault), that both hands
out the offline cpu id and drops an online one; crimson seastar unittests feed
the ctest resource id straight to seastar --cpuset, so hitting the offline id
aborts with "Bad value for --cpuset: N not allowed".

Use the CPUs actually schedulable by the process via sched_getaffinity().

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-16 18:35:04 +08:00
Shweta Sodani
6f59987690 mgr/smb: add tests for client compatibility mode
Add comprehensive test coverage for the new client compatibility
feature that enables macOS-specific SMB optimizations:

- test_enums.py: Add tests for ClientSupportMode enum values
  (DEFAULT and MACOS) and string representation
- test_resources.py: Add tests for cluster client_compat field,
  effective_client_compat property, and is_macos_compatibility_enabled
  property with different mode configurations
- test_smb.py: Add integration tests for cluster_update_client_compat
  CLI command including successful updates and error handling for
  non-existent clusters

These tests ensure the client compatibility mode can be properly
set, retrieved, and updated at the cluster level.

Signed-off-by: Shweta Sodani <shsodani@redhat.com>
2026-06-16 15:58:05 +05:30
Shweta Sodani
45f943ddb8 mgr/smb: Add client support mode for macOS-specific SMB features
This commit introduces a new cluster-level configuration option to enable
client-specific SMB optimizations, starting with macOS support.

Usage:
  ceph smb cluster create <cluster-id> --client-compat macos
  ceph smb cluster update client-compat macos <cluster-id>

Signed-off-by: Shweta Sodani <ssodani@redhat.com>
2026-06-16 15:58:05 +05:30
Kefu Chai
cdb37469b0 test/objectstore: hold split_blob cache shards in unique_ptr
ExtentMap.split_blob, added in be93e121a9, creates the onode and buffer cache
shards as raw pointers and never frees them. once we build with ASan,
LeakSanitizer reports them (and the structures they own):

  Direct leak of 9928 byte(s) ... BlueStore::OnodeCacheShard::create
  Direct leak of 224 byte(s) ... BlueStore::BufferCacheShard::create
  SUMMARY: AddressSanitizer: 10288 byte(s) leaked in 8 allocation(s).

every other test in this file already wraps these shards in a unique_ptr,
declared before the collection that borrows them so they outlive it. do
the same here.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-16 18:10:21 +08:00
Xuehan Xu
a8af36d04e crimson/os/seastore/transaction: clear copied_lba_keys on reset
Before this commit, Transaction::copied_lba_keys was never cleared
on reset, so when a transaction is reset, and another rewrite
transaction is committing, the rewrite transaction would try to
update the copied lba mapping which may not have been copied by
the reset transaction, which would result in unexpected errors.

This commit generally follows the regulations about resetting
transactions, which is clear everything on reset

Fixes: https://tracker.ceph.com/issues/76945
Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-06-16 17:57:19 +08:00
Kefu Chai
59afb3d661 rocksdb: update submodule to fix FTBFS due to missing <cstdint>
43dd4cbd37 bumped the rocksdb submodule to v7.10.2 for CVE-2022-23476,
dropping the <cstdint> includes the v7.9.2 pin carried.
db/blob/blob_file_meta.h uses uint64_t but no longer includes <cstdint>,
so it compiles only where another header pulls <cstdint> in transitively.
GCC with libstdc++ 16.1.0 no longer does, so the build fails:

    db/blob/blob_file_meta.h: error: 'uint64_t' has not been declared

our targeted distros still pull it in, so the failure went unnoticed
there: ubuntu jammy (GCC 11.2.0) and noble (GCC 13.2).

bump the submodule to a cherry-pick of upstream rocksdb 72c3887167,
which fixes the same FTBFS.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-16 17:07:55 +08:00
Samarah Uriarte
5910070aab rgw/d4n: Update policy unit test
Signed-off-by: Samarah Uriarte <samarah.uriarte@ibm.com>
2026-06-16 13:32:11 +05:30
Pritha Srivastava
c1dbe78a14 rgw/d4n: adding a thread to asynchronously update
localweight to the cache backend. Removing the code
to update the localweight from GET and PUT requests.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
2026-06-16 13:31:44 +05:30
Tomer Haskalovitch
e3d9ebb50e mgr/dashboard: align nvmeof cli with missing parameters and functions from the old nvmeof cli
fixes: https://tracker.ceph.com/issues/77108
Signed-off-by: Tomer Haskalovitch <tomer.haska@ibm.com>
2026-06-16 10:41:53 +03:00
Sun Yuechi
d3c8d3fc65 crimson,mgr: mark assert-only variables [[maybe_unused]]
These variables are only read inside assert(), which is compiled out
under NDEBUG. Mark them [[maybe_unused]] to silence the warnings while
keeping the debug-only assert() style used by the surrounding code:

  src/crimson/os/seastore/lba/btree_lba_manager.cc:1078: unused variable 'orig_len' [-Wunused-variable]
  src/crimson/os/seastore/omap_manager/log/log_manager.cc:73: variable 'ret' set but not used [-Wunused-but-set-variable]
  src/crimson/os/seastore/transaction_manager.cc:382: variable 'intermediate_key' set but not used [-Wunused-but-set-variable]
  src/mgr/PyModule.cc:166,186: unused variable 'r' [-Wunused-variable]

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-16 15:24:26 +08:00
Sun Yuechi
1265c6b5cd test/crimson/seastore: use gtest assertion macros instead of assert()
Plain assert() is compiled out under NDEBUG, leaving the checked
variables unused. Use the always-evaluated gtest macros instead.

  src/test/crimson/seastore/test_cbjournal.cc:586: variable 'old_written_to' set but not used [-Wunused-but-set-variable]
  src/test/crimson/seastore/test_btree_lba_manager.cc:345: unused structured binding declaration [-Wunused-variable]

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-16 15:24:26 +08:00
Pritha Srivastava
b943cb6d68 rgw/d4n: deleting LFUDAEntry and LFUDAObjEntry instances
in LFUDAPolicy destructor.

Signed-off-by: Pritha Srivastava <prsrivas@redhat.com>
2026-06-16 12:14:07 +05:30
Sun Yuechi
0b8c738927 crimson,test: remove unused functions and dead variable
Fixing these warnings:

  src/crimson/os/seastore/seastore.cc:83: 'omaptree_initialize' defined but not used [-Wunused-function]
  src/crimson/osd/replicated_recovery_backend.cc:733: 'nullopt_if_empty' defined but not used [-Wunused-function]
  src/test/rgw/test_rgw_kms_cache.cc:63: 'rethrow' defined but not used [-Wunused-function]
  src/test/librados/test_cxx.cc:215: variable 'cmd' set but not used [-Wunused-but-set-variable]

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-16 12:36:11 +08:00
SrinivasaBharathKanta
2286265c0a
Merge pull request #69106 from bigjust/upgrade-rocksdb-7.10.2-cve-2022-23476
rocksdb: upgrade submodule to v7.10.2 to address CVE-2022-23476
2026-06-16 09:54:11 +05:30
SrinivasaBharathKanta
958cd0bbb1
Merge pull request #68662 from kamoltat/wip-ksirivad-stretch-crush-experiment
src/script: init test_stretch_crush_collisions.sh
2026-06-16 09:53:01 +05:30
SrinivasaBharathKanta
7d8bada343
Merge pull request #68425 from dheart-joe/pretty-kvtool-output
tool/ceph-kvstore-tool: add --pretty-binary-key option
2026-06-16 09:52:07 +05:30
SrinivasaBharathKanta
e2c6862ce9
Merge pull request #67747 from indirasawant/wip-isawant-mgr-standby-details
mon/mgr: include standby manager details in ceph mgr stat
2026-06-16 09:51:26 +05:30
Patrick Donnelly
b52f7f53ae
doc/releases/tentacle: add more v20.2.2 blocker fixes
Resolves: https://tracker.ceph.com/issues/77357
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-06-15 22:52:47 -04:00
Yuri Weinstein
35447386ba
doc: tentacle 20.2.2 release notes
Resolves: https://tracker.ceph.com/issues/77357
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
2026-06-15 22:52:47 -04:00
Kotresh HR
fcf6891875
Merge pull request #69467 from kotreshhr/mirror-handle-dup-add-directory-notification
tools/cephfs_mirror: Ignore duplicate directory acquire notifications

Reviewed-by: Venky Shankar <vshankar@redhat.com>
2026-06-16 08:18:44 +05:30
Patrick Donnelly
c580421359
Merge PR #69494 into main
* refs/pull/69494/head:
	doc: Document restarting failed release builds

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
2026-06-15 21:25:50 -04:00
Patrick Donnelly
854f5340ad
Merge PR #69492 into main
* refs/pull/69492/head:
	doc/dev/release-process: update according to supported releases

Reviewed-by: Anthony D Atri <anthony.datri@gmail.com>
2026-06-15 21:22:38 -04:00
Patrick Donnelly
8a2cbbd4f8
doc/dev/release-process: update according to supported releases
From: https://docs.ceph.com/en/latest/start/os-recommendations/

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-06-15 18:41:25 -04:00
Kamoltat (Junior) Sirivadhna
c72f9a86b8 mon: fix ConnectionTracker::notify_rank_removed
Problem:
When multiple monitors are removed from the monmap in a single update,
the iterative rank adjustment in notify_rank_removed() fails to maintain
the invariant that our local rank matches the monmap's new_rank until
all iterations complete.

Problem:
The old code decremented rank for each removed rank:
```
  for (auto i = monmap->removed_ranks.rbegin(); ...) {
    int remove_rank = *i;
    int new_rank = monmap->get_rank(messenger->get_myaddrs());

    // Old logic (in ConnectionTracker):
    if (remove_rank < rank) {
      --rank;
    }

    // This assert would fail on first iteration when removed_ranks.size() > 1
    ceph_assert(rank == new_rank);
  }
```

When removed_ranks = {0, 1} and we are mon.d (originally rank 3):
- new_rank from monmap = 1 (final correct value after removing 0 and 1)

Iteration 1 (remove rank 1):
  Before: rank = 3
  Execute: rank_removed=1 < 3, so --rank -> rank = 2
  Check: rank == new_rank? -> 2 == 1? FAIL

Iteration 2 (remove rank 0):
  Before: rank = 2
  Execute: rank_removed=0 < 2, so --rank -> rank = 1
  Check: rank == new_rank? → 1 == 1? PASS

The invariant is violated during intermediate iterations because each
--rank adjustment only accounts for ONE removed rank, but new_rank
from monmap already accounts for ALL removed ranks.

Solution:
Trust the caller's new_rank from monmap instead of trying to compute it
iteratively. The monmap has already recalculated ranks correctly by
removing all monitors atomically, so we should just use that value.

Changed:
```
  if (rank_removed < rank) {
    --rank;
  }
```
To:
  // Trust the caller's new_rank from the monmap rather than trying to compute it.
  // This handles cases where multiple ranks are removed simultaneously.
  rank = new_rank;

Now the invariant holds on every iteration regardless of how many ranks
are removed.

Fixes: https://tracker.ceph.com/issues/77423

Signed-off-by: Kamoltat (Junior) Sirivadhna <ksirivad@redhat.com>
2026-06-15 22:14:56 +00:00
David Galloway
937ffe0d03 doc: Document restarting failed release builds
Signed-off-by: David Galloway <david.galloway@ibm.com>
2026-06-15 17:37:51 -04:00
Vallari Agrawal
1414770fbb
mgr/dashboard: bump nvmeof submodule to 1.8.2
Update proto files and gateway submodule

Fixes: https://tracker.ceph.com/issues/77422

Signed-off-by: Vallari Agrawal <vallari.agrawal@ibm.com>
2026-06-16 03:07:06 +05:30
stzuraski898
078b140ad1 mgr/ActivePyModules: refactor perf schema section management to use RAII
Refactor get_perf_schema_python() and get_unlabeled_perf_schema_python() to use Formatter::ObjectSection and Formatter::ArraySection RAII helpers instead of manual close_section() calls. Manual closes have have led to crashes when close_section() calls were left unguarded, for example when an OSD with no perf counters had it's perf counters queried.

Fixes: https://tracker.ceph.com/issues/74693
Signed-off-by: stzuraski898 <steven.zuraski@ibm.com>
2026-06-15 21:19:09 +00:00
Patrick Donnelly
648794cce3
script/ptl-tool: do not update PRs for private QA
Assisted-by: Gemini
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-06-15 16:28:16 -04:00
Patrick Donnelly
88b9725be0
script/ptl-tool: source githubmap only when producing credits
Assisted-by: Gemini
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-06-15 16:27:22 -04:00
Patrick Donnelly
13b334507f
script/ptl-tool: address datetime warning
/home/runner/work/ceph/ceph/src/script/ptl-tool.py:1743: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC).

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-06-15 16:25:23 -04:00
Patrick Donnelly
cf5b149eec
Merge PR #69446 into main
* refs/pull/69446/head:
	python-common/cryptotools: stop using the removed X509Req API

Reviewed-by: Patrick Donnelly <pdonnell@ibm.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
2026-06-15 16:14:23 -04:00
John Mulligan
d14e816a13
Merge pull request #69289 from anoopcs9/fix-run-samba-perms
cephadm/smb: Bind mount /run with 0755

Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Xavi Hernandez <xhernandez@gmail.com>
2026-06-15 15:42:48 -04:00
Ericmzhang
69c30a7d54
Merge pull request #68532 from Ericmzhang/wip-overlapped-roots-autoscale
Mgr: Allow autoscaling for overlapped roots
2026-06-15 10:28:56 -07:00
Kefu Chai
3d0188f3c6
Merge pull request #68856 from tchaikov/wip-ceph-dencoder-dlclose
ceph-dencoder: skip dlclose under ASan so leaks symbolise

Reviewed-by: Nitzan Mordechai <nmordec@ibm.com>
2026-06-15 20:14:23 +08:00
Ronen Friedman
b77ff53605
Merge pull request #69415 from ronen-fr/wip-rf-clsrefcount
crimson/osd: fix PGBackend::remove() to return ENOENT on no-op deletes

Reviewed-by: Matan Breizman <mbreizma@redhat.com>
2026-06-15 14:31:57 +03:00
Sun Yuechi
056b706131 rgw: declare rgw_a's dependency on rgw_schedulers and kmip
rgw_a uses rgw::dmclock::* (rgw_schedulers) and kmip_* (kmip) but never
declared either, so each consumer relinked them by hand. ld.bfd hid this
via lazy archive extraction; mold pulls the members and fails with
undefined symbols. Declare it on rgw_a (PRIVATE) and drop the now
redundant explicit links from radosgw and the rgw shared library.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-15 19:08:55 +08:00
Kotresh HR
e74f11546a tools/cephfs_mirror: Ignore duplicate directory acquire notifications
Make PeerReplayer::add_directory() idempotent when the mgr re-sends
acquire for a directory already in the replayer list.

Fixes: https://tracker.ceph.com/issues/77398
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-15 16:15:37 +05:30
Kotresh HR
28a2a2c882 qa/cephfs: Add test for duplicate directory acquire notify
Verify that reloading the mirroring module and removing a directory
does not leave a ghost replayer entry that keeps syncing snapshots.

Fixes: https://tracker.ceph.com/issues/77398
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-15 16:15:33 +05:30
Matan Breizman
123fa4ca39
Merge pull request #69281 from Matan-B/wip-matanb-seastore-p95
seastore: add latency distribution (p95/p99) support

Reviewed-by: Xuehan Xu <xuxuehan@qianxin.com>
Reviewed-by: Josh Durgin <jdurgin@redhat.com>
Reviewed-by: Mohit Agrawal <moagrawa@redhat.com>
2026-06-15 12:11:08 +03:00
Shweta Bhosale
8da5a001d6
Merge pull request #66864 from ShwetaBhosale1/fix_issue_74045_ssh_hardning_feature
mgr/cephadm: Cephadm hardening (SSH channel)
2026-06-15 11:03:35 +05:30
Sun Yuechi
b573738722 build/script: print sccache stats alongside ccache
sccache is enabled via WITH_SCCACHE, but run-make.sh only printed
ccache statistics after the build. Print `sccache --show-stats` too
when sccache is available.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-14 22:33:55 +08:00
Nitzan Mordechai
a3dad559e8 mgr: handle SIGTERM/SIGINT in standby mgr to avoid CEPHADM_FAILED_DAEMON
MgrStandby only registered SIGHUP, leaving SIGTERM/SIGINT at OS defaults.
When a standby is stopped via `ceph orch daemon stop`, SIGTERM terminates
it with exit code 143, causing systemd to mark the unit as failed instead
of stopped, which triggers CEPHADM_FAILED_DAEMON.

Mirror the handler already used in Mgr::init() for the active mgr.

Fixes: https://tracker.ceph.com/issues/77116
Signed-off-by: Nitzan Mordechai <nmordech@ibm.com>
2026-06-14 08:07:10 +00:00
Shai Fultheim
8816532f8a test/crimson/seastore: tolerate enoent in concurrent remap/overwrite helpers
test_remap_pin_concurrent races 32 transactions to remap or overwrite the
same extent; exactly one is expected to commit while the rest detect the
conflict and back out. The harness helpers try_get_pin() and try_get_extent()
treated only ct_error::eagain as a tolerable racing outcome -- returning an
empty result so the caller's is_conflicted()/null check counts the
transaction as conflicted -- and routed every other error to
ct_error::assert_all(), aborting the test.

When a competing transaction retires or remaps a mapping mid-race, a
subsequent get_pin()/read_extent() of that laddr legitimately resolves to
ct_error::enoent (ENOENT from BtreeLBAManager::get_cursor) rather than
eagain. The helpers then aborted with "get_extent got invalid error" instead
of treating it as a lost-the-race back-out. The failure is intermittent and
pre-existing: a gap in the test harness, not a fault in the store.

Handle enoent exactly like eagain in try_get_pin() and both try_get_extent()
overloads: return an empty result so the existing conflict/back-out path runs.

Reproduction:

  ninja -C build unittest-transaction-manager
  ./bin/unittest-transaction-manager \
      --gtest_filter='*test_remap_pin_concurrent*' --smp 1 --gtest_repeat=30

Before this change the run aborts within a few iterations with
"get_extent got invalid error"; with it, all 30 iterations across the four
device variants pass.

Signed-off-by: Shai Fultheim <shai.fultheim@gmail.com>
2026-06-14 09:45:43 +03:00
Kefu Chai
d688bc8f34 mgr/dashboard: fix "welcome" page reference in add-host e2e feature
0fb99092ef renamed the cypress page-registry key "welcome" to
"onboarding" in urls.po.ts and updated 01-create-cluster-welcome.feature,
but missed the Background of 02-create-cluster-add-host.feature. so
urlsCollection.pages["welcome"] is undefined and the step fails with

  TypeError: Cannot read properties of undefined (reading 'url')
   at cypress/e2e/common/global.feature.po.ts:12

this fails the Background of every scenario in the spec, and cascades to
the later create-cluster specs that depend on the expanded cluster.

update the reference to "onboarding" to match the registry.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-14 09:27:52 +08:00
SrinivasaBharathKanta
ca0bc48a2a
Merge pull request #69005 from Jayaprakash-ibm/wip-jaya-mon-features-test-fix
qa: fix TEST_mon_features feature checks in mon/misc.sh
2026-06-14 04:24:52 +05:30
SrinivasaBharathKanta
7ab948826e
Merge pull request #68650 from leonidc/fix_force_exit_gw
nvmeofgw:fix forcing unavalable gw exit by sending
2026-06-14 04:24:33 +05:30
SrinivasaBharathKanta
ab5937a491
Merge pull request #68435 from stzuraski898/wip-sz-76048
mgr: ActivePyModules does not set Description in labeled get_perf_schema_python
2026-06-14 04:23:49 +05:30
Kotresh HR
5bfb9a8db3
Merge pull request #68018 from kotreshhr/mirror-asok-metrics
tools/cephfs_mirror: Add metrics
2026-06-13 23:59:00 +05:30
Sun Yuechi
ede8e34f61 cmake: find Protobuf via cmake config before module mode
Module-mode FindProtobuf defines only the classic targets, so
gRPCConfig.cmake's find_dependency(Protobuf) fails on the partial
export set. Prefer config mode; fall back to module mode.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-06-13 20:15:25 +08:00
Kefu Chai
1dda56b1a0 python-common/cryptotools: stop using the removed X509Req API
pyOpenSSL deprecated OpenSSL.crypto.X509Req in 24.2.0 (2024-07-20) and
removed it in 26.3.0 (2026-06-12). as we don't pin pyopenssl, CI picked
up the new release, and create_self_signed_cert() started failing with:

  AttributeError: module 'OpenSSL.crypto' has no attribute 'X509Req'

this took down run-tox-mgr, run-tox-mgr-dashboard-py3 and the mypy check.

we only used X509Req to build a subject name and then copied it into the
X509 cert. so drop it, and set the subject on the cert directly. the
resulting cert stays the same: subject from dname, issuer set to the same
subject, self-signed.

Fixes: https://tracker.ceph.com/issues/77391
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-13 10:05:28 +08:00
Kefu Chai
84ac1de643
Merge pull request #69421 from ShwetaBhosale1/fix_issue_77340_remove_-P_from_shebang_flags
ceph.spec.in: disable -P in python shebang for cephadm binary

Reviewed-by: Redouane Kachach <rkachach@redhat.com>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Reviewed-by: Kefu Chai <k.chai@proxmox.com>
2026-06-13 09:13:48 +08:00
Josh Durgin
d5da88d349 scripts/ceph-release-notes: use raw strings for escaped characters
And don't escape literals in strip().
This is required in newer python versions.

Signed-off-by: Josh Durgin <jdurgin@redhat.com>
2026-06-12 13:50:15 -07:00
Kotresh HR
1d676c2455 qa: Add mirror metrics testcases
Add testcases for newly introduced mirror
metrics and validate it via 'fs mirror peer status'
asok interface.

Fixes: https://tracker.ceph.com/issues/73453
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-13 00:45:31 +05:30
Kotresh HR
55eceaae3d doc: Update the mirroring doc with new metrics fields
Update the mirroring documentation and also the
release notes with new metrics introduced and it's
availability via 'fs mirror peer status' asok
interface.

Fixes: https://tracker.ceph.com/issues/73453
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-13 00:44:20 +05:30
Kotresh HR
75093b138e qa: Fix the mirroring tests with new nested peer_status output
Fixes: https://tracker.ceph.com/issues/73453
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-13 00:43:15 +05:30
Kotresh HR
2b32553717 tools/cephfs_mirror: Nest peer_status metrics by dir path and peer uuid
Restructure peer_status output so mirrored directory paths can be
shared by multiple peers without key collisions. Metrics are grouped
as metrics/<dir_path>/peer/<peer_uuid>/ instead of flat dir keys.

Sample output:
--------------
1. When two dirs are syncing.
{
    "metrics": {
        "/parent/d0": {
            "peer": {
                "8a85ab25-70f9-48e9-b82d-56324e75209b": {
                    "state": "syncing",
                    "current_syncing_snap": {
                        "id": 2,
                        "name": "d0_snap0",
                        "sync-mode": "full",
                        "avg_read_throughput_bytes": "9.01 MiB/s",
                        "avg_write_throughput_bytes": "26.74 MiB/s",
                        "crawl": {
                            "state": "completed",
                            "duration": "2s"
                        },
                        "datasync_queue_wait": {
                            "state": "completed",
                            "duration": "0s"
                        },
                        "bytes": {
                            "sync_bytes": "60.83 MiB",
                            "total_bytes": "149.94 MiB",
                            "sync_percent": "40.57%"
                        },
                        "files": {
                            "sync_files": 2028,
                            "total_files": 5000,
                            "sync_percent": "40.56%"
                        },
                        "eta": "10s"
                    },
                    "snaps_synced": 0,
                    "snaps_deleted": 0,
                    "snaps_renamed": 0
                }
            }
        },
        "/parent/d1": {
            "peer": {
                "8a85ab25-70f9-48e9-b82d-56324e75209b": {
                    "state": "syncing",
                    "current_syncing_snap": {
                        "id": 3,
                        "name": "d1_snap0",
                        "sync-mode": "full",
                        "avg_read_throughput_bytes": "6.80 MiB/s",
                        "avg_write_throughput_bytes": "20.04 MiB/s",
                        "crawl": {
                            "state": "in-progress",
                            "duration": "2s"
                        },
                        "datasync_queue_wait": {
                            "state": "completed",
                            "duration": "1s"
                        },
                        "bytes": {
                            "sync_bytes": "4.12 MiB",
                            "total_bytes": "124.98 MiB",
                            "sync_percent": "3.30%"
                        },
                        "files": {
                            "sync_files": 125,
                            "total_files": 4189,
                            "sync_percent": "2.98%"
                        },
                        "eta": "18s"
                    },
                    "snaps_synced": 0,
                    "snaps_deleted": 0,
                    "snaps_renamed": 0
                }
            }
        }
    }
}
---------
2. When two directories are synced

------------------------------------------
{
    "metrics": {
        "/parent/d0": {
            "peer": {
                "8a85ab25-70f9-48e9-b82d-56324e75209b": {
                    "state": "idle",
                    "last_synced_snap": {
                        "id": 2,
                        "name": "d0_snap0",
                        "crawl_duration": "2s",
                        "datasync_queue_wait_duration": "0s",
                        "sync_duration": "30s",
                        "sync_time_stamp": "422538.254127s",
                        "sync_bytes": "149.94 MiB",
                        "sync_files": 5000
                    },
                    "snaps_synced": 1,
                    "snaps_deleted": 0,
                    "snaps_renamed": 0
                }
            }
        },
        "/parent/d1": {
            "peer": {
                "8a85ab25-70f9-48e9-b82d-56324e75209b": {
                    "state": "idle",
                    "last_synced_snap": {
                        "id": 3,
                        "name": "d1_snap0",
                        "crawl_duration": "2s",
                        "datasync_queue_wait_duration": "1s",
                        "sync_duration": "33s",
                        "sync_time_stamp": "422546.205798s",
                        "sync_bytes": "149.94 MiB",
                        "sync_files": 5000
                    },
                    "snaps_synced": 1,
                    "snaps_deleted": 0,
                    "snaps_renamed": 0
                }
            }
        }
    }
}

Fixes: https://tracker.ceph.com/issues/73453
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-13 00:43:03 +05:30
Kotresh HR
66c05f080b tools/cephfs_mirror: Add datasync_queue_wait_duration metric
Add the metric which measures the time spent by the snapshot
in the data queue waiting for the datasync threads.

Sample output:
When still 'waiting' in queue
{
    "/d1": {
        "state": "syncing",
        "current_syncing_snap": {
            "id": 18,
            "name": "d1_snap5",
            "sync-mode": "delta",
            "avg_read_throughput_bytes": "0.00 B/s",
            "avg_write_throughput_bytes": "0.00 B/s",
            "crawl": {
                "state": "in-progress",
                "duration": "13s"
            },
            "datasync_queue_wait": {
                "state": "waiting",
                "duration": "12s"
            },
            "bytes": {
                "sync_bytes": "0.00 B",
                "total_bytes": "110.99 MiB",
                "sync_percent": "0.00%"
            },
            "files": {
                "sync_files": 0,
                "total_files": 3719,
                "sync_percent": "0.00%"
            },
            "eta": "calculating..."
        },
        "last_synced_snap": {
            "id": 15,
            "name": "d1_snap4"
        },
        "snaps_synced": 0,
        "snaps_deleted": 0,
        "snaps_renamed": 0
    },
}
---------------
After 'complete'
{
    "/d1": {
        "state": "syncing",
        "current_syncing_snap": {
            "id": 18,
            "name": "d1_snap5",
            "sync-mode": "delta",
            "avg_read_throughput_bytes": "11.66 MiB/s",
            "avg_write_throughput_bytes": "34.55 MiB/s",
            "crawl": {
                "state": "completed",
                "duration": "17s"
            },
            "datasync_queue_wait": {
                "state": "completed",
                "duration": "19s"
            },
            "bytes": {
                "sync_bytes": "149.94 MiB",
                "total_bytes": "149.94 MiB",
                "sync_percent": "100.00%"
            },
            "files": {
                "sync_files": 5000,
                "total_files": 5000,
                "sync_percent": "100.00%"
            },
            "eta": "0s"
        },
        "last_synced_snap": {
            "id": 15,
            "name": "d1_snap4"
        },
        "snaps_synced": 0,
        "snaps_deleted": 0,
        "snaps_renamed": 0
    }
}
-----
Also stored in last_sync_snap section
{
    "/d1": {
        "state": "idle",
        "last_synced_snap": {
            "id": 18,
            "name": "d1_snap5",
            "crawl_duration": "17s",
            "datasync_queue_wait_duration": "19s",
            "sync_duration": "44s",
            "sync_time_stamp": "8172.009480s",
            "sync_bytes": "149.94 MiB",
            "sync_files": 5000
        },
        "snaps_synced": 1,
        "snaps_deleted": 0,
        "snaps_renamed": 0
    }
}

Fixes: https://tracker.ceph.com/issues/73453
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-13 00:42:17 +05:30
Kotresh HR
0ae1ade4bc tools/cephfs_mirror: Add eta metrics
Add estimate time of completion for the current
syncing snapshot. The calculation takes into
account the average read/write throughput from
the start of snapshot sync and not the current
read/write throughput. So the ETA is affected
accordingly.

Sample output:
-------------
{
    "/d0": {
        "state": "syncing",
        "current_syncing_snap": {
            "id": 2,
            "name": "d0_snap0",
            "sync-mode": "full",
            "avg_read_throughput_bytes": "3.28 MiB/s",
            "avg_write_throughput_bytes": "71.03 MiB/s",
            "crawl": {
                "state": "completed",
                "duration": "1s"
            },
            "bytes": {
                "sync_bytes": "2.31 MiB",
                "total_bytes": "149.94 MiB",
                "sync_percent": "1.54%"
            },
            "files": {
                "sync_files": 67,
                "total_files": 5000,
                "sync_percent": "1.34%"
            },
            "eta": "calculating..."
        },
        "snaps_synced": 0,
        "snaps_deleted": 0,
        "snaps_renamed": 0
    }
}
------------------------------------------
{
    "/d0": {
        "state": "syncing",
        "current_syncing_snap": {
            "id": 2,
            "name": "d0_snap0",
            "sync-mode": "full",
            "avg_read_throughput_bytes": "12.17 MiB/s",
            "avg_write_throughput_bytes": "66.46 MiB/s",
            "crawl": {
                "state": "completed",
                "duration": "1s"
            },
            "bytes": {
                "sync_bytes": "26.64 MiB",
                "total_bytes": "149.94 MiB",
                "sync_percent": "17.77%"
            },
            "files": {
                "sync_files": 892,
                "total_files": 5000,
                "sync_percent": "17.84%"
            },
            "eta": "10s"
        },
        "snaps_synced": 0,
        "snaps_deleted": 0,
        "snaps_renamed": 0
    }
}

Fixes: https://tracker.ceph.com/issues/73453
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-13 00:42:02 +05:30
Kotresh HR
4294819d95 tools/cephfs_mirror: Add read/write throughput
The read throughput added measures the bytes
read per second from the source ceph filesystem.
Similarly, the write throughput added measures
the bytes written per second to the remote ceph
filesystem. It's derived from the time spent
in preadv and pwritev calls.

Sample output:
-------------
{
    "/d0": {
        "state": "syncing",
        "current_syncing_snap": {
            "id": 2,
            "name": "d0_snap0",
            "sync-mode": "full",
            "avg_read_throughput_bytes": "12.69 MiB/s",
            "avg_write_throughput_bytes": "54.49 MiB/s",
            "crawl": {
                "state": "completed",
                "duration": "1s"
            },
            "bytes": {
                "sync_bytes": "149.94 MiB",
                "total_bytes": "149.94 MiB",
                "sync_percent": "100.00%"
            },
            "files": {
                "sync_files": 5000,
                "total_files": 5000,
                "sync_percent": "100.00%"
            }
        },
        "snaps_synced": 0,
        "snaps_deleted": 0,
        "snaps_renamed": 0
    }
}
-------------

Fixes: https://tracker.ceph.com/issues/73453
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-13 00:41:56 +05:30
Kotresh HR
d593ad8aef tools/cephfs_mirror: Add crawl-state and sync-mode metric
The 'crawl' and 'sync-mode' metric is added.

sync-mode: full/delta,
"crawl": {
           "state": "completed",
           "duration": "37s"
       }

sync-mode:
---------
The 'sync-mode: full/delta' is added to peer status.
The 'delta' means, blockdiff along with snapdiff is
being used to sync the files where as 'full' means
full directory is crawled and each file is synced
entirely.

crawl:
-----
The state can be in-progress/completed. This
identifies whether the crawler thread is done
queuing the files for data sync threads.

The time taken for the duration is also shown.
If the crawl is in-progress, the duration
would show the time taken till then from the
start of the crawl. If the crawl state is
completed, then duration indicates total
time taken for the crawl.

The crawl duration is shown in "d h m s" format.
The existing 'sync_duration' in last_synced_snap
is also formatted

The values are as below. When crawl state is
completed, the 'total_files' metric doesn't
grow anymore.

crawl_duration:
--------------
The crawl_duration of last snapshot is saved in last_synced_snap
section as well.

Sample outputs:
---------------
{
    "/d0": {
        "state": "syncing",
        "current_syncing_snap": {
            "id": 2,
            "name": "d0_snap0",
            "sync-mode": "full",
            "crawl": {
                "state": "in-progress",
                "duration": "21s"
            },
            "bytes": {
                "sync_bytes": "149.25 MiB",
                "total_bytes": "176.47 MiB",
                "sync_percent": "84.57%"
            },
            "files": {
                "sync_files": 4931,
                "total_files": 5845,
                "sync_percent": "84.36%"
            }
        },
        "snaps_synced": 0,
        "snaps_deleted": 0,
        "snaps_renamed": 0
    }
}
------------------------------------------
{
    "/d0": {
        "state": "syncing",
        "current_syncing_snap": {
            "id": 2,
            "name": "d0_snap0",
            "sync-mode": "full",
            "crawl": {
                "state": "completed",
                "duration": "37s"
            },
            "bytes": {
                "sync_bytes": "891.39 MiB",
                "total_bytes": "901.52 MiB",
                "sync_percent": "98.88%"
            },
            "files": {
                "sync_files": 29656,
                "total_files": 30000,
                "sync_percent": "98.85%"
            }
        },
        "snaps_synced": 0,
        "snaps_deleted": 0,
        "snaps_renamed": 0
    }
}
---------
  {
        "/d0": {
            "state": "syncing",
            "current_syncing_snap": {
                "id": 3,
                "name": "d0_snap1",
                "sync-mode": "delta",
                "crawl": {
                    "state": "completed",
                    "duration": "15s"
                },
                "bytes": {
                    "sync_bytes": "120.20 MiB",
                    "total_bytes": "149.94 MiB",
                    "sync_percent": "80.16%"
                },
                "files": {
                    "sync_files": 4032,
                    "total_files": 5000,
                    "sync_percent": "80.64%"
                }
            },
            "last_synced_snap": {
                "id": 2,
                "name": "d0_snap0",
                "crawl_duration": "17s",
                "sync_duration": 45,
                "sync_time_stamp": "5642.805770s",
                "sync_bytes": "300.85 MiB",
                "sync_files": 10000
            },
            "snaps_synced": 1,
            "snaps_deleted": 0,
            "snaps_renamed": 0
        }
    }
-------------
{
    "/d0": {
        "state": "idle",
        "last_synced_snap": {
            "id": 2,
            "name": "d0_snap0",
            "crawl_duration": "17s",
            "sync_duration": "2m 38s",
            "sync_time_stamp": "9259.225009s",
            "sync_bytes": "901.52 MiB",
            "sync_files": 30000
        },
        "snaps_synced": 1,
        "snaps_deleted": 0,
        "snaps_renamed": 0
    }
}

Fixes: https://tracker.ceph.com/issues/73453
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-13 00:40:57 +05:30
Kotresh HR
1b6cbbbe23 tools/cephfs_mirror: Add inprogress bytes and files metric
Add following mirroring progress metrics to current_syncing_snap
as below

bytes:
  sync_bytes - bytes synced till now
  total_bytes - total bytes to be synced
  sync_percent - Percentage of bytes synced till now
files:
  total_files - Total files to be synced
  sync_files - files synced till now
  sync_percent - Percentage of files synced till now

sync_files and sync_bytes are also stored in last_synced_snap section
after the snapshot is synced.

The bytes is formatted as below.

Sample output:
--------
{
    "/d0": {
        "state": "syncing",
        "current_syncing_snap": {
            "id": 3,
            "name": "d0_snap1",
            "bytes": {
                "sync_bytes": "120.20 MiB",
                "total_bytes": "149.94 MiB",
                "sync_percent": "80.16%"
            },
            "files": {
                "sync_files": 4032,
                "total_files": 5000,
                "sync_percent": "80.64%"
            }
        },
        "last_synced_snap": {
            "id": 2,
            "name": "d0_snap0",
            "sync_duration": 45,
            "sync_time_stamp": "5642.805770s",
            "sync_bytes": "300.85 MiB",
            "sync_files": 10000
        },
        "snaps_synced": 1,
        "snaps_deleted": 0,
        "snaps_renamed": 0
    }
}

Fixes: https://tracker.ceph.com/issues/73453
Signed-off-by: Kotresh HR <khiremat@redhat.com>
2026-06-13 00:24:14 +05:30
Adam Kupczyk
23d7ec54d0 doc/rados/bluestore: Fix flags for bluestore_allocation_from_file
Set bluestore_allocation_from_file flags to 'startup'.
Without it, documentation claims the flag to be 'runtime updateable'.

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
2026-06-12 19:13:06 +02:00
Adam Kupczyk
ad02618007 doc/man/8/ceph-bluestore-tool: Add doc for recovery-compare command
Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
2026-06-12 19:12:49 +02:00
Adam Kupczyk
d7fedf1ece doc/rados/bluestore: Fast onode recovery
Add new page of documentation about new feature: multithread onode
recovery.

Signed-off-by: Adam Kupczyk <akupczyk@ibm.com>
2026-06-12 19:12:35 +02:00
Kobi Ginon
b4499d4f9a mgr/test_orchestrator: accept force_delete_data in remove_daemons
PR #68428 plumbed force_delete_data through orchestrator daemon/service
removal; update the test_orchestrator stub so orchestrator_cli QA
(test_mds_rm, etc.) does not fail with an unexpected keyword argument.

Fixes: https://tracker.ceph.com/issues/77374
Signed-off-by: Kobi Ginon <kginon@redhat.com>
2026-06-12 19:01:44 +03:00
Redouane Kachach
6a348c3b49
Merge pull request #69300 from rkachach/fix_issue_mgmt_gw_qa
qa: extend the ignore-list for the mgmt-gateway test suite

Reviewed-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-12 15:16:14 +02:00
Sridhar Seshasayee
4541e95ed0
Merge pull request #64618 from sseshasa/wip-fix-mclock-slow-ops-during-recovery
osd/scheduler: Classify subOp reads according to op priority for mClock

Reviewed-by: Radoslaw Zarzynski <rzarzyns@redhat.com>
Reviewed-by: Samuel Just <sjust@redhat.com>
2026-06-12 18:37:50 +05:30
Jon Bailey
1f365102a8
Merge pull request #69284 from JonBailey1993/remove_incorrect_unit_tests
test: Remove invalid unit test

Reviewed-by: Alex Ainscow <aainscow@uk.ibm.com>
2026-06-12 13:49:01 +01:00
Redouane Kachach
7a4cdf1e70
Merge pull request #68859 from ashjosh1git/ceph-tracker-74872-cephadm-debug-log
mgr/cephadm: Control cephadm files logging based on a mgr flag

Reviewed-by: Redouane Kachach <rkachach@ibm.com>
Reviewed-by: Adam King <adking@redhat.com>
2026-06-12 13:04:13 +02:00
Redouane Kachach
745f16a58a
qa: extend the ignore-list for the mgmt-gateway test suite
Let's add CEPHADM_AGENT_DOWN and CEPHADM_STRAY_DAEMON errors

Fixes: https://tracker.ceph.com/issues/77131
Signed-off-by: Redouane Kachach <rkachach@ibm.com>
2026-06-12 13:01:07 +02:00
Redouane Kachach
f2de87a95a
Merge pull request #69247 from rkachach/fix_issue_76979
mgr/cephadm: Don't skip OSDs with non-empty osdspec_affinity

Reviewed-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
Reviewed-by: Laura Flores <lflores@redhat.com>
2026-06-12 12:51:21 +02:00
Redouane Kachach
a46a9e45a3
Merge pull request #69299 from rkachach/fix_issue_77130
qa/cephadm: fix test_repos.sh for jammy nodes

Reviewed-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
Reviewed-by: Adam King <adking@redhat.com>
2026-06-12 12:50:24 +02:00
Redouane Kachach
33df2a3965
Merge pull request #67466 from rkachach/fix_new_secrets_mgr_module_v0
Adding new secrets mgr module support

Reviewed-by: Ernesto Puerta <epuertat@redhat.com>
2026-06-12 12:48:32 +02:00
Redouane Kachach
831c5c0dd4
Merge pull request #67384 from ashjosh1git/ceph-tracker-74986-validate-pro-name
python-common: Improve profile name string validation

Reviewed-by: Redouane Kachach <rkachach@ibm.com>
2026-06-12 12:36:59 +02:00
Redouane Kachach
ace8eb0aaf
Merge pull request #69080 from kginonredhat/issue-75365-Grafana-container-fails-to-start-reject-localhost
cephadm: set Grafana http_addr to 0.0.0.0 when unset

Reviewed-by: Redouane Kachach <rkachach@ibm.com>
Reviewed-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-12 12:34:42 +02:00
Redouane Kachach
75ee57a840
Merge pull request #68158 from rhcs-dashboard/fix-75826-main
mgr/cephadm: set default prometheus template in config-key store unless overridden by the user

Reviewed-by: Redouane Kachach <rkachach@ibm.com>
2026-06-12 12:30:14 +02:00
Redouane Kachach
1200eb68d3
Merge pull request #68428 from kginonredhat/wip-74058-force-delete-data
mgr/cephadm: plumb force_delete_data through daemon/service removal

Reviewed-by: Redouane Kachach <rkachach@ibm.com>
2026-06-12 12:28:50 +02:00
Aishwarya Mathuria
cb95853a1d
Merge pull request #66885 from amathuria/wip-amat-crimson-merge-support
Crimson PG merging support
2026-06-12 15:18:37 +05:30
Matty Williams
58640e23f0 osd: Enable omaps in Fast EC pools
Enable supports_omap flag in Fast EC pools.
Enable testing of omap operations in Fast EC pools.

Fixes: https://tracker.ceph.com/issues/77329
Signed-off-by: Matty Williams <Matty.Williams@ibm.com>
2026-06-12 09:54:04 +01:00
Ashwin M. Joshi
e81c276963 doc/cephadm: Document cephadm_binary_logging_level option
Add documentation for the new cephadm binary logging level configuration

Fixes: https://tracker.ceph.com/issues/74872
Signed-off-by: Ashwin M. Joshi <ashjosh1@in.ibm.com>
2026-06-12 14:09:18 +05:30
Ashwin M. Joshi
63b91836b0 mgr/cephadm: Control cephadm.log messages based on a new mgr logging level flag
Introduces a new 'cephadm_binary_logging_level' config option to control
  the verbosity of cephadm logging to persistent destinations (cephadm.log, syslog).

  - Adds --logging-level CLI flag (info, debug, error, warning)
  - Adds mgr/cephadm/cephadm_binary_logging_level config option
  - Applies logging level to file and syslog handlers
  - Console handlers maintain their defaults for terminal UX

Fixes: https://tracker.ceph.com/issues/74872
Signed-off-by: Ashwin M. Joshi <ashjosh1@in.ibm.com>
2026-06-12 13:37:31 +05:30
Kefu Chai
4a07196fde
Merge pull request #69329 from tchaikov/wip-crimson-cleanup
crimson/osd: coroutinize OSD::start and remove OSD::startup_time

Reviewed-by: Aishwarya Mathuria <amathuri@redhat.com>
2026-06-12 15:43:57 +08:00
Aashish Sharma
a1f90efb92
Merge pull request #69412 from rhcs-dashboard/fix-77263-main
mgr/dashboard: fix zone creation in rgw service creation form

Reviewed-by: Abhishek Desai <Abhishek.Desai1@ibm.com>
2026-06-12 13:02:10 +05:30
Sridhar Seshasayee
2cf1e713ce doc/rados/configuration: Remove wpq recommendation warning for EC clusters
Remove the warning that recommends using wpq scheduler as a fallback for EC
clusters. This issue is addressed by considering EC recovery reads as
background, assigning an accurate cost for those reads and tuning the QoS
parameters associated with best-effort class of operations.

Signed-off-by: Sridhar Seshasayee <sridhar.seshasayee@ibm.com>
2026-06-12 12:28:48 +05:30
Sridhar Seshasayee
77421c8366 mclock_common: adjust mClock profile parameters to prevent backfill starvation
Adjust the 'background_best_effort' queue parameters across the
three standard mClock profiles (high_client_ops, balanced, and
high_recovery_ops) to ensure best effort ops are not starved.

Previously, the 'background_best_effort' queue carried a default allocation
of 0% (MIN) reservation and a weight of 1 under these profiles. When
concurrent client traffic is dense, the zero-reservation for example completely
starves backfill sub-ops (MSG_OSD_EC_READ) on pools with
'allow_ec_optimizations' set to false. This starvation forces the Primary OSD
to hold internal BlueStore transactions and PG object locks for extended
windows, causing severe client median (50th) latency inflation.

To prevent background starvation and resolve the effects of the primary lock
retention, the profile configurations are tuned as follows:

The following profile changes forces low-cost sub-ops to clear out of peer
queues rapidly to drop  primary locks, which helps improve the client
completion latency and tail latency (95th, 99th and 99.5th) percentile.

1. high_client_ops profile:
   - Grant 'background_best_effort' a safe 5% minimum reservation.
   - Scale the queue weight to 4.

2. balanced profile:
   - Grant 'background_best_effort' a 5% minimum reservation.
   - Set the queue weight to 2.

3. high_recovery_ops profile:
   - Grant 'background_best_effort' a 5% minimum reservation.
   - Set the queue weight to 2.

4. Modify the mClock config reference documentation to reflect the tuning
   changes to the best-effort QoS parameters across the profiles.

Note on Proportional Scaling Compatibility:
Configuring these changes shifts total reservations to 105% (e.g., 50%
client + 50% recovery + 5% best-effort under the Balanced profile). Under
heavy concurrent saturation, mClock's internal controls resolves this
gracefully via proportional down-scaling, preserving the underlying
device bandwidth limits for different classes of clients. For example instead
of the client being allocated 50% bandwidth, a slightly lower reservation is
allocated while shifting the remaining bandwidth to the best-effort queue.
This minor scaling shift is virtually unnoticeable to the client application,
but it prevents the internal queue deadlocks.

Signed-off-by: Sridhar Seshasayee <sridhar.seshasayee@ibm.com>
2026-06-12 12:28:48 +05:30
Sridhar Seshasayee
5ae77683c1 mclock_common, mClockScheduler: Add perf counters for scheduler ops
Add perf counters to show the status pertaining to the number of ops,
dynamic queue lengths, queue latency and bytes read for the following
ops handled in the high queues and in the scheduler queues:
 - peering
 - client
 - ec reads/writes
 - ec recovery reads

Additional counters can be added in the future based on the requirement.

Signed-off-by: Sridhar Seshasayee <sridhar.seshasayee@ibm.com>
2026-06-12 12:28:48 +05:30
Sridhar Seshasayee
e79befea32 src/messages, osd: Calculate and set cost for subOpReads for mClock scheduler
Previously, sub-op reads returned a hardcoded cost of 0, bypassing
mClock's background bandwidth and tag calculation mechanisms. This
allowed backfill operations to proceed un-metered, occasionally causing
backend resource contention and driving up client tail latencies.

Cost is calculated based on whether the complete chunk/shard or a subchunk
needs to be read. The possible cases are:
1. Read the complete chunk aligned length:
   - Cost is set to the length of the chunk aligned extent size.
2. Fragmented reads:
   - Consider the subchunk length and count to calculate the cost.
   - compute_cost evaluates the exact layout of fragmented shard bytes on
     disk by summing up the active subchunk allocations exactly once
     (`fragmented_shard_bytes += k.second * subchunk_size`).
   - Linear Extent Scaling: Scale the baseline footprint cleanly by
     multiplying it against the true count of read extents (`tl.size()`),
     achieving a highly efficient O(N) time complexity.

This linear cost model is compatible with pools running with
'allow_ec_optimizations' set to true. Under the FastEC optimized
pipeline, most operations are unified and bypass fragment slicing,
meaning requests will primarily match the Case 1 chunk-aligned path.
In Case 2 where applicable, the O(N) loop ensures that cost will
scale proportionally according to the layout.

It is important to note that the amount of data to read was set to an upper
bound defined by osd_recovery_max_chunk (8 MiB) and was rounded up to the
stripe width. The reason for setting a higher than actual upper bound is that
there may be cases where the object doesn't have the xattrs yet to determine
its size. Therefore, the amount to read was ultimatly set to ~(8 MiB / k)
where k is the number of data shards. This can cause mClock to prolong
the recovery times as items stay longer in the queue. To address this, the
amount to read is set to the remaining length of the object to recover
if the object size is known. Otherwise, the amount to read is set to the
recovery chunk size as before. Therefore, in some cases, only the first
recovery read could be costly if the object context is not known.

The MOSDECSubOpRead class introduces the following:
 - cost member. This necessitates an increment to the HEAD_VERSION and
   appropriate handling within the encode and decode methods.
 - compute_cost() that is called when creating the message by
   ECCommonL::ReadPipeline::do_read_op(). This calls into ECSubRead::cost()
   that performs the actual calculations to set the cost based on the cases
   mentioned above.
 - The same sequence applies to the EC optimized path in
   ECCommon::ReadPipeline::do_read_op().

Fixes: https://tracker.ceph.com/issues/71655
Signed-off-by: Sridhar Seshasayee <sridhar.seshasayee@ibm.com>
2026-06-12 12:28:09 +05:30
Sridhar Seshasayee
a0464c3d82 osd/scheduler: Classify EC subOp reads according to op priority for mClock
The change brings MSG_OSD_EC_READ into the fold of mClock scheduler. This
improves the scheduling of client and other classes of operation as they
are no longer unnecessarily preempted by the 'immediate' queue.
EC SubOps are now handled as follows:

 - EC SubOp reads generated during recovery will either go into the
   'background_recovery' or 'background_best_effort' class based on
   the recovery priority set for the op. EC SubOp reads generated due
   to client will continue to be classified as 'immediate'.

 - EC SubOp writes generated as a result of client operations will
   continue to be classified as 'immediate'.

 - EC SubOp replies are considered high priority and therefore
   continue to be classed as 'immediate'.

Fixes: https://tracker.ceph.com/issues/71655
Signed-off-by: Sridhar Seshasayee <sridhar.seshasayee@ibm.com>
2026-06-12 12:26:52 +05:30
Sridhar Seshasayee
913e891176 osd/scheduler/mClockScheduler: Fix line alignments
Signed-off-by: Sridhar Seshasayee <sridhar.seshasayee@ibm.com>
2026-06-12 12:26:52 +05:30
Sridhar Seshasayee
89c7b217ef osd/scheduler/mClockScheduler: Log the size of high priority queues.
Signed-off-by: Sridhar Seshasayee <sridhar.seshasayee@ibm.com>
2026-06-12 12:26:52 +05:30
Sridhar Seshasayee
0b84dbe701 src/common/mclock_common: Fix output formatting of SchedulerClass
The earlier output formatting was resulting in the value and string
representation of the SchedulerClass being clubbed together for
e.g., "3client"

The formatting is now fixed to log SchedulerClass as "3 (client)".

Signed-off-by: Sridhar Seshasayee <sridhar.seshasayee@ibm.com>
2026-06-12 12:26:52 +05:30
Kefu Chai
8dfb7b10f9 common/cmdparse: tighten the check for positional and req in desc
we encode the description of a certain command, with comma-separated
values, which are represented in "key=value" form. the keys are a set of
configuration names, and their values' types are known. among them,
"positional" and "req" are of boolean type. before this change, we
consider "true" and "True" as true, otherwise we consider the value as
false. but this setting leads to confusion and could be a headache for
maintainers.

since all commands are determined at compile-time, and all commands'
descs use "req=true" or "req=false" if "req" is specified, we are
allowed to tighten the check to enforce that these values are "true"
or "false".

this change serves as part of a sanity check of all command
descriptions, as all command descriptions are formatted into JSON
with the "get_command_description" asok command and the mon command
with the same name. and it does not change the runtime behavior.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-12 14:25:31 +08:00
Kefu Chai
1f43c35096 mon: correct typo in command desc
in 46586b22, we introduced the "osd pool application get" command, but
in this command's desc field, we had a typo: "req=fasle". this didn't
cause any issue, because we always consider "true" or "True" as true,
and false otherwise when formatting a command desc into JSON.

but it's still not correct. let's fix this typo.

Fixes: https://tracker.ceph.com/issues/77251
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-12 14:25:31 +08:00
Kobi Ginon
0347cc8591 cephadm: address Copilot review on list-networks BGP parsing
Skip default routes in ip -j BGP JSON parsing, fix black line-length
in loopback check, and update list-networks CLI help for --allow-lo-routes
and --allow-bgp-routes.
Signed-off-by: Kobi Ginon <kginon@redhat.com>
2026-06-12 09:22:18 +03:00
Shweta Bhosale
fbfe653454 ceph.spec.in: disable -P in python shebang for cephadm binary
Fixes: https://tracker.ceph.com/issues/77340
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-12 10:42:48 +05:30
Afreen Misbah
2017866ee6
Merge pull request #69136 from syedali237/rhcs-dashboard/hosts
mgr/dashboard: migrated host table tabs to resource pages

Reviewed-by: Afreen Misbah <afreen@ibm.com>
2026-06-12 00:38:30 +05:30
Gregory Farnum
57d34f8e19
libcephfs: increment extra version to indicate new flock functions
Signed-off-by: Greg Farnum <gfarnum@redhat.com>

Signed-off-by: Gregory Farnum <greg@gregs42.com>
2026-06-11 11:50:16 -07:00
Casey Bodley
d27261d0c2 Reapply "qa/rgw/crypt: disable failing kmip testing"
This reverts commit fd20467981.

kmip tests are failing again on ubuntu 24 because PyKMIP doesn't support
python 3.12. we'll be removing ubuntu 22 from main, so can't just pin the
test to that distro in the meantime

we're expecing the nvmeof team to add python 3.12 support in our
ceph/PyKMIP fork, and can reenable kmip testing once that happens

Fixes: https://tracker.ceph.com/issues/76995

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2026-06-11 14:43:31 -04:00
David Galloway
b6f8147510
Merge pull request #69408 from tchaikov/wip-spec-update-alternatives
ceph.spec.in: add update-alternatives as runtime dep and correct macro call
2026-06-11 14:01:05 -04:00
Afreen Misbah
02bcaaec5d
Merge pull request #68031 from syedali237/rhcs-dashboard/osd-component
mgr/dashboard : carbonize OSD form component

Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Devika Babrekar <devika.babrekar@ibm.com>
2026-06-11 20:46:32 +05:30
Kamoltat (Junior) Sirivadhna
cce2a2fcd1
Merge pull request #69402 from Ericmzhang/wip-fix-mon-stretch_cluster
qa: Fix stretch_cluster.py missing function call
Reviewed-by: Kamoltat Sirivadhna <ksirivad@redhat.com>
2026-06-11 09:47:19 -04:00
Sridhar Seshasayee
6761549c5a mgr/DaemonServer: Aggregate and globally sort OSDs for ok-to-upgrade
The 'ok-to-upgrade' command output sorting did not scale accurately
when target CRUSH buckets contained multiple child buckets (e.g., a
chassis containing multiple hosts). OSDs were previously sorted
individually per child bucket and appended sequentially. This created
fragmented, per-host sort segments rather than a globally sorted list
for the parent bucket.

Changes:

1. Fix the issue above by aggregating all child OSDs into a single vector prior
to executing a single, global sort operation based on PG counts. Additionally,
optimize memory efficiency and future-proof the logic by reserving continuous
vector blocks to avoid dynamic heap reallocations.

2. Add integration tests with chassis and rack based CRUSH hierarchies which
verifies the ok-to-upgrade functionality. In addition, the tests crucially
verify the order of OSDs returned is according to the ascending order of
acting PG count. Additionally, make minor fix-ups to lines that determine the
length of a list in JSON response by removing the redundant "| bc".

Fixes: https://tracker.ceph.com/issues/77272
Signed-off-by: Sridhar Seshasayee <sridhar.seshasayee@ibm.com>
2026-06-11 17:42:59 +05:30
ShreeJejurikar
b126cffe85 rgw: emit bucket logging record for lifecycle multipart upload abort
Signed-off-by: ShreeJejurikar <shreemj8@gmail.com>
2026-06-11 17:42:28 +05:30
Ronen Friedman
cae12eccdd crimson/osd: fix PGBackend::remove() to return ENOENT on no-op deletes
PGBackend::remove() was returning success when asked to delete a
non-existent object or an already-whiteout object that must remain
a whiteout. The classic OSD returns -ENOENT in both cases. Fix both
paths to return enoent, and remove the duplicate !os.exists check.

Fixes: https://tracker.ceph.com/issues/76529
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
2026-06-11 11:38:02 +00:00
Prasanna Kumar Kalever
e98037a639 rbd-mirror: fix missing initialization of Peer UUID
Initialize the uuid member in the Peer constructor and include it in
stream output operator to ensure correct peer identification in logs.

Signed-off-by: Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
2026-06-11 16:46:45 +05:30
Aashish Sharma
95a8c70be8 mgr/dashboard: fix zone creation in rgw service creation form
The zone creation request from the rgw service creation form was missing
the tier_type, sync_from and sync_from_all properties as a result the
zone creation was failing. This PR tends to fix this issue.

Fixes: https://tracker.ceph.com/issues/77263

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2026-06-11 14:49:04 +05:30
Redouane Kachach
8a28a1578e
mgr: adding ceph_secrets_xxx.py files to the build/packaging
Fixes: https://tracker.ceph.com/issues/74562
Assisted-by: Claude <claude.ai>
Assisted-by: ChatGPT <chatgpt.com>
Signed-off-by: Redouane Kachach <rkachach@ibm.com>
2026-06-11 10:58:06 +02:00
Redouane Kachach
ead4ef7f9a
doc/mgr/ceph_secrets: add documentation for the ceph_secrets module
Document CLI commands (set/get/get-value/ls/rm), the Python API via
CephSecretsClient, secret URI embedding and resolution, and epoch-based
change detection.

Fixes: https://tracker.ceph.com/issues/74562
Assisted-by: Claude <claude.ai>
Assisted-by: ChatGPT <chatgpt.com>
Signed-off-by: Redouane Kachach <rkachach@ibm.com>
2026-06-11 10:57:34 +02:00
Redouane Kachach
431fac7ade
mgr/ceph_secrets: add unit tests for all modules
Add pytest coverage for the full stack: secret types and URI/path
parsing, storage backend contract, Mon KV store (CRUD, epoch,
serialization, corruption handling), SecretMgr (scan/resolve),
module RPC surface and CLI handlers, and the CephSecretsClient
wrapper. Gate test imports on the UNITTEST env var following the
SMB module pattern.

Fixes: https://tracker.ceph.com/issues/74562
Assisted-by: Claude <claude.ai>
Assisted-by: ChatGPT <chatgpt.com>
Signed-off-by: Redouane Kachach <rkachach@ibm.com>
2026-06-11 10:56:42 +02:00
Redouane Kachach
92981c7687
mgr/ceph_secrets: add ceph_secrets module to tox.ini
Adds ceph_secrets to tox.ini mypy and flake8 targets.

Signed-off-by: Redouane Kachach <rkachach@ibm.com>
2026-06-11 10:56:42 +02:00
Redouane Kachach
e737f8b73f
mgr: add CephSecretsClient wrapper for ceph_secrets RPC
Add a thin typed client around mgr.remote() for consuming the
ceph_secrets module. Exposes get/set/rm, epoch and version queries,
batch version fetch, scan and resolve helpers. Lives alongside
ceph_secrets_types.py so any mgr module can import it without
depending on the ceph_secrets package directly.

Fixes: https://tracker.ceph.com/issues/74562
Assisted-by: Claude <claude.ai>
Assisted-by: ChatGPT <chatgpt.com>
Signed-off-by: Redouane Kachach <rkachach@ibm.com>
2026-06-11 10:56:42 +02:00
Redouane Kachach
349c12c6d0
mgr/ceph_secrets: add 'ceph secret' CLI commands and input parsing
This commit has the following changes:

1) Add the ceph_secrets mgr module entrypoint and wire it to
SecretMgr. Implement the core RPC surface consumed by other mgr
modules (secret_ls/get/set/rm, secret_get_value, secret_get_version)
and keep the implementation focused on the internal API.

2) Add user-facing CLI commands (ceph secret ls/get/set/rm) using
parse_secret_path. Secret data is accepted via -i (inbuf) only for
script-friendly usage. Add secret get-value for plain-string output
without a JSON envelope. Ensure consistent JSON output and error
mapping to EINVAL/ENOENT, while preserving safe non-reveal defaults
unless explicitly requested.

3) Add the scanning and resolution helpers (scan_refs,
scan_unresolved_refs, resolve_object) through the ceph_secrets module
RPC API. This lets consumers reliably detect secret:/... references and
resolve them inside nested objects without duplicating logic. The
behavior is delegated to SecretMgr to keep parsing/resolution
consistent across the stack.

Fixes: https://tracker.ceph.com/issues/74562
Assisted-by: Claude <claude.ai>
Assisted-by: ChatGPT <chatgpt.com>
Signed-off-by: Redouane Kachach <rkachach@ibm.com>
2026-06-11 10:56:37 +02:00
Shweta Bhosale
5cfab473be mgr/cephadm: For updating NFS backends in HAProxy, send a SIGHUP signal to reload the configuration instead of restart
Fixes: https://tracker.ceph.com/issues/73633

Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 14:18:51 +05:30
Redouane Kachach
9c220eec0e
mgr/ceph_secrets: add SecretMgr for secrets handling and resolution
Introduce SecretMgr to encapsulate higher-level behavior on top of
SecretStoreMon: listing helpers, scan_refs, scan_unresolved_refs, and
resolve_object (walk nested dict/list structures). This keeps
parsing/substitution logic out of the mgr module entrypoint and makes
consumer behavior consistent. The module can now resolve secret://…
references deterministically and provide structured scan output.

Fixes: https://tracker.ceph.com/issues/74562
Assisted-by: Claude <claude.ai>
Assisted-by: ChatGPT <chatgpt.com>
Signed-off-by: Redouane Kachach <rkachach@ibm.com>
2026-06-11 10:48:27 +02:00
Redouane Kachach
0937270bdd
mgr/ceph_secrets: add SecretStoreMon mon-kv store implementation
Introduce SecretRecord (data, metadata, versioning, timestamps) and
the canonical KV prefix (secret_store/v1/…). Add JSON serialization
helpers (to_json) including the ability to omit secret data unless
explicitly requested. This commit defines the “what we store and how
it looks” without wiring any mgr interactions yet.

Add SecretStoreMon implementing the backend using mgr’s KV
store (get_store, set_store, prefix listing). Implement set/get/rm
semantics, version increments, and list-by-prefix queries for
namespace/scope/target. This isolates persistence logic from CLI/RPC
concerns and provides deterministic record behavior for later layers.

Fixes: https://tracker.ceph.com/issues/74562
Assisted-by: Claude <claude.ai>
Assisted-by: ChatGPT <chatgpt.com>
Signed-off-by: Redouane Kachach <rkachach@ibm.com>
2026-06-11 10:47:36 +02:00
Matty Williams
20991721bb osd: Fix bugs with omap support in EC pools
Change handling of reads_sent bool in do_read_op.
Use errors to determine omap_satisfied in handle_sub_read_reply.
When a head object is deleted, and a whiteout is created, a REPLACE op should be used so that the ECOmapJournal knows to handle the operation as a delete. This should avoid the scenario where omap updates that should be applied to a clone, are instead applied to the whiteout object.

Fixes: https://tracker.ceph.com/issues/77329
Signed-off-by: Matty Williams <Matty.Williams@ibm.com>
2026-06-11 09:46:44 +01:00
Redouane Kachach
16ae0b3d2b
mgr/ceph_secrets: add storage backend protocol for mgr KV secrets
Define a minimal backend protocol for secret persistence
operations (get/set/rm/list), keeping the module implementation
decoupled from the backing store details. For now we will start with
monstore-db as secure KV store but the idea is to extend this to other
backends such as Vault.

Fixes: https://tracker.ceph.com/issues/74562
Assisted-by: Claude <claude.ai>
Assisted-by: ChatGPT <chatgpt.com>
Signed-off-by: Redouane Kachach <rkachach@ibm.com>
2026-06-11 10:45:58 +02:00
Redouane Kachach
91da0fd6ae
mgr/ceph_secrets: add secret reference types and parsing helpers
Introduce the shared types and parsing logic used across the secrets
module: secret scopes, secret references, and the exception hierarchy.
Includes validation for all supported addressing forms and clear
error messages on malformed input.

Fixes: https://tracker.ceph.com/issues/74562
Assisted-by: Claude <claude.ai>
Assisted-by: ChatGPT <chatgpt.com>
Signed-off-by: Redouane Kachach <rkachach@ibm.com>
2026-06-11 10:43:41 +02:00
Shubha Jain
391383dae2
cephadm: report intentionally stopped daemons as stopped
node-exporter exits with status 143 when terminated via SIGTERM.
Systemd treats this as a failure by default, causing cephadm to
report an intentionally stopped daemon as being in an error state.

Allow daemon forms to provide daemon-specific success exit status
codes and generate a corresponding systemd SuccessExitStatus
drop-in when required.

For node-exporter, configure exit status 143 as successful so
intentional stops are reported as stopped rather than failed.

Fixes: https://tracker.ceph.com/issues/68863
Signed-off-by: Shubha Jain <SHUBHA.JAIN1@ibm.com>
2026-06-11 13:37:10 +05:30
Guillaume Abrioux
7b779e84d7
Merge pull request #69391 from guits/fix-raw-activate
ceph-volume: fix raw activate when device path is stale
2026-06-11 09:34:49 +02:00
ShreeJejurikar
efeb0fe52f rgw: emit bucket logging records for lifecycle delete actions
LC delete actions now emit LIFECYCLE.DELETE.OBJECT journal records
via no-req_state overloads of the bucketlogging frontend.

Signed-off-by: ShreeJejurikar <shreemj8@gmail.com>
2026-06-11 12:48:30 +05:30
Shweta Bhosale
8ef8a1e2f3 mgr/cephadm: Signal handling while daemon reconfig
Fixes: https://tracker.ceph.com/issues/73633
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 11:37:37 +05:30
Kefu Chai
7fc3a9e12f ceph.spec.in: require update-alternatives for the osd scriptlets
ceph-osd-crimson and ceph-osd-classic call update-alternatives in their
%posttrans and %preun scriptlets but don't depend on it. declare it as a
scriptlet dependency so the binary is there when they run.

Fixes: https://tracker.ceph.com/issues/77323
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-11 13:32:25 +08:00
Kefu Chai
b571051663 ceph.spec.in: use %{_sbindir} instead of ${_sbindir} in osd %preun
a37b5b5bde added %preun scriptlets that use ${_sbindir}, which is
shell syntax rather than an rpm macro, so it expands to empty at run
time and the scriptlet runs "/update-alternatives", failing on
uninstall/upgrade with:

  /var/tmp/rpm-tmp.K1fvm3: line 2: /update-alternatives: No such file or directory
  error: %preun(ceph-osd-crimson-2:20.3.0-5054.g33c1d671.el9.x86_64) scriptlet failed, exit status 127
  Error in PREUN scriptlet in rpm package ceph-osd-crimson.

use %{_sbindir}, like the %posttrans --install lines already do, so it
expands to /usr/sbin/update-alternatives at build time.

Fixes: https://tracker.ceph.com/issues/77323
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-11 13:32:14 +08:00
Shweta Bhosale
e05ba723be cephadm: moved check-online,sysctl-dir subcommands under _orch
Moved above mentioned commands under the existing cephadm _orch namespace

Fixes: https://tracker.ceph.com/issues/74045
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 10:40:34 +05:30
Shweta Bhosale
4d35c61283 mgr/cephadm: adding cephadm deploy-file command for mgr file writes
Add a cephadm "deploy-file" subcommand that reads raw file bytes from stdin
and atomically installs them at an absolute destination (--fsid, --path,
optional --mode, --uid/--gid). Use it from the mgr for client conf/keyring
sync and tuned profiles instead of SSHManager.write_remote_file.
Keep staging the cephadm binary over SSH via _write_remote_file when sudo
hardening is off, the invoker deploy_binary path is unchanged when hardening
is on.
SSH: pass encoding=None to asyncssh conn.run when stdin is bytes so binary
payloads (deploy-file) are not UTF-8-encoded as str. Only add the input kwarg
when stdin is not None.

Fixes: https://tracker.ceph.com/issues/74045
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 10:40:34 +05:30
Shweta Bhosale
56895ee07b cephadm: added sysctl-dir command for tuned profiles, use remove-file for strays
Added cephadm sysctl-dir with mutually exclusive --list (sorted basenames
under /etc/sysctl.d) and --apply-system (sysctl --system).
The cephadm mgr tuned profile logic calls sysctl-dir for listing and
reload, and remove-file for stray *-cephadm-tuned-profile.conf files

Fixes: https://tracker.ceph.com/issues/74045
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 10:40:34 +05:30
Shweta Bhosale
c40c6296a1 cephadm: added remove-file command for client file cleanup
Add a cephadm remove-file subcommand that deletes only regular files.
Have the mgr call it from CephadmServe._write_client_files when pruning
stale client keyrings instead of running ssh rm, so removal goes through
the same cephadm/invoker path as other host operations.

Fixes: https://tracker.ceph.com/issues/74045
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 10:40:34 +05:30
Shweta Bhosale
07edf265cb cephadm: Added cephadm check-online command for offline host watcher
Added a `cephadm check-online` subcommand that returns 0,
and use it from the mgr cephadm offline host watcher via
`CephadmServe._run_cephadm` instead of SSHing `true` directly.

Fixes: https://tracker.ceph.com/issues/74045
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 10:40:34 +05:30
Shweta Bhosale
03aa50ef23 cephadm: Removed 'cephadm exec' command
Fixes: https://tracker.ceph.com/issues/74045

Revert "mgr/cephadm: Use 'cephadm exec' to execute bash commands, just commands requires to deploy cephadm binary and which command should be executed directly"
This reverts commit 3a7000aa5010a8a4ad862b321aa32f36a49ac283.

Revert "cephadm: added cephadm exec command to execute shell commands"
This reverts commit 53d87b0697b5a6bd08caa03d0d795880ba4b742b.

Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 10:40:34 +05:30
Shweta Bhosale
23cc154e6a cephadm: Addressed review comments for validate_user_exists and command_run of invoker.py
Fixes: https://tracker.ceph.com/issues/74045

Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 10:40:34 +05:30
Shweta Bhosale
9b9493345a mgr/cephadm: Handled cephadm binary creation with different ssh user
Since we execute the cephadm binary directly with sudo,
we get exit code 1 with a “command not found” error if the binary does not exist.
Handled it in run_cephadm to generate cephadm binary in this case.

Fixes: https://tracker.ceph.com/issues/74045
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 10:40:34 +05:30
Shweta Bhosale
4995a59247 Revert "mgr/cephadm: Handle cephadm set user configuration for add node"
This reverts commit 46bcde4f812afb6a822ecb782815c49156674ea9.
Fixes: https://tracker.ceph.com/issues/74045

Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 10:40:33 +05:30
Shweta Bhosale
30cb2e354b cephadm: fix bootstrap ordering, configure SSH keys before set-user
Fixes: https://tracker.ceph.com/issues/74045

Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 10:40:33 +05:30
Shweta Bhosale
0ecd33ca59 doc: Documentation for ssh hardening
Fixes: https://tracker.ceph.com/issues/74045
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 10:40:33 +05:30
Shweta Bhosale
3635d77507 cephadm: Add cephadm_invoker.py in cephadm rpm at /usr/libexec/cephadm_invoker.py
Fixes:https://tracker.ceph.com/issues/74045
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 10:40:33 +05:30
Shweta Bhosale
d4f6188d15 mgr/cephadm: Added prepare host for ssh hardning command and add node handling if ssh hardning is enabled
It will perform following steps
1. prepare host for ssh hardning (enable passwordless ssh, install cephadm RPM, update sudoers)
2. enable ssh hardning
3. set cephadm user

Fixes:https://tracker.ceph.com/issues/74045
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 10:40:33 +05:30
Shweta Bhosale
a22ee36598 mgr/cephadm: Cephadm to call invoker if ssh hardening is enabled
Fixes: https://tracker.ceph.com/issues/74045

Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 10:40:33 +05:30
Shweta Bhosale
5bac52b443 cephadm: Added invoker script
Fixes: https://tracker.ceph.com/issues/74045

Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 10:40:33 +05:30
Shweta Bhosale
a026eb34a2 mgr/cephadm: Use 'cephadm exec' to execute bash commands, just commands requires to deploy cephadm binary and which command should be executed directly
Fixes: https://tracker.ceph.com/issues/74045
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 10:40:33 +05:30
Shweta Bhosale
79924f8fc9 cephadm: added cephadm exec command to execute shell commands
Fixes: https://tracker.ceph.com/issues/74045
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 10:40:33 +05:30
Shweta Bhosale
790181e282 mgr/cephadm: Handle cephadm set user configuration for add node
Fixes: https://tracker.ceph.com/issues/74045
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 10:40:33 +05:30
Shweta Bhosale
04ec9abec4 mgr/cephadm: Setup user's sudoers and public keys before setting cephadm user
Fixes: https://tracker.ceph.com/issues/74045
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-06-11 10:40:32 +05:30
Aashish Sharma
b2ed52b2a4 mgr/dashboard: Fix username validation for special characters by URL-encoding user lookup requests
When validating usernames, special characters such as '#' were not being URL-encoded before being appended to the user lookup endpoint.
Because the browser treats '#' as a URL fragment, the request path was truncated, causing the backend to return an incorrect response and the UI to display a misleading "User already exists" validation error.

Fixes: https://tracker.ceph.com/issues/77129

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2026-06-11 10:39:57 +05:30
Aashish Sharma
50bf8ce2eb mgr/dashboard: rbd-mirroring - hide create/import token buttons for
read-only users

Fixes: https://tracker.ceph.com/issues/77115

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2026-06-11 10:38:49 +05:30
Aishwarya Mathuria
f94860bd17 crimson/osd: reject Seastore PG merges across shards
Seastore cannot merge collections between reactor shards currently.
On cross-shard detection, tell the monitor the source PG is not ready
(via MOSDPGReadyToMerge{ ready=false }) so the unsafe pg_num decrement
is never proposed, then send MOSDPGStopMerge to clamp pg_num_target and
permanently disable further shrink for the pool.

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
2026-06-11 10:25:49 +05:30
Aishwarya Mathuria
4e44d8a160 qa/suites/crimson: Add a test for PG merging in the crimson suite
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
2026-06-11 10:25:49 +05:30
Aishwarya Mathuria
1f1db7d0cd mon/OSDMonitor: introduce per-pool crimson_allow_pg_merge flag
Switch Crimson PG merge gating from a global config to a pool-scoped flag.

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
2026-06-11 10:25:45 +05:30
Aishwarya Mathuria
0aa3da13cd doc/dev/crimson: Add explanation for PG merging
Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
2026-06-11 10:19:14 +05:30
Aishwarya Mathuria
7c033ce199 crimson/osd: implement PG merge detection and orchestration in PGAdvanceMap
Integrate PG merge handling into the map advancement pipeline.

When pg_num shrinks between epochs, check_for_merges() returns a
merge_result_t describing whether this PG is a merge source, target, or
not involved. start() stops advancing through later epochs once a merge
is detected, then either finish_merge_advance() or the normal activate
path runs so complete_rctx() always happens in one place.

- check_for_merges(): detect pg_num shrink and dispatch merge_pg().
- merge_pg(): merge-only work — Seastore eligibility, source handoff
  setup, target rendezvous collection and PG::merge_from().
- finish_merge_advance(): commit rctx and complete the role-specific
  steps (source: complete_rctx, stop, register_merge_source; target:
  handle_advance_map, handle_activate_map, complete_rctx).

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
2026-06-11 10:19:14 +05:30
Aishwarya Mathuria
7f7439cc81 crimson/osd/pg: implement PG::merge_from
Add PG::merge_from to execute the merge of source PGs into a target PG.
This function builds a transaction to remove source-specifc metadata
objects and merge source collections into the target collection.

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
2026-06-11 10:19:14 +05:30
Aishwarya Mathuria
e12454b5fe crimson/osd: per-PG rendezvous for cross-shard merge source handoff
Add infrastructure so source PGs can be extracted from their birth
shard, moved to the target shard, and collected by the target PG before
merge proceeds.

Cross-shard safety: PGs are tied to their birth_shard for destruction.
register_merge_source() uses extract_pg() to detach the source,
seastar::foreign_ptr to hop cores, and
crimson::local_shared_foreign_ptr on the target so release routes
destruction back to the birth shard.

Synchronization: replace the per-shard ShardServices merge_info_t
registry (shared_promise waiters, ready_pgs staging, and cleanup hooks)
with merge state on the target PG itself. Source-side
register_merge_source() delivers PGs via PG::add_merge_source(); the
target waits in PG::collect_merge_sources(n) on a per-PG semaphore.
Duplicate source registrations are ignored. PG::stop() breaks the
semaphore so shutdown does not hang.

ShardServices::register_merge_source() and extract_pg() live in
shard_services; rendezvous types and methods live on PG.

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
2026-06-11 10:19:14 +05:30
Aishwarya Mathuria
e7cf68899f crimson/osd/pg: modify stop() function
This function ensures that when a PG is being removed or
merged and it calls stop() - it will clear primary state, and
notify the Monitor to clear any pending merge flags.

It will also call client_request_orderer.clear_and_cancel() ensuring
all remaining client requests are properly completed. This is needed
for merging in particular since on_change() is never called for the merge epoch
(handle_advance_map is skipped after merge detection), so
clear_and_cancel() is never invoked on the source PG's orderer.

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
2026-06-11 10:19:14 +05:30
Aishwarya Mathuria
7f970fcda0 crimson/osd/shard_services: inherit from peering_sharded_service
Update ShardServices to inherit from seastar::peering_sharded_service.
This allows the service to access its own sharded container directly
via container() rather than manually storing a reference to it.

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
2026-06-11 10:19:14 +05:30
Aishwarya Mathuria
473104c243 crimson/osd: Add functions to notify mon when PGs are ready to merge
When a PG is in the pending merge state it is >= pg_num_pending and <
pg_num. When this happens, IO is paused and once the PG peers we notify
the mon that we are idle and safe to merge.
Use Gated for merge notify callbacks.

Signed-off-by: Aishwarya Mathuria <amathuri@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-11 10:19:14 +05:30
Eric Zhang
c22d5b4f41 qa: Fix task missing function call
lambda was missing function call so always returned true

Signed-off-by: Eric Zhang <emzhang@ibm.com>
2026-06-10 12:41:21 -07:00
Syed Ali Ul Hasan
15e47f930d mgr/dashboard: migrated host table tabs to resource pages
Fixes :  https://tracker.ceph.com/issues/76712
Signed-off-by: Syed Ali Ul Hasan <syedaliulhasan19@gmail.com>
2026-06-10 23:08:29 +05:30
Syed Ali Ul Hasan
78dad340ab mgr/dashboard: carbonized OSD form component
Fixes: https://tracker.ceph.com/issues/68265
Signed-off-by: Syed Ali Ul Hasan <syedaliulhasan19@gmail.com>
2026-06-10 23:00:36 +05:30
Devika Babrekar
8e7dc4e4e7 mgr/dashboard: Fix for EC profile creation modal scrollbar
Fixes: https://tracker.ceph.com/issues/77298

Signed-off-by: Devika Babrekar <devika.babrekar@ibm.com>
2026-06-10 20:11:48 +05:30
Anoop C S
31e1c7bd5e qa/smb: Test SID resolution in AD joined containers
Verify that rpcclient lookupsids resolves domain user SIDs correctly
inside the smbd container, preventing regressions on /run bind mount
permissions that break smbd to winbindd communication.

Fixes: https://tracker.ceph.com/issues/77120
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2026-06-10 17:05:18 +05:30
Guillaume Abrioux
1ee1fc14fe
ceph-volume: fix raw activate when device path is stale
This changes unlink_bs_symlinks to use os.path.lexists instead
of os.path.exists. It can happen that devices get renumbered,
in that case, the OSD symlink still exists but its target device
is gone which means os.path.exists returns False, so the symlink
is never cleaned up and ceph-volume activate can fail later.

Fixes: https://tracker.ceph.com/issues/77295

Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
2026-06-10 13:22:14 +02:00
Anoop C S
c0fda141e7 cephadm/smb: Bind mount /run with 0755
The host side 'run' directory under /var/lib/ceph/<fsid>/<svc_dir> bind
mounted into SMB containers is created with mode 0770, preventing any
non owner processes from accessing unix domain sockets or named pipes
under /run. This breaks smbd to winbindd communication, causing SID to
name resolution failures for AD joined deployments. Therefore change
permissions to 0755 to match standard /run semantics.

Fixes: https://tracker.ceph.com/issues/77120
Signed-off-by: Anoop C S <anoopcs@cryptolab.net>
2026-06-10 12:16:47 +05:30
Matthew N. Heler
d2e487acfe rgw/restore: take the hash mod HASH_PRIME when picking a shard
choose_oid fed ceph_str_hash_linux straight into % max_objs, and the
low bits of that hash are weak enough that similar object names keep
landing on the same shard. LC takes the hash mod HASH_PRIME first for
exactly this reason, do the same here.

Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
2026-06-09 18:34:47 -05:00
Eric Zhang
7f9e383ecb qa: Add overlapped roots test case
Added overlapped roots test case for CRUSH rules
rule zone-1-only
  step take zone us-east-1
  step chooseleaf firstn 3 type host
  step emit

rule zone-2-only
  step take zone us-east-2
  step chooseleaf firstn 3 type host
  step emit

rule zone-3-only
  step take zone us-east-3
  step chooseleaf firstn 3 type host
  step emit

rule default
   step take default
  step chooseleaf firstn 3 type host
  step emit

and osd tree
    -1         6.00000  root default
     -8         6.00000      region us-east
     -5         2.00000          zone us-east-1
    -13         1.00000              host host0
      0    ssd  1.00000                  osd.0
    -14         1.00000              host host1
      1    ssd  1.00000                  osd.1
     -6         2.00000          zone us-east-2
    -15         1.00000              host host2
      2    ssd  1.00000                  osd.2
    -16         1.00000              host host3
      3    ssd  1.00000                  osd.3
     -7         2.00000          zone us-east-3
    -17         1.00000              host host4
      4    ssd  1.00000                  osd.4
    -18         1.00000              host host5
      5    ssd  1.00000                  osd.5

    -1: pg_target = 300
    -5: pg_target = 100
    -6  pg_target = 100
    -7: pg_target = 100

Fixes: https://tracker.ceph.com/issues/73892

Signed-off-by: Eric Zhang <emzhang@ibm.com>
2026-06-09 13:16:33 -07:00
Eric Zhang
6831e7c94a docs: Add docs about allowing PG autoscaling for overlapped roots
Remove mention of overlapped roots failing to scale and added
example of adding overlapped roots.
Added mention of changes in PendingReleaseNotes

Fixes: https://tracker.ceph.com/issues/73892

Signed-off-by: Eric Zhang <emzhang@ibm.com>
2026-06-09 13:16:33 -07:00
Eric Zhang
47c8ab5cac mgr: Modify test_overlapping_roots for PG budget
No longer need to detect which roots are overlapping. Instead test PG budget allocation when roots overlap

Fixes: https://tracker.ceph.com/issues/73892

Signed-off-by: Eric Zhang <emzhang@ibm.com>
2026-06-09 13:16:33 -07:00
Eric Zhang
43dfdc8fdb mgr: Allow autoscaling for overlapped roots
Remove references to overlapped roots. Assign PG budget per root as sum(mon_target_pg_per_osd/# roots that contain OSD)

Fixes: https://tracker.ceph.com/issues/73892

Signed-off-by: Eric Zhang <emzhang@ibm.com>
2026-06-09 13:16:33 -07:00
Shilpa Jagannath
04aceefad0 common/async: async_cond::notify/cancel must post to handler's associated executor
RGWDeleteMultiObj spawns child coroutines via spawn_throttle to delete
objects in parallel. each child coroutine carries the connection strand
as its associated executor, serializing concurrent operations on shared
state like the response formatter and ops_log_entries.

but in multisite env, concurrent deletions in the same bucket shard
contend on async_cond in RGWDataChangesLog::add_entry(). when notify()
fires, waiting coroutines resume on the raw io_context executor
instead of their connection strand, breaking the serialization that prevents data
races in send_partial_response()

any_completion_handler doesn't support post(). so post() to the
default executor and dispatch to the associated executor from there

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2026-06-09 15:13:04 -04:00
Patrick Donnelly
1bb67ee394
librados: print nspace only if present
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-06-09 11:40:51 -04:00
Emmanuel Ameh
24f96c09f4 doc: Remove empty README.md and relocate nvmeof HA design doc
doc/README.md was a 0-byte file with no content; remove it.

doc/nvmeof/ha.md described the internal NVMeofGwMon/NVMeofGwMap
HA implementation (Paxos messaging, ANA state machines, blocklist
logic) -- a developer design document, not user-facing content.
User-facing NVMe-oF documentation already exists in doc/rbd/.
Move to doc/dev/nvmeof-ha-design.md where developer design docs live.

Fixes: https://tracker.ceph.com/issues/77208

Signed-off-by: Emmanuel Ameh <eameh@contractor.linuxfoundation.org>
2026-06-09 13:17:55 +01:00
Emmanuel Ameh
83541fd93c doc/security: Add Security Lead and Working Group pages to toctree
Both securitylead.rst and workinggroup.rst existed in the tree but
were not referenced in the security/index.rst toctree, making them
invisible in the built documentation. Add RST document titles and
include both pages in the toctree.

Fixes: https://tracker.ceph.com/issues/77206

Signed-off-by: Emmanuel Ameh <eameh@contractor.linuxfoundation.org>
2026-06-09 13:15:29 +01:00
Shai Fultheim
8f3a166569 crimson/os/seastore: wake blocked IO on BackgroundProcess wakeup
BackgroundProcess::run() blocks on blocking_background when
background_should_run() returns false. When the loop wakes up —
typically because arm_blocking_io_and_wake() kicked it — the loop
re-checks background_should_run(). If that still returns false
(e.g. the cleaner has no pending work), the loop goes straight back
to sleep.

But the wake may also have been triggered by a change in the space-
availability condition (should_block_io) — and there is no path that
re-evaluates blocked user IO when background_should_run is false. The
blocked IO has no future trigger to re-check, and stays stuck until
the next unrelated wake-up happens to come at the right moment, or
never.

Add a maybe_wake_blocked_io() call right after the wake. It's a no-op
when nothing is blocked, and unblocks IO that the space condition has
already cleared otherwise.

Observed in sustained random-write benches that wedged with IO blocked
indefinitely while the cleaner sat idle (should_run=false).

Signed-off-by: Shai Fultheim <shai.fultheim@gmail.com>
2026-06-09 14:13:49 +03:00
Sagar Gopale
6dadd2f030 mgr/dashboard: Remove global RGW tenant Roles tab and decommission routes
Fixes: https://tracker.ceph.com/issues/77262

Signed-off-by: Sagar Gopale <sagar.gopale@ibm.com>
2026-06-09 16:29:37 +05:30
Kautilya Tripathi
91878740bb crimson/cbt: use yaml.safe_load in t2c and add unit tests
t2c.py translates teuthology benchmark YAML into CBT configuration for
run-cbt.sh. Switch input parsing from yaml.load() to yaml.safe_load()
so the module works with current PyYAML and avoids unsafe deserialization.

Add test_t2c.py covering get_cbt_tasks(), Translator.translate(), and
main(), and register it with make check via add_ceph_test. Document the
translator in doc/dev/crimson/index.rst and describe the
ceph-perf-pull-requests Jenkins workflow in
doc/dev/continuous-integration.rst.

Signed-off-by: Kautilya Tripathi <kautilya.tripathi@ibm.com>
2026-06-09 16:00:01 +05:30
Kefu Chai
1879f2470b crimson/osd: init PerShardState::startup_time per-shard
previously, we got the mono_clock::now() in OSD::start() and passed it
to PerShardState. this worked fine. but it was a little bit convoluted
-- we pass the startup_time all the way to PerShardState.

in this change, we just use call mono_clock::now() in the contructor
of PerShardState. simpler this way.

the startup_time has two consumers:

- the PGs hosted by the sharded_service use it as a reference for the
  monotonic timestamp
- Heartbeat::send_heartbeats() uses it as for the mono_ping_stamp.

because, strictly speaking, we cannot gurantee that all PerShardState
sharded services share the identical startup timestamp, as they are
constructed on different shards. but this does not matter, as PGs
always use the hosting shard service for the referencing timestamp,
and OSD always uses the shard service on local shard for sending
heartbeats.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-09 15:30:29 +08:00
Kefu Chai
acbfd2de97 crimson/osd: remove OSD::startup_time
OSD::startup_time was added in 91c0df81, in which was added to provide
a monotomic increasing timestamp representing the startup time. but
later, we decided to keep track of this timestamp in PerShardState.
but we didn't remove OSD::startup_time when adding
PerShardState::get_mnow().

in this change, we remove the unused OSD::startup_time.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-09 15:30:29 +08:00
Kefu Chai
840ee2cfad crimson/osd: coroutinize OSD::start()
OSD::start() is a long, deeply nested .then() continuation chain. let's
rewrite it as a coroutine to make it readable. the chain was already
sequential and the one concurrent step keeps its when_all_succeed(), so
the rewrite preserves both ordering and concurrency.

start() runs once at boot, off the i/o path, so the small overhead of
co_await over a hand-rolled continuation chain is a fine price for the
readability.

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-09 15:30:29 +08:00
Aashish Sharma
a1906f997a mgr/cephadm: add unit tests
Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2026-06-09 09:55:38 +05:30
Aashish Sharma
32dbed0be1 mgr/cephadm: set default prometheus template in config-key store unless
overridden by the user

Fixes: https://tracker.ceph.com/issues/75826

Signed-off-by: Aashish Sharma <aasharma@redhat.com>
2026-06-09 09:51:28 +05:30
Brad Hubbard
d6f1bcc890 common: Log when we leak and disallow inlining
eaa2013 added a generic suppression that exposed the fact that the newer
compiler was inlining CephContext::do_command(). In order to avoid such
scenario in the future, and to keeo the behaviour constant between
daemons we'll disallow inlining.

Fixes: https://tracker.ceph.com/issues/76473

Signed-off-by: Brad Hubbard <bhubbard@redhat.com>
2026-06-09 08:04:08 +10:00
ramin.najarbashi
c147865379 test/rgw: init CephContext in unittest_rgw_cors
unittest_rgw_cors used GMock::Main without global_init, so dout() in
RGWCORSRule::matches() dereferenced a null g_ceph_context and segfaulted
on the first test. Link unit-main like unittest_rgw_compression.

Assisted-by: Cursor:composer-2.5-fast
Signed-off-by: ramin.najarbashi <ramin.najarbashi@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-08 23:43:23 +03:30
ramin.najarbashi
830f0e5628 rgw: include str_list.h for ceph::for_each_substr in CORS header
get_multi_cors_method_flags() calls ceph::for_each_substr(), but
rgw_cors.h did not include str_list.h. The CORS unit test includes
only this header, so the build failed with an undeclared identifier.
Qualify the call and add the missing include.

Signed-off-by: ramin.najarbashi <ramin.najarbashi@gmail.com>
2026-06-08 22:47:52 +03:30
Patrick Donnelly
4d0e27808f
.github/workflows/releng-audit: catch override removal error
This can happen if an action is retried or the label was removed through
other means (manually).

Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-06-08 14:10:47 -04:00
Matan Breizman
ca0bddb891 crimson/os/seastore: track onode lookup latency in do_transaction
we might be able to reduce onode lookup stage by storing the prev
lookup already made when getting the obc.

according to to the stats I've colleceted onode lookup takes 40%
of total build time (which is ~20% of total txn time)

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-06-08 15:03:14 +00:00
Alex Ainscow
c7bbe58ae7 test/osd: Improve test logging with OSD identification
Modified the logging system to show which OSD is executing in test logs,
making it easier to debug multi-OSD test scenarios.

Changes:
- Added Log::set_prefix_hook() to allow tests to customize log prefixes
- Modified Log::_flush() to use prefix_hook when set, replacing thread IDs
- Added EventLoop::get_log_prefix() to return 'osd.X' or 'harness' based on context
- Updated EventLoop to track current_executing_osd during event execution
- Simplified PGBackendTestFixture to use NoDoutPrefix directly
- Removed redundant 'shard X:' prefix from ShardDpp (info already in PG identifier)
- Removed verbose console output from EventLoop (now redundant with OSD prefixes)
- Converted EventLoop message scheduling to use dout at level 20

Benefits:
- Test logs now show 'osd.X' instead of thread IDs during event execution
- Shows 'harness' for test setup/teardown code outside events
- Message scheduling visible in logs at debug level 20
- Makes multi-OSD test scenarios much easier to follow and debug
- No changes to production logging code

AI assisted code, with careful human review and checking.

AI-assisted-by: IBM Bob IDE:Claude Sonnet (Anthropic)
Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>
2026-06-08 13:06:01 +01:00
Jon Bailey
b05cfd1612 test: Remove invalid unit test
This test was talking about testing invalid ops, however with the inclusion of sync reads in EC (https://github.com/ceph/ceph/pull/67079), it is valid to perform class reads in EC. In addition, work was done around illegal ops here: https://github.com/ceph/ceph/pull/66258 and the existance of TEST(ClsHello, BadMethods) in test_cls_hello.cc covers illegal ops in that PR leading me to think this is unneccisairy. Because of these reasons, I think its better this test is removed as it is incorrect and also not working.

Signed-off-by: Jon Bailey <jonathan.bailey1@ibm.com>
2026-06-08 12:41:46 +01:00
gmallet
63b5a5aebd libcephfs: add public api for ceph_ll_flock
Signed-off-by: gmallet <malletgaetan@protonmail.com>
2026-06-08 08:00:31 +02:00
Matan Breizman
6f7f9779e7 crimson/os/seastore: break down submit phases
Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-06-07 12:22:02 +00:00
Matan Breizman
89e913d279 seastore: add seastore_do_transaction_stage_lat metric
Introduce a new latency breakdown metric for do_transaction, exposing
per-stage timing (build, collock_wait, submit, throttler_wait).

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-06-07 09:24:51 +00:00
Matan Breizman
8447dfd23c crimson/os/seastore: track per-transaction conflict/replay counts
Add a num_replays counter to Transaction, incremented in
Cache::mark_transaction_conflicted whenever a transaction is marked
conflicted.

Only relevant for user-MUTATE path (do_transaction_no_callbacks).

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-06-07 08:28:09 +00:00
ramin.najarbashi
8b3f907f7c rgw: address CORS rule-matching review feedback
- matches_method: reject null/empty method; short-circuit RGW_CORS_ALL
- matches_preflight_headers: add logging for skip and disallowed headers
- generate_cors_headers: capture req_meth/req_hdrs in lambda
- remove unused validate_cors_rule_method/header helpers

Signed-off-by: ramin.najarbashi <ramin.najarbashi@gmail.com>

On branch fix/rgw-cors-aws-rule-matching
Changes to be committed:
	modified:   src/rgw/rgw_cors.cc
	modified:   src/rgw/rgw_op.cc
2026-06-06 17:01:56 +03:30
ramin.najarbashi
b5624b63f2 fix: some issue based on CORS spec 6.2.4
Signed-off-by: ramin.najarbashi <ramin.najarbashi@gmail.com>
2026-06-06 16:46:53 +03:30
ramin.najarbashi
bbec695790 reject null or empty method in matches_method
Signed-off-by: ramin.najarbashi <ramin.najarbashi@gmail.com>
2026-06-06 16:35:06 +03:30
ramin.najarbashi
cc1dcecee0 rgw: match CORS rules like Amazon S3
Select the first CORSRule whose AllowedOrigin, AllowedMethod, and (for
preflight) AllowedHeader all match the request. Previously host_name_rule
matched origin only, so a second rule with the same origin but different
methods was never used.

Add RGWCORSRule::matches() and RGWCORSConfiguration::match_rule(), wire
generate_cors_headers and RGWOptionsCORS to them (including global CORS), and
add unittest_rgw_cors.

Signed-off-by: ramin.najarbashi <ramin.najarbashi@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-06 16:35:06 +03:30
Kobi Ginon
c8ffac91f0 cephadm: set Grafana http_addr to 0.0.0.0 when unset
Grafana 11.1+ rejects non-literal http_addr values (e.g. localhost)
in grafana-apiserver. Use 0.0.0.0 by default; stop bracket-wrapping
IPv6 addresses in http_addr.
Fixes: https://tracker.ceph.com/issues/75365
Signed-off-by: Kobi Ginon <kginon@redhat.com>
2026-06-05 19:32:56 +03:00
Redouane Kachach
d486328b71
qa/cephadm: fix test_repos.sh for jammy nodes
The test uses add-repo --release 17.2.6 to verify version-string repo
handling, but debian-17.2.6 only has focal and bullseye suites and
jammy packages weren't built until 17.2.7. This causes apt-get update
to fail with a 404 on ubuntu_22.04 nodes.

see: https://download.ceph.com/debian-17.2.6/dists/

This fix bumps the version to 17.2.7 which includes a jammy suite.

Fixes: https://tracker.ceph.com/issues/77130

Signed-off-by: Redouane Kachach <rkachach@ibm.com>
2026-06-05 10:07:39 +02:00
Vallari Agrawal
7114fd10f6
monitoring: fix NVMeoFMultipleNamespacesOfRBDImage
Do not trigger alert NVMeoFMultipleNamespacesOfRBDImage
for same pool/image name used in multiple nvmeof namespaces,
if they are in different rados namespaces (rados_namespace_name)

These are valid repeation of pool/image name:
- mypool/rados_ns1/myimage1
- mypool/rados_ns2/myimage1
- mypool/myimage1 (default rados namespace)

Fixes: https://tracker.ceph.com/issues/77128

Signed-off-by: Vallari Agrawal <vallari.agrawal@ibm.com>
2026-06-05 10:48:08 +05:30
Oguzhan Ozmen
7d62871ba4 rgw/rest: add TODO for concurrent endpoint DNS resolution
Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-04 19:32:19 +00:00
Oguzhan Ozmen
28f2a5e22a rgw/multisite: fix endpoint unreachable detection in RGWRESTConn sync paths
The checks that decide whether to call set_endpoint_unconnectable()
were comparing against -EIO, but the actual error codes returned on
connection failure changed after commit 37352a9074 ("rgw: change
rgw_http_error_to_errno default to -ERR_INTERNAL_ERROR").

- complete_request() -> wait() returns req_data->ret which is set to
  rgw_http_error_to_errno(0) = -ERR_INTERNAL_ERROR when http_status is 0
  (TCP connect failed). Fix the six call sites to check -ERR_INTERNAL_ERROR.

- forward_request() returns tl::unexpected(-ERR_SERVICE_UNAVAILABLE)
  when http_status == 0 (no HTTP response received at all). Fix the two
  forward/forward_iam conditionals to check -ERR_SERVICE_UNAVAILABLE.

Without this fix, connection failures are never detected in the sync
paths, so set_endpoint_unconnectable() is never called and the
IP failover / retry logic is effectively dead.

The .h coroutine paths were already fixed by dbb409e21b ("rgw: fix
endpoint detection in RGWRESTConn") but that commit missed all .cc
sync paths.

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-04 19:32:19 +00:00
Oguzhan Ozmen
3513cd888a rgw: store RGWEndpoint URL as boost::urls::url
Replace raw string manipulation in RGWEndpoint with boost::urls::url.
URL path/query/host changes now use set_path(), set_query(), set_host()
instead of string concatenation.

Rename original_url to endpoint_url_lookup_id to clarify its role as a
health-tracking key for ResolvedEndpoint lookup in
set_endpoint_unconnectable().

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-04 19:32:19 +00:00
Shilpa Jagannath
78b6005865 rgw: fix log_remove spurious -EIO on non-existent FIFO shards
the shared error code `ec` was not cleared before continuing
past a non-existent shard (ENOENT from get_meta). and if the last
shard returned ENOENT, `ec` retained that value and the
fired EIO even though all shards were cleanly removed.

also switch to sys::system_error(ec) to preserve the real errno

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2026-06-04 13:52:57 -04:00
Matt Benjamin
47c6618d81 rgwlc: faithfully store lifecycle filters
Fix the bug where an empty lifecycle filter was stored
identically to one with an empty old-format prefix rule--
which caused the recovered rules to be formed incorrectly
as prefix rules.

Found by Kyle Bader and Tekton workflow.  Updated to not declare
virtual methods in LCFilter.

Fixes: https://tracker.ceph.com/issues/76806

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
2026-06-04 10:31:34 -04:00
Matan Breizman
18562abade crimson/os/seastore: switch op_lat sample_sum from ms to us
Sub-ms ops rounded to zero under the old millisecond cast, making
the average meaningless for fast NVMe workloads. sample_sum is now
in microseconds; divide by 1000 for ms average.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-06-04 09:39:11 +00:00
Matan Breizman
e94e76f058 crimson/os/seastore: add op_lat histogram buckets
Initialize lat_hist_bounds_us bucket boundaries (0.25ms–20ms in 10
log-ish steps) for the op_lat Seastar histogram in register_metrics(),
and populate them in add_latency_sample(). Previously op_lat only
tracked sample_count and sample_sum, giving an average but no
percentile visibility.

Signed-off-by: Matan Breizman <mbreizma@redhat.com>
2026-06-04 09:39:06 +00:00
Dhairya Parmar
881ed78b51 PendingReleaseNotes: add a note about deprecation warning
Fixes: https://tracker.ceph.com/issues/61482
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2026-06-04 13:57:44 +05:30
Kefu Chai
05a28d1b63 ceph-dencoder: skip dlclose under ASan so leaks symbolise
The DencoderPlugin destructor calls dlclose() on each loaded plugin .so
before main() returns.  When LeakSanitizer's atexit-registered leak
check then runs, the plugin .so is no longer mapped in the process
address space, and any leak whose allocation stack passes through
plugin code is reported with "<unknown module>" — useless for
investigation.

This is not an ASan bug or a missing -g flag; it is the unavoidable
ordering of dlclose-before-leak-check.  Skipping dlclose under
__SANITIZE_ADDRESS__ leaves the plugins mapped until process exit,
restoring symbolisation.  The functional cost is zero: the kernel will
unmap the .so segments when the process exits anyway.

Without this, a developer triaging dencoder leaks under ASan has no
visibility into which plugin or which static initialiser is
responsible.  With this, the leak stacks resolve to real symbols and
real bugs become triagable.

This commit does not change leak-detection results in non-sanitiser
builds and is a no-op on FreeBSD (which already skipped dlclose).

Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-06-04 14:02:05 +08:00
Matthew N. Heler
9c07120d10 rgw/lc: retry cloud-tier 500 InternalError for bucket create and object PUT
The cloud-tier transfer only treated 503 (-EBUSY) as retryable. A 500 from the
remote surfaced as -ERR_INTERNAL_ERROR and failed the object outright, but S3
backends use 500 as a transient "try again" signal. Retry it the same way we
retry -EBUSY. The helper is no longer EBUSY-specific, so rename it to
retry_on_transient_error; the restore path uses it too and gets the same
behaviour.

Even then a 500 never reached the retry: cloud_tier_create_bucket mapped an
error response to -EIO and an error with no body to 0 (success). Return the
real errno for transient failures, and stop treating an empty-body error as a
successful create.

Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
2026-06-03 16:44:35 -05:00
Alex Ainscow
f178960906 osd: get_primary_shard should not call pgtemp_undo_primaryfirst
In this variant of get_primary_shard, the function attempts to undo
the primaryfirst ordering.  This is incorrect, because pg_to_acting_osds
has already applied this.

The fix is simply to remove this code.

This is unlikely to be a significant problem, as this function is not
used in the critical locations.  We are building new test harnesses
which will rely on this behaviour.

Signed-off-by: Alex Ainscow <aainscow@uk.ibm.com>
2026-06-03 17:05:26 +01:00
Ashwin M. Joshi
7b37c21cfa python-common: Move validation function to utils and remove unused
Fixes: https://tracker.ceph.com/issues/74986
Signed-off-by: Ashwin M. Joshi <ashjosh1@in.ibm.com>

 Conflicts:
	src/python-common/ceph/deployment/service_spec.py
	src/python-common/ceph/deployment/utils.py
2026-06-03 21:19:18 +05:30
Oguzhan Ozmen
9eda20715c doc/radosgw: expose rgw_rest_conn_connect_to_resolved_ips and rgw_rest_conn_ip_fail_timeout_secs
Expose the confval information for the confval "rgw_rest_conn_connect_to_resolved_ips"
and "rgw_rest_conn_ip_fail_timeout_secs" so that they can be seen in the
"Ceph Object Gateway Config Reference" as these are meant to be client-facing configs.

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-03 11:04:29 +00:00
Oguzhan Ozmen
9f0c1093a0 rgw: rename round-robin counters for brevity
Rename endpoint_round_robin_counter to endpoint_rr_index and
endpoint_ips_round_robin_counter to ip_rr_index for shorter,
cleaner variable names while maintaining clarity.

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-03 11:04:29 +00:00
Oguzhan Ozmen
11aa2c86cd rgw/zone: increase visibility into zone connections via admin socket
Adds a new admin socket command to dump zone connection details
including endpoints, resolved IPs, and health status. Useful for
debugging multisite connectivity issues.

Usage: ceph daemon <radosgw.asok> zone connections

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-03 11:04:29 +00:00
Oguzhan Ozmen
2d8061cf75 rgw: make CONN_STATUS_EXPIRE_SECS a cfg option
Introduce a new radosgw option 'rgw_rest_conn_ip_fail_timeout_secs' to
be able to set the constant CONN_STATUS_EXPIRE_SECS dynamically.

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-02 22:16:20 +00:00
Oguzhan Ozmen
2d7bc7818c rgw/rest: track connection failures per-IP instead of per-endpoint
Previously, when a connection to a zone endpoint failed, the entire
endpoint was marked as unavailable for a timeout period. Since we now
resolve endpoints to all their IP addresses (via DNS A/AAAA records),
we can be more granular: track failures at the individual IP level.

Introduce ResolvedIP struct that pairs each IP's connect_to string
with its own failure timestamp. When selecting an IP for a request,
round-robin skips IPs that have recently failed, allowing traffic to
continue flowing to healthy nodes even when some are down.

An endpoint-level last_failure_time is maintained as a fast-path
optimization to avoid scanning all IPs when none have failed recently.

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-02 22:16:20 +00:00
Oguzhan Ozmen
afab77cae3 rgw/rest: remove unused headers
Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-02 22:16:20 +00:00
Oguzhan Ozmen
3e310e0c85 rgw/rest: consolidate endpoint_urls and resolved_endpoints into single vector
Previously RGWRESTConn stored endpoints in two data structures:
- endpoint_urls: vector<string> for ordered round-robin iteration
- resolved_endpoints: unordered_map<string, ResolvedEndpoint> for lookup

This was redundant since the URL was stored in both places.

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-02 22:16:20 +00:00
Oguzhan Ozmen
71af1fc945 rgw: add operator<< for RGWEndpoint and simplify logging
Add ostream operator<< to RGWEndpoint struct for convenient logging of
endpoint details (url, original_url when different, and connect_to).
Update log statements across rgw_http_client.cc, rgw_rest_client.cc,
and rgw_rest_conn.cc to use the new operator for cleaner, more
consistent output.

Add unittest_rgw_http_client to test RGWEndpoint functionality.

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-02 22:16:20 +00:00
Oguzhan Ozmen
b8d8c40edc rgw: track original URL within RGWEndpoint instead of separate member (refactor)
Refactor endpoint tracking by adding original_url to RGWEndpoint struct
instead of maintaining a separate endpoint_orig member in RGWHTTPClient.
This simplifies the code by having each endpoint self-track its original
URL, which is needed for connection status lookups after URL modifications.

No functional changes intended.

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-02 22:16:20 +00:00
Oguzhan Ozmen
e468f0f65f rgw: fix incomplete RGWRESTConn move constructor/assignment
Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-02 22:16:20 +00:00
Oguzhan Ozmen
030e62b8f1 rgw/rest: consolidate endpoint status tracking into ResolvedEndpoint
Refactor RGWRESTConn to eliminate the separate endpoints_status map by
moving the connection status (std::atomic<ceph::real_time>) directly
into the ResolvedEndpoint struct. This reduces redundancy and simplifies
endpoint state management.

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-02 22:16:20 +00:00
Oguzhan Ozmen
6aca73a5f5 rgw/http: apply RGWEndpoint connect_to via libcurl CURLOPT_CONNECT_TO
If endpoint.connect_to is non-empty, populate and attach a curl slist to
CURLOPT_CONNECT_TO for this request.

Ensure the slist is freed with the request lifetime to avoid leaks across
retries/requests.

Fixes: https://tracker.ceph.com/issues/74677
Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-02 22:16:20 +00:00
Oguzhan Ozmen
510e9ab1f0 rgw/rest: round-robin resolved endpoint IPs into curl CONNECT_TO mapping
Add logic to select an IP for a given endpoint URL (RR over resolved addresses)
and build a host:port:ip:port mapping.

Store the mapping in the RGWEndpoint so the HTTP layer can apply it per request.

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-02 22:16:20 +00:00
Oguzhan Ozmen
af7abeea00 rgw/rest: resolve multisite endpoints to all A/AAAA records (optional)
When rgw_resolve_endpoints_into_all_addresses=true, parse each configured
endpoint URL, extract host/port, and resolve the host into all IP addresses.

Store resolution results (including a round-robin index) alongside the original
endpoint URL for later selection.

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-02 22:16:15 +00:00
Oguzhan Ozmen
d989ef7b68 rgw: add rgw_resolve_endpoints_into_all_addresses config option
Introduce an advanced boolean config option (default: false) to enable resolving
multisite endpoint hostnames into all A/AAAA records.

When enabled, RGW can distribute outgoing inter-zone traffic across DNS-provided
backends even without an external load balancer.

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-02 22:01:23 +00:00
Oguzhan Ozmen
2690e2a2cb rgw/http: introduce RGWEndpoint to carry url + connect_to (refactor)
This commit is meant to be non-functional.

Replace the "URL as plain string" plumbing with an RGWEndpoint value type
that carries:

  - the URL string, and
  - optional per-request connect_to data for libcurl,
  - it also encapsulates related functions/methods

This commit is intended to be non-functional: behavior should remain unchanged
until callers populate connect_to.

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-06-02 22:01:16 +00:00
Kobi Ginon
b09f161916 cephadm: start rpcbind before NFS-Ganesha when NFSv3 is enabled
With Protocols including NFSv3, Ganesha 9.x on Rocky 10 needs a running
portmapper to register NFSv3 on TCP. Start rpcbind from the container
entrypoint only when ganesha.conf enables v3; skip for NFSv4-only configs.
Fixes: https://tracker.ceph.com/issues/76981
Signed-off-by: Kobi Ginon <kginon@redhat.com>
2026-06-02 21:37:07 +03:00
Redouane Kachach
6260b33f1d
mgr/cephadm: Don't skip OSDs with non-empty osdspec_affinity
`ceph cephadm osd activate` calls `deploy_osd_daemons_for_existing_osds`
with a synthetic DriveGroupSpec where service_id=''. Commit fbe3a053
introduced an unconditional osdspec_affinity filter:

    if osd['tags']['ceph.osdspec_affinity'] != spec.service_id:
        continue

The fix is to only enforce the affinity check when spec.service_id is
non-empty. An empty service_id means the caller is osd activate, which
should adopt any existing OSD regardless of its affinity tag.

Fixes: https://tracker.ceph.com/issues/76979
Signed-off-by: Redouane Kachach <rkachach@ibm.com>
2026-06-02 16:33:17 +02:00
Leonid Chernin
ca090f3a35 nvmeofgw: introduce nvme-gw show-all
Introduce a new read-only command, nvme-gw show-all, which
iterates over all configured pools and groups and runs
nvme-gw show for each, simplifying gateway inspection.

fixes https://tracker.ceph.com/issues/74854
Signed-off-by: Leonid Chernin <leonidc@il.ibm.com>
2026-06-01 12:02:41 +03:00
Ashwin M. Joshi
d69477cd02 python-common: Improve profile name string validation
Fixes: https://tracker.ceph.com/issues/74986
Signed-off-by: Ashwin M. Joshi <ashjosh1@in.ibm.com>

	src/python-common/ceph/tests/test_service_spec.py

 Conflicts:
	src/python-common/ceph/deployment/service_spec.py
2026-06-01 14:28:34 +05:30
Casey Bodley
84040d8829 qa/rgw/multisite: use random aes algorithm
test coverage for `rgw crypt sse algorithm: aes-256-gcm` was recently
added to the multisite suite, doubling the number of jobs from 2 to 4

instead of duplicating the multisite jobs for both cbc and gcm, select
a random algorithm for each of the 2 existing jobs

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2026-05-29 13:44:47 -04:00
Dhairya Parmar
303f617f1c mgr/nfs: warn when deprecated export/cluster delete commands are run
change the hander from EmptyResponder() to ErrorResponseHandler() to
emit the warning making use of the command's status channel.

Fixes: https://tracker.ceph.com/issues/61482
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2026-05-29 19:00:20 +05:30
senol.colak
ec59888070 mgr: preserve reported hostname when CRUSH host is unavailable
The previous patch only injected hostname into the metadata map for OSDs
that were placed under a CRUSH host bucket. For mds/mgr/mon daemons and
for OSDs absent from CRUSH, the original 'state->hostname = ...' line
was removed without a replacement, so update_metadata received a map
with no 'hostname' key. DaemonState::set_metadata only updates
state->hostname when the map carries it, so a daemon restarting on a
different host would be re-inserted in by_server under its stale
hostname.

Always seed m["hostname"] from the metadata-reported hostname, then
prefer the CRUSH host when available. Apply the same shape to the
new-daemon branch so set_metadata uniformly drives state->hostname,
removing the redundant direct assignment.

Reported-by: code review on PR #66757
Signed-off-by: senol.colak <senol.colak@sap.com>
2026-05-29 11:33:36 +02:00
Sun Yuechi
4e4dd6b1cd cephadm/tests: mock find_program in test_container_engine
test_container_engine mocks find_executable to keep command_check_host
off the real host, but command_check_host looks up 'lvcreate' (and
'systemctl') through find_program, which isn't mocked. So the test
actually needs lvcreate installed and only passes where it is.

Mock find_program too so the test doesn't depend on host binaries.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-05-29 16:59:38 +08:00
Sun Yuechi
5a83c60387 test/mds: don't drop await_idle_v under NDEBUG
await_idle_v() is wrapped in assert(), so with NDEBUG it is never
called and the test reads a default-constructed ack, racing the agent
thread. Use EXPECT_TRUE() like the rest of the file.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-05-29 16:24:39 +08:00
Sun Yuechi
dada074b34 monitoring/ceph-mixin: bump jsonnet-bundler to v0.6.0
v0.4.0 fails to build on riscv64; bump to v0.6.0 to fix it.

Signed-off-by: Sun Yuechi <sunyuechi@iscas.ac.cn>
2026-05-29 14:51:52 +08:00
Kefu Chai
43dd4cbd37 rocksdb: upgrade submodule to v7.10.2 to address CVE-2022-23476
This commit upgrades the rocksdb submodule from v7.9.2 to v7.10.2 to
address CVE-2022-23476, a security vulnerability in the Nokogiri gem
used by RocksDB's documentation build system.

CVE-2022-23476 Details:
- Affects: RocksDB versions < 7.10.2
- Issue: Unchecked return value from xmlTextReaderExpand in Nokogiri
- Fix commit: 6648dec0a3eee0a329af8342038ce099baa55122
- Fixed in: Nokogiri 1.13.10, included in RocksDB v7.10.2

The vulnerability was fixed in the upstream facebook/rocksdb repository
on December 8, 2022, and included in the v7.10.2 release. The v7.9.2
version currently used by Ceph does not contain this fix, as the 7.9.x
branch diverged before the security patch was applied.

Analysis confirmed that:
- Current Ceph RocksDB: v7.9.2 (commit 24ea35870fe9b3ba15285ec8746ba97ed5d67ff3)
- CVE fix present in v7.9.2: NO
- First version with fix: v7.10.2 (commit 9107c1059b4656513ae0d51e8976e4f69f59a9c3)

References:
- CVE-2022-23476: https://nvd.nist.gov/vuln/detail/CVE-2022-23476
- Nokogiri Advisory: GHSA-qv4q-mr5r-qprj
- Fix commit: facebook/rocksdb@6648dec

also, in this change, we adapt BinnedLRUCache to CacheItemHelper API.
rocksdb 7.10 reshaped `rocksdb::Cache`, so we have to update our
overrides accordingly to address the build failure. the check for
RocksDB version is dropped, as mainstream distros ship RocksDB >=
7.10.2 at the time of writing:

- alpine 3.23:    10.9.1
- fedora 44:      10.2.1
- debian stable:   9.10.0
- ubuntu nobel:    8.9.1
- ubuntu resolute: 9.11.2

Signed-off-by: Justin Caratzas <jcaratza@ibm.com>
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
2026-05-29 09:55:35 +08:00
Patrick Donnelly
4681cb7b34
qa/suites/upgrade: ignore fs down variant
Fixes: https://tracker.ceph.com/issues/76969
Signed-off-by: Patrick Donnelly <pdonnell@ibm.com>
2026-05-28 13:54:38 -04:00
Casey Bodley
72483c1eb0 rgw: use local error code in handle_individual_object()
op_ret is a member variable of RGWDeleteMultiObj shared between
concurrent callers of handle_individual_object(). because that coroutine
may suspend between writing to op_ret and later reading it back for
send_partial_response(), it's likely that other callers have modified it
in the meantime

store this error code in a local variable instead, so each coroutine has
its own consistent copy of it

Fixes: https://tracker.ceph.com/issues/76960

Signed-off-by: Casey Bodley <cbodley@redhat.com>
2026-05-28 09:39:27 -04:00
Max Kellermann
2718fd2792 include/CompatSet: remove the FeatureSet ctor
Signed-off-by: Max Kellermann <max.kellermann@ionos.com>
2026-05-28 10:22:18 +02:00
Max Kellermann
7b5e7c4260 include/CompatSet: get_name() returns std::string_view
Creating a temporary copy is useless overhead for all callers.

Signed-off-by: Max Kellermann <max.kellermann@ionos.com>
2026-05-28 08:22:35 +02:00
Max Kellermann
5a45cd213e include/CompatSet: use std::string_view in struct Feature
Almost all callers pass a C string (in temporary Feature instances),
only `CompatSetHandler::handle()` passes a `std::string` reference.

Allocating a new `std::string` is useless overhead.  This patch
reduces the binary size by 21 kB.

Signed-off-by: Max Kellermann <max.kellermann@ionos.com>
2026-05-28 08:22:35 +02:00
senol.colak
dcd9c5bac5 mgr: override OSD hostname with CRUSH physical host in all metadata paths
In containerized deployments (e.g. Rook), OSD daemons report the
pod/container hostname. The CRUSH map records the physical host.
This mismatch causes 'ceph device ls' to show pod names instead of
physical hosts, breaking host-based device correlation.

Fix by deriving the CRUSH host bucket name for each OSD and substituting
it as the canonical hostname in all three metadata paths:
- MetadataUpdate::finish (async mon metadata fetch, primary path)
- Mgr::load_all_metadata OSD bulk-load (startup/failover)
- DaemonServer::fetch_missing_metadata (via MetadataUpdate)

The override is a no-op when the OSD is absent from CRUSH or when the
OSD map is not yet available.

Fixes: https://tracker.ceph.com/issues/73080
Signed-off-by: senol.colak <senol.colak@sap.com>
2026-05-27 22:43:27 +02:00
Oguzhan Ozmen
c29b9f0c86 rgw/multisite: include error code in object sync failure log message
Append retcode and its string representation to the log line to facilitate the
diagnosis of sync issues.

Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-05-27 20:10:53 +00:00
Oguzhan Ozmen
21be389c67 rgw/multisite: do not log EBUSY/EAGAIN in per-object sync error log
The bucket-instance level sync (RGWDataSyncSingleEntryCR) excludes EBUSY and
EAGAIN from the sync error log as these are meant to be transient errors that
resolve on retry without an admin intervention.

Also, add EBUSY and EAGAIN to ignore_sync_error(), which is used by the
per-object level sync in RGWSyncObjectCR, to match the bucket-instance
level behavior.

Fixes: https://tracker.ceph.com/issues/76950
Signed-off-by: Oguzhan Ozmen <oozmen@bloomberg.net>
2026-05-27 20:10:24 +00:00
Max Kellermann
adfbcec205 osd/SnapMapper: use fmt::format() instead of fmt::sprintf()
fmt::sprintf() is a legacy API and cannot use custom formatters.

Signed-off-by: Max Kellermann <max.kellermann@ionos.com>
2026-05-26 13:12:43 +02:00
Max Kellermann
fdf82a5d8d common/options: convert long_desc to C string
All parameters are C strings (either C literals, except for
MgrMonitor::update_from_paxos()) and there's no point in copying all
of them onto the heap during startup.

Signed-off-by: Max Kellermann <max.kellermann@ionos.com>
2026-05-26 13:02:27 +02:00
Zac Dover
4f0bec39bf doc/mgr: document ceph-exporter
Add stub-level information for the documentation of ceph-exporter. Explicit configuration instructions are still missing from this documentation, and should in the future be supplied.

Signed-off-by: Zac Dover <zac.dover@clyso.com>
2026-05-26 20:45:15 +10:00
Harsimran Singh
3a9ab7a5f6 rgw: add HEAD Object op metrics to perf counters and Prometheus export
Expose HEAD Object operation metrics in the same format as existing RGW op metrics so operators can track HEAD request volume and latency at gateway, user, and bucket scope.

Previously, HEAD Object requests were silently counted under get_obj_ops because RGWGetObj handles both GET and HEAD. The HEAD-specific early-return path (get_data == false) had no counter instrumentation.

Fixes: ISCE-3042

Signed-off-by: Harsimran Singh <harsimransingh@ibm.com>
2026-05-26 11:41:06 +05:30
Xuehan Xu
981d678971 crimson/os/seastore/cached_extent: renew mutation_pending extents'
last_committed_crc when committing rewrite transactions

Signed-off-by: Xuehan Xu <xuxuehan@qianxin.com>
2026-05-26 13:42:22 +08:00
Matt Benjamin
b2f5d1876e rgwlc: fix a likely null dereference in LCObjsLister::get_obj()
Found by Opus 4.6:

  In get_obj() (lines 417-446), when obj_iter ==
  list_results.objs.end() and the listing is truncated, the code calls
  fetch() to get the next batch. fetch() sets obj_iter =
  list_results.objs.begin(). But if the new fetch returns zero objects
  (possible if objects were deleted between listing batches, or due to
  filtering/race conditions), then begin() == end() and the code falls
  through to line 437:

  if (obj_iter->key.name == pre_obj.key.name) {

  This dereferences end() — undefined behavior / crash. The
  return-value check at line 445 (return obj_iter !=
  list_results.objs.end()) comes too late; the dereference already
  happened.

Fixes: https://tracker.ceph.com/issues/76790

Signed-off-by: Matt Benjamin <mbenjamin@redhat.com>
Assisted-by: Claude Code, Opus 4.6 1M
2026-05-25 12:28:22 -04:00
Leonid Chernin
8e32fff173 nvmeofgw:fix forcing unavailable gw exit by sending
empty map to it

Signed-off-by: Leonid Chernin <leonidc@il.ibm.com>
2026-05-25 09:48:09 +03:00
Neeraj Pratap Singh
b066d4cd3a qa: fixing failures in TestShellOpts
We were seeing failures in TestShellOpts again
and again due the (newline added within the output)
output structure change with the cmd2 releases.
Now, we directly extract the final output of 'set editor'
command and use it, instead of traversing on complete output
through hardcoded indexes.This fix is better than earlier
hardcoded splitting/striping via indexes.

Fixes: https://tracker.ceph.com/issues/71795
Signed-off-by: Neeraj Pratap Singh <Neeraj.Pratap.Singh1@ibm.com>
2026-05-20 22:53:09 +05:30
Ville Ojamo
4fe47d6723 doc/man: use Monitor consistently in ceph.rst
Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>
2026-05-20 14:05:17 +07:00
Jaya Prakash
ac6dd5571e qa: fix TEST_mon_features persistent feature checks in mon/misc.sh
The test reused the "tentacle" jq filter while validating
"nvmeof_beacon_diff", causing the comparison to fail. Also fix
the umbrella feature validation and update the expected
persistent feature count.

Fixes: https://tracker.ceph.com/issues/76472

Signed-off-by: Jaya Prakash <jayaprakash@ibm.com>
2026-05-19 17:02:32 +00:00
Nathan Hoad
a9ba39df2a common: Fix the default for rgw_gc_max_queue_size, per review.
This should be 1KB short of the maximum, not 3MB over.

Signed-off-by: Nathan Hoad <nhoad@bloomberg.net>
2026-05-18 15:23:53 -04:00
Matthew N. Heler
579eb83143 rgw/cloud-tier: tighten -EBUSY handling
Push the -EBUSY early-return into init_multipart and the bucket
exists/create path so a 503 stops printing a bogus ERROR before the
outer retry catches it. Wrap abort_multipart in retry_on_busy too,
since a stuck 503 there leaves an orphan multipart on the remote.
retry_on_busy also logs when it gives up so a silent exhaustion
doesn't get lost.

Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
2026-05-18 13:17:24 -05:00
Kobi Ginon
43a62944af cephadm: list-networks opt-in loopback and BGP route supplements
Fixes: https://tracker.ceph.com/issues/62195
Extend cephadm list-networks (and the cephadm mgr module options that pass
flags through to it) so hosts can expose loopback and BGP-derived routes
when operators need them, while keeping the default conservative for
orchestrator network discovery.

- Add cephadm flags --allow-lo-routes and --allow-bgp-routes (after the
  list-networks subcommand). Wire mgr options allow_lo_routes and
  allow_bgp_routes to the remote cephadm invocation.

- IPv4: build from ``ip route ls`` (scope link + src). When allow_lo_routes
  is true, merge extra lines from the same output via _parse_ipv4_lo_route
  (lo /32 without src, host-scoped routes including dummy-style /32 from
  76229). Always omit prefixes wholly inside 127.0.0.0/8. When
  allow_lo_routes is false, skip any route whose device is lo at parse time.
  When allow_bgp_routes is true, merge ``ip route ls proto bgp`` (ECMP
  nexthops and lo BGP /32), again respecting allow_lo_routes for lo.

- IPv6: build from ``ip -6 route ls`` and ``ip -6 addr ls``; omit ::1/128.
  When allow_lo_routes is false, skip routes on lo (including global /128 on
  lo). When allow_bgp_routes is true, merge ``ip -6 route ls proto bgp`` in
  the same way as IPv4, including lo BGP /128 when allow_lo_routes is true.

Signed-off-by: Kobi Ginon <kginon@redhat.com>
2026-05-18 14:30:08 +03:00
Matthew N. Heler
5c51fef311 rgw/cloud-tier: switch backoff jitter to ceph::util RNG
include/random.h already provides a thread-local engine; use
it for backoff jitter

Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
2026-05-15 19:19:11 -05:00
Adam C. Emerson
066b33e805
build: Add cls_timeindex_types to cls_timeindex objclass
`cls_timeindex_entry` defines `encode()` and `decode()` inline in
`cls_timeindex_types.h`, but `dump()` and `generate_test_instances()` are
defined out of line in `cls_timeindex_types.cc`. That `.cc` was compiled
only into `cls_timeindex_client`, not into the `cls_timeindex objclass`
shared library. At `-O2` the optimizer inlines and dead-code-eliminates
these unused `dump()` paths, so `cls_timeindex.so` ends up with no
reference to the out-of-line symbols and loads fine. At `-O0` nothing is
inlined or eliminated: the header-inline `dump()` is emitted and leaves
an undefined reference to `cls_timeindex_entry::dump()` in
`cls_timeindex.so`, so `dlopen()` of the objclass fails at load time.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2026-05-15 17:54:46 -04:00
Matthew N. Heler
e6de8833b3 rgw/cloud-tier: retry on 503 from remote endpoint
We were aborting the lifecycle transition or restore on the first 503
SlowDown back from the remote. Wrap both paths in a small retry helper
that backs off and tries again on -EBUSY, using the caller's yield
context to sleep.

Tunable via rgw_cloud_tier_retry_limit (8), rgw_cloud_tier_retry_delay_ms
(500), and rgw_cloud_tier_retry_max_ms (30000). On the streamed restore
GET, RGWRadosPutObj::handle_headers rejects 5xx so the response body
doesn't get streamed into the put-processor before we know to retry.

This also affects the multi-zone fetch path that uses the same
put-processor; 503 there now returns -EBUSY cleanly instead of
streaming the error body.

Fixes: https://tracker.ceph.com/issues/76406
Co-authored-by: Kevin Lott <klott@backblaze.com>
Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
2026-05-14 19:08:33 -05:00
Adam C. Emerson
e28e7544b7
rgw/multi: Reliably recover from future generation sync
Since we can't map shards between generations, if we get a sync entry
for a future shard, synthesize an entry on the current generation for
all shards.

We use the bucket cache to keep track of when we've done this so we
don't give ourselves unbounded work when we get bursts of future
generation shards.

Fixes: https://tracker.ceph.com/issues/75786
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2026-05-14 19:51:48 -04:00
Adam C. Emerson
411ab6e865
rgw/multi: Add GenCache to ShardCache
The GenCache will keep track of per-bucket sync information, like
if we have had to synthesize a change-per-shard as a result of hitting
the wrong generation.

Templatize tests.

Fixes: https://tracker.ceph.com/issues/75786
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
2026-05-14 19:26:05 -04:00
David Galloway
6481785756 doc: sign-debs script for release process
Fixes: tracker.ceph.com/issues/63336

Signed-off-by: David Galloway <david.galloway@ibm.com>
2026-05-14 11:58:16 -04:00
Shweta Bhosale
59b1191ed7 python-common: Adding precheck for placement.count_per_host in NFSServiceSpec
NFS service specs should not allow placement.count_per_host.

Fixes: https://tracker.ceph.com/issues/76579
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-05-13 19:51:01 +05:30
Jacques Heunis
f8d3db622a rgw: Fix ops logs sometimes having several entries per line.
Although not explicitly documented, the RGW ops log is generally
formatted with one entry per line. This makes it work well with log
shipping/ingestion services (many of which default to treating each line
as a separate entry) and in particular works well with log servers that
index based on the available JSON fields.

The current implementation separates the log call from the resulting
disk IO: Logs are written to a buffer and a separate thread flushes them
to disk (possibly in batches). The current code appends a newline only
at the end of the batch being flushed to disk and in many cases when
under load this means that several log entries are concatenated onto a
single line, which complicates attempts to process those logs.

This PR separates the addition of a new line from the flush to disk,
appending a newline after every log entry but still only flushing at the
end of the batch to avoid additional IO overhead.

Fixes: https://tracker.ceph.com/issues/76566
Signed-off-by: Jacques Heunis <jheunis@bloomberg.net>
2026-05-13 09:31:03 +00:00
dheart
e334498be3 tool/ceph-kvstore-tool: add --pretty-binary-key option
Signed-off-by: dheart <dheart_joe@163.com>
2026-05-12 23:25:10 +08:00
Shubha Jain
b8e3155ef0
mgr/nfs: improve cluster info implementation and fix lint regressions
- Improve cluster info implementation
- Fix deployment type logic
- Resolve lint issues for check jobs

Signed-off-by: Shubha jain <SHUBHA.JAIN1@ibm.com>
2026-05-11 18:42:24 +05:30
Max Kellermann
ec8fad9a78 nvmeof: add missing includes
Signed-off-by: Max Kellermann <max.kellermann@ionos.com>
2026-05-07 11:51:10 +02:00
Nathan Hoad
ceabc96f16 doc: Update release notes for removed options.
Signed-off-by: Nathan Hoad <nhoad@bloomberg.net>
2026-05-01 09:19:03 -04:00
Nathan Hoad
b40ee54935 rgw: Remove GC deferred entries options and code.
This code has been disabled since v16.2.8.

Signed-off-by: Nathan Hoad <nhoad@bloomberg.net>
2026-04-30 15:52:14 -04:00
Shubha Jain
4a64b83c6c
mgr/orchestrator: default NFS ingress to haproxy-protocol mode
Change default ingress mode from haproxy-standard to haproxy-protocol
to preserve client IP addresses for proper IP-level export restrictions
in NFS Ganesha.

Fixes: https://tracker.ceph.com/issues/74260

Signed-off-by: Shubha Jain <SHUBHA.JAIN1@ibm.com>
2026-04-30 23:31:25 +05:30
Shweta Bhosale
f4a70f197f mgr/cephadm: Fixed unittest for NFS protocol
Fixes: https://tracker.ceph.com/issues/74492
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-04-30 19:16:43 +05:30
Shweta Bhosale
64732c127d mgr/nfs: Updated enable_nfsv3 check while export creation
Fixes: https://tracker.ceph.com/issues/74492
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-04-30 19:11:22 +05:30
Shweta Bhosale
44c1362028 mgr/nfs: set nfs export protocols based on cluster's protocol settings
Fixes: https://tracker.ceph.com/issues/74492

Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-04-30 19:11:17 +05:30
Shweta Bhosale
9b0becfdf2 mgr/cephadm: Updated NFS default protocol to v4 and v3 is enabled only when enable_nfsv3 is set in the spec
Fixes: https://tracker.ceph.com/issues/74492
Signed-off-by: Shweta Bhosale <Shweta.Bhosale1@ibm.com>
2026-04-30 18:52:45 +05:30
Kamoltat (Junior) Sirivadhna
8899968104 src/test: update show-choose-tries.t tests
Since we modified crushtool cli commands, we need to also update
its test with new flag: --show-retry-exhaustion
and also the modified --show-choose-tries option

Also added /src/script/run-cli-tests.sh to run the cram test
easily without having the config headache

Signed-off-by: Kamoltat (Junior) Sirivadhna <ksirivad@redhat.com>
2026-04-29 18:38:06 +00:00
Shilpa Jagannath
2eb274eef2 rgw/archive: record the bucket creation time at archive zone
source bucket deletion triggers a new bucket instance creation
at archive zone. record its mtime and display in bucket stats.
store it as an attr RGW_ATTR_ARCHIVE_INSTANCE_MTIME

Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
2026-04-29 18:05:38 +00:00
Kamoltat (Junior) Sirivadhna
8e9e1fa46d doc: update crushtool.rst
Add --show-retry-exhaustion flag to doc

Signed-off-by: Kamoltat (Junior) Sirivadhna <ksirivad@redhat.com>
2026-04-29 14:16:22 +00:00
Kamoltat (Junior) Sirivadhna
a22a63030f src/script: init test_stretch crush_collisions.sh
Add script to test for CRUSH retry exhaustion in stretch mode with
2 datacenters. Tests unbiased stretch rules by running multiple
iterations of PG mappings and checking for collisions that exceed
the 50-try limit.

Also add --show-retry-exhaustion flag to crushtool to detect and
report when CRUSH mapping hits the maximum retry limit.

Signed-off-by: Kamoltat (Junior) Sirivadhna <ksirivad@redhat.com>
2026-04-28 20:13:10 +00:00
Redouane Kachach
8499124ada mgr/cephadm: plumb force_delete_data through daemon/service removal
This PR wires the `force_delete_data` already existing flag in the
binary through cephadm’s daemon and service removal paths, so that
commands such as `ceph orch rm service` or equivalent daemon removal
can explicitly ask for data deletion instead of the default "move
under <fsid>/removed/" for daemons such as Prometheus, osd and mon.

Fixes: https://tracker.ceph.com/issues/74058
Signed-off-by: Kobi Ginon <kginon@redhat.com>
2026-04-27 14:00:00 +03:00
Rishabh Dave
c75b8eefd7 qa/cephfs: minor fix in comment
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2026-04-23 23:04:28 +05:30
Rishabh Dave
241cfc6c06 qa/cephfs: give more time to tests in test_clone_stats.py
Currently tests wait for 10 seconds for clone progress bar to appear and
it usually takes 8 seconds to find it. In case of the host where test is
being run is slow, the progress bar can take more time to appear and the
test can fail unnecessarily. To avoid this, increase the waiting
duration.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
2026-04-23 23:04:28 +05:30
Rishabh Dave
269e76bb13 qa/cephfs: increase number of files to cloned in test_clone_stats.py
There's a race condition between cloning operation and clone progress
reporter thread to open the subvolume, former does so to copy/clone
files and latter to get details of the source and destination subvolume.

A low number of files in the snapshot gives clone progress reporter
thread less or no chance to fetch and process the details needed to
display the progress bar. This causes tests to fails when progress bar
isn't displayed in the output of "ceph status" output.

Increasing the number of files will give clone progress reporter more
time to display the progress bar and therefore these unnecessary test
failures wlll reduce/stop.

This failure wasn't seen earlier because Python code was being used to
copy files which was comparatively slow (compared to C++ code that was
merged to copy files as a part of fscrypt work) which allowed enough
time to clone progress reporter thread to process stats and display
clone progress bar.

Fixes: https://tracker.ceph.com/issues/74082
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2026-04-23 23:04:28 +05:30
Rishabh Dave
2599fc10c5 volumes/stats_util: improve log messages
Signed-off-by: Rishabh Dave <ridave@redhat.com>
2026-04-23 23:04:07 +05:30
Joshua Blanch
bd412d1f17 os/bluestore: restore previous offline expansion logic
expanding allocator before opening for offline path
fixes to comments and null check

Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
439e1b04df os/bluestore: fix offline device expansion in expand_devices
guard bluefs->expand_device for online-only
fixes offline expansion by reopening db from r/o to r/w
fixes int64_t/uint64_t type mismatch for old_size

Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
b8b8fa9707 os/bluestore: dout free space added starts at aligned_old_size
Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
4f93487e2d os/bluestore: open bluestore in read only mode during device expansion
Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
60197ef72c os/bluestore: fix nullptr derefence for bdev in single-device setup
Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
8d1dadb90c os/bluestore: adds assert for device mounted before expand
Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
5962c7e8e9 os/bluestore: round old size by min_alloc
Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
17f2979774 os/bluestore: rounds size0 up such that expand is [p2roundup(size0), aligned_size)
Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
12eaa37f7b os/bluestore: remove unused if branch
Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
2186c8bec2 os/bluestore: skip bitmapfm exapnd if old and new size are the same
Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
0eb1b730d1 qa/standalone: bdev-expand calls with no underlying device expansion
Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
36d005e02d os/bluestore: make Allocator::device_size atomic and protect BitmapAllocator::expand
Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
ba5fb6bef2 qa/standalone/bluefs: add online device expansion test
verify online device expansion via admin socket works correctly

Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
2ab1311f38 os/bluestore: refactors expand_devices for online and offline use
Refactors device expansion to use a expand_devices()
that handles both online (admin socket) and offline (ceph-bluestore-tool)
use cases.

Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
2fc6f81941 os/bluestore: update bluefs volume selector
Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
a201ee80fb blk/BlockDevice: changes block device size as atomic
Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
57e3d3bb34 os/bluestore: add asok for online device expansion
expands bluestore and bluefs while OSD is running.
Instead of recreating allocators, expands allocator and FM
in-place.

ceph tell osd.X bluestore bluefs-bdev-expand

Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
d7bd19a508 bluestore/bluefs: add expand_device for online device usage
adds support for expanding the device capacity while fs is online

Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
530faae4a5 os/bluestore: expose expand() method for FreelistManager
Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
bd6c72e01d os/bluestore: implement expand() for bitmap allocator hierarchy
Implements device expansion support for bitmap-based allocators
BitmapAllocator and HybridAllocator. Bitmap allocators uses allocated
fixed-size bit vectors (L0, L1, L2) and requires vector resizing
and metadata updates when underlying device expands.

Some different cases that are handled:
- padding reclamation when alignment boundaries change
- Sub-granularity expansions (L2 doesn't grow, only L2 bits update)

To be used such as expand() followed by init_add_free() to ensure
available space accounting happens once.

Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
8703fec36f blk: update to check new block device size
Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:55 +00:00
Joshua Blanch
1af9896da8 os/bluestore: admin socket for bdev expand
Signed-off-by: Joshua Blanch <joshua.blanch@clyso.com>
2026-04-23 15:05:51 +00:00
Christopher Hoffman
c6c208e67d qa: Add direct_io test for fscrypt
Fixes: https://tracker.ceph.com/issues/73347
Signed-off-by: Christopher Hoffman <choffman@redhat.com>
2026-04-20 19:14:56 +00:00
stzuraski898
f410c49e60 mgr: Properly set description in labeled get_perf_schema_python
Fixes: https://tracker.ceph.com/issues/76048
Signed-off-by: stzuraski898 <steven.zuraski@ibm.com>
2026-04-16 22:00:45 +00:00
Vivek Maurya
0709f29a27 osd: Rephrase unclear OSD_FLAGS warning message
Setting maintenance mode show below warning,which appears incorrect
 or ambiguous.
 [WRN] OSD_FLAGS: 1 OSDs or CRUSH {nodes, device-classes}have
 {NOUP,NODOWN,NOIN,NOOUT} flags set

 Change warning as shown bellow.
 [WRN] OSD_FLAGS: 1 OSDs or CRUSH nodes/device-classes have
 one or more of these flags set: NOUP, NODOWN, NOIN, NOOUT

 Fixes: https://tracker.ceph.com/issues/71716

 Signed-off-by: Vivek Maurya <vivek.maurya@ibm.com>
2026-04-13 04:42:04 -05:00
Ville Ojamo
77b3dcb38c doc/radosgw: change all intra-docs links to use ref (3 of 6)
Part 3 of 6 to make backporting easier. Depends on part 1.

Use the the ref role for all remaining links in doc/radosgw/ with the
exception of config-ref.rst which will depend on changes to rgw.yaml.in.

The "external link definitions" syntax being removed is intended for
linking to external websites and not for intra-docs links. Validity of
ref links will be checked during the docs build process.

Add labels for links targets if necessary.
Remove unused external link definitions in the modified files.

Use confval instead of literal text for 2 configuration keys in
vault.rst.

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>
2026-03-17 11:18:28 +07:00
Indira Sawant
91c8003f47 mon/MgrMonitor: include standby manager details in ceph mgr stat
Extend `ceph mgr stat` to include basic information about standby
manager daemons in a new `standbys` field. This improves visibility
into standby managers without requiring `ceph mgr dump`.

Fixes: https://tracker.ceph.com/issues/75464
Signed-off-by: Indira Sawant <indira.sawant@ibm.com>
2026-03-16 14:48:29 -05:00
Dhairya Parmar
4e7c5b11bb doc/cephfs: document resizing subvolumes using units
Fixes: https://tracker.ceph.com/issues/62673
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2026-02-19 10:16:02 +05:30
Dhairya Parmar
839355f700 qa: test resizing subvolumes with various valid and invalid sizes
Fixes: https://tracker.ceph.com/issues/62673
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2026-02-19 10:16:02 +05:30
Dhairya Parmar
15e5a9dc0b mgr/volumes: simplify comaparison
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2026-02-19 10:16:02 +05:30
Dhairya Parmar
efe11565b8 mgr/volumes: accept unit with subvolume resize
Specifying the quota or resize for a subvolume requires the value in bytes.
This value can now be accepted as <num><unit>. Also allows floating point numbers.

Fixes: https://tracker.ceph.com/issues/62673
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
2026-02-19 10:16:02 +05:30
Ville Ojamo
85d8bcd75a doc/radosgw: change all intra-docs links to use ref (5 of 6)
Part 5 of 6 to make backporting easier. Depends on part 1.

Use the the ref role for all remaining links in doc/radosgw/ with the
exception of config-ref.rst which will depend on changes to rgw.yaml.in.

The "external link definitions" syntax being removed is intended for
linking to external websites and not for intra-docs links. Validity of
ref links will be checked during the docs build process.

Add labels for links targets if necessary.
Remove unused external link definitions in the modified files.

Use confval instead of literal text for 2 configuration keys in
vault.rst.

Signed-off-by: Ville Ojamo <git2233+ceph@ojamo.eu>
2026-02-13 12:44:48 +07:00
1957 changed files with 161025 additions and 43665 deletions

View File

@ -1,6 +1,16 @@
---
name: "Check for Incompatible Licenses"
on: [pull_request]
on:
pull_request_target:
branches:
- main
- umbrella
- tentacle
- squid
permissions:
contents: read
pull-requests: read
jobs:
pull_request:

View File

@ -38,7 +38,7 @@ jobs:
# Backport checks need to be run ONLY on the main branch on ceph/ceph (not forks)
if: github.ref == 'refs/heads/main' && github.repository == 'ceph/ceph'
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
sparse-checkout: |
src/script/backport-create-issue

View File

@ -25,7 +25,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout main branch for pull_request_target
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: 'refs/heads/main'
path: ceph

View File

@ -1,15 +1,22 @@
---
name: "Pull Request Checklist"
on:
pull_request:
pull_request_target:
branches:
- main
types:
- edited
- opened
- reopened
# https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#defining-access-for-the-github_token-scopes
permissions:
pull-requests: read
jobs:
pr_checklist:
runs-on: ubuntu-latest
name: Verify
name: Verify Checklist
steps:
- name: Sleep for 30 seconds
run: sleep 30s

View File

@ -1,11 +1,15 @@
---
name: "Check for missing .qa links"
on:
pull_request:
pull_request_target:
branches:
- main
- umbrella
- tentacle
- squid
types:
- opened
- synchronize
- edited
- reopened
permissions:
contents: read
@ -15,18 +19,21 @@ jobs:
runs-on: ubuntu-latest
if: github.repository == 'ceph/ceph'
steps:
- name: Checkout main branch for pull_request_target
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Checkout verify-qa script from main branch
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: 'refs/heads/main'
path: main
sparse-checkout: |
src/script/verify-qa
sparse-checkout-cone-mode: false
- name: Checkout PR HEAD
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: ${{ github.event.pull_request.head.sha }}
path: pull_request
- name: verify .qa links
run: ../main/src/script/verify-qa
working-directory: pull_request/
- name: Run verification script
run: |
./main/src/script/verify-qa ./pull_request

View File

@ -41,7 +41,7 @@ jobs:
steps:
- name: Checkout main branch for pull_request_target
if: github.event_name == 'pull_request_target'
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
ref: 'refs/heads/main'
path: 'ceph'
@ -49,7 +49,7 @@ jobs:
- name: Checkout default branch for other events
if: github.event_name != 'pull_request_target'
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
path: 'ceph'
fetch-depth: 0

View File

@ -17,15 +17,16 @@ concurrency:
jobs:
audit:
name: Backport Audit
name: Execution and Routing
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
issues: write
statuses: write
steps:
- id: router
name: Evaluate Workflow Routing & Overrides
name: Evaluate Workflow Routing
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
env:
ORG_TOKEN: ${{ secrets.ORG_READ_PAT }}
@ -35,39 +36,192 @@ jobs:
const payload = context.payload;
const actor = context.actor;
const isBot = actor === 'github-actions[bot]' || actor === 'github-actions';
const runUrl = `https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`;
// Safely removes a label and logs execution, ignoring 404s if label is not attached
async function removeLabelSafely(labelName) {
try {
core.info(`[Router] Attempting to remove label '${labelName}'...`);
await github.rest.issues.removeLabel({
owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number, name: labelName
});
core.info(`[Router] Successfully removed label '${labelName}'.`);
} catch (e) {
if (e.status === 404) {
core.info(`[Router] Label '${labelName}' was not present on PR (404 ignored).`);
} else {
core.warning(`[Router] Failed to remove label '${labelName}': ${e.message}`);
}
}
}
// Verifies if the user has maintainer/admin collaborator permissions or is an active
// member of the ceph-release-manager team. Required for /audit override and test-branch.
async function checkAuthorization(username) {
core.info(`[Router] Checking collaborator permission level for @${username}...`);
let authorized = false;
try {
const { data: permData } = await github.rest.repos.getCollaboratorPermissionLevel({
owner: context.repo.owner, repo: context.repo.repo, username: username
});
authorized = (permData.permission === 'admin' || permData.permission === 'maintain')
core.info(`[Router] Collaborator permission level for @${username}: ${permData.permission}`);
authorized = (permData.permission === 'admin' || permData.permission === 'maintain');
} catch (e) {
core.info(`[Router] Failed to fetch repo permissions: ${e.message}`);
}
if (!authorized && context.repo.owner === 'ceph' && process.env.ORG_TOKEN) {
core.info(`[Router] @${username} does not have maintain/admin repo rights. Checking ceph-release-manager team membership...`);
try {
const orgOctokit = getOctokit(process.env.ORG_TOKEN);
const orgOctokit = github.getOctokit(process.env.ORG_TOKEN);
const { data: teamData } = await orgOctokit.rest.teams.getMembershipForUserInOrg({
org: 'ceph', team_slug: 'ceph-release-manager', username: username
});
core.info(`[Router] Org team membership state for @${username}: ${teamData.state}`);
authorized = (teamData.state === 'active');
} catch (e) {
core.info(`[Router] Failed to fetch org team membership: ${e.message}`);
}
}
core.info(`[Router] Authorization result for @${username}: ${authorized ? 'AUTHORIZED' : 'NOT AUTHORIZED'}`);
return authorized;
}
// Retrieves the HEAD SHA of the pull request. Falls back to an API lookup
// when triggered by issue_comment events where the PR object is omitted from payload.
async function getPrSha() {
let sha = context.payload.pull_request?.head?.sha;
if (!sha) {
core.info(`[Router] PR SHA not found in payload. Fetching from API for issue #${context.issue.number}...`);
try {
const { data: pr } = await github.rest.pulls.get({
owner: context.repo.owner,
repo: context.repo.repo,
pull_number: context.issue.number
});
sha = pr.head.sha;
core.info(`[Router] Successfully retrieved SHA from API: ${sha}`);
} catch (e) {
core.error(`[Router] Failed to fetch PR details from API: ${e.message}`);
}
} else {
core.info(`[Router] Retrieved PR SHA from payload: ${sha}`);
}
if (sha) core.setOutput('pr_sha', sha);
return sha;
}
// Pushes a pending commit status to the PR HEAD SHA to provide visual feedback
// and block branch protection while the audit executes. For standard audit runs,
// strips pass/fail state labels before starting.
async function triggerAuditRun(description, statusContext = 'Backport Audit') {
core.info(`[Router] Initiating audit run trigger for context '${statusContext}': ${description}`);
const isStandard = (statusContext === 'Backport Audit');
if (isStandard) {
core.info('[Router] Clearing any previous labels before starting standard audit run...');
await removeLabelSafely('releng-audit-fail');
await removeLabelSafely('releng-audit-override');
await removeLabelSafely('releng-audit-pass');
await removeLabelSafely('releng-audit-queue');
}
const sha = await getPrSha();
if (!sha) {
core.error('[Router] Cannot initiate audit: PR SHA could not be retrieved.');
core.setOutput('run_audit', 'false');
return;
}
core.setOutput('status_context', statusContext);
if (isStandard) {
try {
core.info(`[Router] Setting commit status to 'pending' on SHA ${sha}...`);
await github.rest.repos.createCommitStatus({
owner: context.repo.owner,
repo: context.repo.repo,
sha: sha,
state: 'pending',
context: statusContext,
description: description,
target_url: `https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`
});
core.info(`[Router] Set pending commit status (${statusContext}) on SHA: ${sha}`);
} catch (e) {
core.error(`[Router] Failed to set commit status: ${e.message}`);
}
} else {
core.info(`[Router] Non-standard audit run ('${statusContext}') — skipping pending commit status creation.`);
}
core.setOutput('run_audit', 'true');
}
// Pushes an immediate success commit status when an authorized override occurs,
// unblocking required branch protection rules without executing the audit script.
async function setOverrideStatus(actorName) {
core.info(`[Router] Setting override success commit status for @${actorName}...`);
try {
const sha = await getPrSha();
if (sha) {
await github.rest.repos.createCommitStatus({
owner: context.repo.owner,
repo: context.repo.repo,
sha: sha,
state: 'success',
context: 'Backport Audit',
description: `Audit requirement overridden by @${actorName}`,
target_url: `https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`
});
core.info(`[Router] Set success (override) commit status on SHA: ${sha}`);
} else {
core.error('[Router] Cannot set override commit status: SHA is undefined or could not be retrieved.');
}
} catch (e) {
core.error(`[Router] Failed to set override commit status: ${e.message}`);
}
}
// Performs the complete override state transition atomically: applies the override label,
// strips pass/fail labels, updates commit status to success, and posts a confirmation comment.
async function executeOverride(actorName, addLabel = true, postComment = true) {
core.info(`[Router] Executing override lifecycle for @${actorName} (addLabel=${addLabel}, postComment=${postComment})...`);
if (addLabel) {
try {
core.info('[Router] Applying releng-audit-override label...');
await github.rest.issues.addLabels({
owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number, labels: ['releng-audit-override']
});
core.info('[Router] Successfully added releng-audit-override label.');
} catch (e) {
core.error(`[Router] Failed to add override label: ${e.message}`);
}
}
core.info('[Router] Stripping pass/fail labels as part of override execution...');
await removeLabelSafely('releng-audit-fail');
await removeLabelSafely('releng-audit-pass');
await setOverrideStatus(actorName);
if (postComment) {
try {
core.info('[Router] Posting override confirmation comment...');
await github.rest.issues.createComment({
owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number,
body: `✅ **Audit Override Applied** by @${actorName}.\n\n[View workflow run](${runUrl})`
});
core.info('[Router] Successfully posted override confirmation comment.');
} catch (e) {
core.error(`[Router] Failed to post override comment: ${e.message}`);
}
}
}
core.info(`[Router] Evaluating event: ${eventName}, action: ${payload.action || 'N/A'}`);
// ==========================================
// 1. HANDLE ISSUE COMMENTS (/audit commands)
// 1. HANDLE ISSUE COMMENTS (State Machine Drivers)
// ==========================================
// Issue comments (/audit commands) act strictly as state machine drivers.
// For standard commands (/audit retest, /audit override), we only apply labels
// here. This routes execution cleanly through the 'labeled' event handler,
// ensuring a single source of truth for audit triggers and override state changes.
if (eventName === 'issue_comment') {
core.info('[Router] Processing issue_comment event.');
if (!payload.issue.pull_request) {
core.info('[Router] Comment is not on a pull request. Skipping.');
core.setOutput('run_audit', 'false');
@ -77,180 +231,211 @@ jobs:
const commentBody = payload.comment.body.trim();
if (commentBody.startsWith('/audit retest')) {
core.info('[Router] Detected /audit retest command. Triggering audit.');
core.setOutput('run_audit', 'true');
return;
}
if (commentBody.startsWith('/audit override')) {
core.info(`[Router] Validating if user @${actor} is authorized to apply override.`);
const isAuthorized = await checkAuthorization(actor);
if (isAuthorized) {
core.info(`[Router] User @${actor} is authorized. Applying override and stripping fail label.`);
await github.rest.issues.addLabels({
owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number, labels: ['releng-audit-override']
});
try {
await github.rest.issues.removeLabel({
owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number, name: 'releng-audit-fail'
});
} catch (e) {}
// Directly trigger the audit run instead of bouncing through an ephemeral label
core.info('[Router] /audit retest detected. Triggering immediate audit execution...');
try {
await github.rest.issues.createComment({
owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number,
body: `✅ **Audit Override Applied** by @${actor}.`
body: `🔄 **Audit Retest Triggered** by @${actor}.\n\n[View workflow run](${runUrl})`
});
} catch (e) {
core.error(`[Router] Failed to post retest comment: ${e.message}`);
}
await triggerAuditRun('Running backport audit...');
} else if (commentBody.startsWith('/audit override')) {
// Execute the full override lifecycle atomically within this run
core.info(`[Router] /audit override detected. Validating @${actor}...`);
if (await checkAuthorization(actor)) {
core.info(`[Router] @${actor} is authorized. Executing override...`);
await executeOverride(actor, true, true);
} else {
core.info(`[Router] User @${actor} NOT authorized. Removing override label.`);
core.setFailed(`User @${actor} is not authorized to override audits.`);
core.error(`User @${actor} is not authorized to override audits.`);
}
core.setOutput('run_audit', 'false');
return;
} else if (commentBody.startsWith('/audit test-branch')) {
// Test branch mode: Execute PTL tool from an alternative branch for testing/debugging.
// To prevent side effects on PR mergeability, this mode does NOT touch state labels
// and will report a success commit status even if issues are found (findings are posted as PR comments).
core.info(`[Router] /audit test-branch detected. Validating @${actor}...`);
if (await checkAuthorization(actor)) {
const parts = commentBody.split(/\s+/);
const testBranch = parts[2] || 'testing/releng-audit';
try {
core.info(`[Router] Validating test branch '${testBranch}' exists...`);
await github.rest.repos.getBranch({ owner: context.repo.owner, repo: context.repo.repo, branch: testBranch });
core.info(`[Router] Test branch '${testBranch}' verified.`);
} catch (e) {
core.error(`Test branch '${testBranch}' does not exist: ${e.message}`);
await github.rest.issues.createComment({
owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number,
body: `❌ **Audit Test Mode Failed**\n\nBranch \`${testBranch}\` was not found in \`${context.repo.owner}/${context.repo.repo}\`.\n\n[View workflow run](${runUrl})`
});
core.setOutput('run_audit', 'false');
return;
}
try {
core.info('[Router] Posting test mode activation comment...');
await github.rest.issues.createComment({
owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number,
body: `🧪 **Audit Test Mode Activated** by @${actor}.\n\nExecuting audit using tooling checked out from branch \`${testBranch}\`. *This run will not affect required PR commit statuses or state labels.*\n\n[View workflow run](${runUrl})`
});
core.info('[Router] Successfully posted test mode activation comment.');
} catch (e) {
core.error(`[Router] Failed to post test mode comment: ${e.message}`);
}
core.setOutput('checkout_ref', testBranch);
await triggerAuditRun(`Running audit using branch ${testBranch}...`, `Audit Test Mode (${testBranch})`);
} else {
core.error(`User @${actor} is not authorized to invoke test branches.`);
core.setOutput('run_audit', 'false');
}
} else {
core.info('[Router] Comment is not a recognized /audit command. Skipping.');
core.setOutput('run_audit', 'false');
}
core.info('[Router] Comment is not an audit command. Skipping.');
core.setOutput('run_audit', 'false');
return;
}
// ==========================================
// 2. HANDLE PR EVENTS
// 2. HANDLE PR EVENTS & LABELS
// ==========================================
const hasFailLabel = payload.pull_request?.labels.some(l => l.name === 'releng-audit-fail');
const hasPassLabel = payload.pull_request?.labels.some(l => l.name === 'releng-audit-pass');
const hasOverrideLabel = payload.pull_request?.labels.some(l => l.name === 'releng-audit-override');
const hasQueueLabel = payload.pull_request?.labels.some(l => l.name === 'releng-audit-queue');
core.info(`[Router] Current labels - Fail: ${hasFailLabel}, Pass: ${hasPassLabel}, Override: ${hasOverrideLabel}`);
core.info(`[Router] Current state labels -> Fail: ${hasFailLabel}, Pass: ${hasPassLabel}, Override: ${hasOverrideLabel}, Queue: ${hasQueueLabel}`);
core.setOutput('has_override', hasOverrideLabel ? 'true' : 'false');
// --- SYNCHRONIZE (New Commits) ---
if (eventName === 'pull_request_target' && payload.action === 'synchronize') {
core.info('[Router] Processing synchronize event (new commits).');
if (hasOverrideLabel) {
core.info('[Router] PR had override label. Removing it because of new commits.');
try {
await github.rest.issues.removeLabel({ owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number, name: 'releng-audit-override' });
await github.rest.issues.createComment({
owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number,
body: '⚠️ **Audit Override Removed**\n\nNew commits were pushed to this PR, so the previous `releng-audit-override` has been removed.'
});
} catch (e) {
core.info(`[Router] Failed to remove override label: ${e.message}`);
}
// --- LABELED EVENTS ---
if (eventName === 'pull_request_target' && payload.action === 'labeled') {
const labelName = payload.label.name;
core.info(`[Router] Labeled event triggered for label '${labelName}' by @${actor}.`);
// The releng-audit-queue label acts as the primary trigger for manual retests
if (labelName === 'releng-audit-queue') {
core.info('[Router] Queue label detected. Stripping label and triggering audit...');
await removeLabelSafely('releng-audit-queue');
await triggerAuditRun('Running backport audit...');
return;
}
if (hasFailLabel) {
core.info('[Router] PR is currently in a failed audit state. Halting automated checks until failure label is removed.');
core.setFailed("PR is currently in a failed audit state. Remove the releng-audit-fail label to re-run.");
// Handle authorized overrides applied via label or /audit override comment
if (labelName === 'releng-audit-override') {
core.info(`[Router] Evaluating override label application by @${actor}...`);
if (!isBot && !(await checkAuthorization(actor))) {
core.error(`[Router] User @${actor} is not authorized to apply releng-audit-override. Removing label...`);
await removeLabelSafely(labelName);
core.error(`User @${actor} is not authorized to override audits.`);
} else {
core.info(`[Router] Override authorized for @${actor}. Executing override state transition...`);
// Execute override state transition without re-adding label (addLabel = false).
// Only post a confirmation comment if applied by a human user in the UI (!isBot).
await executeOverride(actor, false, !isBot);
}
core.setOutput('run_audit', 'false');
return;
}
if (hasPassLabel) {
core.info('[Router] Removing pass label so PR reflects pending state.');
try {
await github.rest.issues.removeLabel({ owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number, name: 'releng-audit-pass' });
} catch (e) {
core.info(`[Router] Failed to remove pass label: ${e.message}`);
}
}
core.info('[Router] Triggering audit for new commits.');
core.setOutput('run_audit', 'true');
return;
}
// --- UNLABELED ---
if (eventName === 'pull_request_target' && payload.action === 'unlabeled') {
const removedLabel = payload.label.name;
core.info(`[Router] Processing unlabeled event for label: ${removedLabel}`);
if (['releng-audit-fail', 'releng-audit-pass', 'releng-audit-override'].includes(removedLabel)) {
if (isBot) {
core.info(`[Router] Label ${removedLabel} removed by bot. Skipping audit trigger.`);
core.setOutput('run_audit', 'false');
return;
}
if (hasOverrideLabel && removedLabel === 'releng-audit-fail') {
core.info(`[Router] PR has releng-audit-override but removed releng-audit-fail. Skipping audit.`);
core.setOutput('run_audit', 'false');
return;
}
core.info(`[Router] User @${context.actor} removed ${removedLabel}. Stripping other state labels and triggering fresh audit.`);
try { await github.rest.issues.removeLabel({ owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number, name: 'releng-audit-fail' }); } catch (e) {}
try { await github.rest.issues.removeLabel({ owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number, name: 'releng-audit-pass' }); } catch (e) {}
core.setOutput('run_audit', 'true');
return;
}
}
// --- LABELED ---
if (eventName === 'pull_request_target' && payload.action === 'labeled') {
const labelName = payload.label.name;
core.info(`[Router] Processing labeled event for label: ${labelName} by actor: ${actor}`);
// Block humans from applying machine labels
// Prevent unauthorized users from manually spoofing machine-owned state labels
if (!isBot && (labelName === 'releng-audit-pass' || labelName === 'releng-audit-fail')) {
core.warning(`[Router] User @${actor} cannot manually apply ${labelName}.`);
try { await github.rest.issues.removeLabel({ owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number, name: 'releng-audit-fail' }); } catch (e) {}
try { await github.rest.issues.removeLabel({ owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number, name: 'releng-audit-pass' }); } catch (e) {}
if (hasOverrideLabel) {
core.info(`[Router] override label applied, skipping audit.`);
core.setOutput('run_audit', 'false');
core.warning(`[Router] User @${actor} cannot manually apply machine-owned label '${labelName}'. Removing...`);
await removeLabelSafely(labelName);
if (!hasOverrideLabel) {
core.info('[Router] No override active. Triggering fresh audit run...');
await triggerAuditRun('Running backport audit...');
} else {
core.info(`[Router] forcing audit.`);
core.setOutput('run_audit', 'true');
core.info('[Router] Override is active. Skipping fresh audit run.');
}
return;
}
// Enforce permissions for manual override label application
if (labelName === 'releng-audit-override') {
if (!isBot) {
core.info(`[Router] Validating if user @${actor} is authorized to apply override.`);
const isAuthorized = await checkAuthorization(actor);
if (!isAuthorized) {
core.info(`[Router] User @${actor} NOT authorized. Removing override label.`);
await github.rest.issues.removeLabel({ owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number, name: labelName });
core.setFailed(`User @${actor} is not authorized to override audits.`);
} else {
core.info(`[Router] User @${actor} is authorized. Stripping fail/pass labels to visually unblock PR.`);
try { await github.rest.issues.removeLabel({ owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number, name: 'releng-audit-fail' }); } catch (e) {}
try { await github.rest.issues.removeLabel({ owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number, name: 'releng-audit-pass' }); } catch (e) {}
}
} else {
core.info(`[Router] Bot applied ${labelName}. Permitted.`);
}
}
core.info(`[Router] Labeled event handled without triggering audit.`);
core.info(`[Router] Labeled event for '${labelName}' requires no action. Skipping.`);
core.setOutput('run_audit', 'false');
return;
}
// --- OPENED / REOPENED ---
if (eventName === 'pull_request_target' && (payload.action === 'opened' || payload.action === 'reopened')) {
core.info(`[Router] PR ${payload.action}. Triggering audit.`);
if (payload.action === 'reopened' && hasOverrideLabel) {
core.info('[Router] PR had override label. Removing it on reopen.');
// --- SYNCHRONIZE (New Commits) ---
// When new commits are pushed, existing overrides are revoked and audits re-evaluate.
// If the PR is already failed, execution halts to avoid comment spam until re-requested.
if (eventName === 'pull_request_target' && payload.action === 'synchronize') {
core.info('[Router] Processing synchronize event (new commits pushed)...');
if (hasOverrideLabel) {
core.info('[Router] PR had active override label. Revoking override due to new commits...');
await removeLabelSafely('releng-audit-override');
core.setOutput('has_override', 'false');
try {
await github.rest.issues.removeLabel({ owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number, name: 'releng-audit-override' });
core.info('[Router] Posting override removal notification comment...');
await github.rest.issues.createComment({
owner: context.repo.owner, repo: context.repo.repo, issue_number: context.issue.number,
body: '⚠️ **Audit Override Removed**\n\nThis PR was reopened, so the previous `releng-audit-override` has been removed.'
body: `⚠️ **Audit Override Removed**\n\nNew commits were pushed to this PR, so the previous override has been removed.\n\n[View workflow run](${runUrl})`
});
core.info('[Router] Override removal notification posted.');
} catch (e) {
core.info(`[Router] Failed to remove override label: ${e.message}`);
core.error(`[Router] Failed to post override removal comment: ${e.message}`);
}
}
core.setOutput('run_audit', 'true');
if (hasFailLabel) {
core.warning("[Router] PR is currently in a failed audit state. Halting automated execution on synchronize.");
core.info("PR is currently in a failed audit state. Remove the releng-audit-fail label or comment /audit retest to re-run.");
const sha = await getPrSha();
if (sha) {
core.info(`[Router] Setting commit status to 'failure' on new SHA ${sha}...`);
await github.rest.repos.createCommitStatus({
owner: context.repo.owner,
repo: context.repo.repo,
sha: sha,
state: 'failure',
context: 'Backport Audit',
description: 'PR is in a failed audit state. Remove releng-audit-fail label or comment /audit retest to re-run.',
target_url: runUrl
});
}
core.setOutput('run_audit', 'false');
return;
}
await triggerAuditRun('Running backport audit...');
return;
}
core.info('[Router] Event did not match any trigger criteria. Skipping audit.');
// --- UNLABELED ---
// If a user manually strips a status label, trigger a fresh audit evaluation
if (eventName === 'pull_request_target' && payload.action === 'unlabeled') {
const removed = payload.label.name;
core.info(`[Router] Processing unlabeled event: label '${removed}' was removed by @${actor}.`);
if (['releng-audit-fail', 'releng-audit-pass', 'releng-audit-override'].includes(removed) && !isBot) {
if (hasOverrideLabel && removed === 'releng-audit-fail') {
core.info('[Router] PR has active override label; ignoring manual removal of releng-audit-fail.');
core.setOutput('run_audit', 'false');
return;
}
core.info(`[Router] Status label '${removed}' removed by human. Triggering fresh audit evaluation...`);
await triggerAuditRun('Running backport audit...');
return;
} else {
core.info(`[Router] Unlabeled event for '${removed}' requires no action.`);
}
}
// --- OPENED / REOPENED ---
if (eventName === 'pull_request_target' && (payload.action === 'opened' || payload.action === 'reopened')) {
core.info(`[Router] Processing PR ${payload.action} event...`);
if (payload.action === 'reopened' && hasOverrideLabel) {
core.info('[Router] PR reopened with previous override label. Removing stale override...');
await removeLabelSafely('releng-audit-override');
core.setOutput('has_override', 'false');
}
await triggerAuditRun('Running backport audit...');
return;
}
core.info('[Router] Event did not match any active execution triggers. Skipping audit.');
core.setOutput('run_audit', 'false');
- name: Checkout Trusted Base Repository
@ -258,7 +443,15 @@ jobs:
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
fetch-depth: 0
ref: ${{ steps.router.outputs.checkout_ref || '' }}
- name: Fetch main for parity check
if: steps.router.outputs.run_audit == 'true'
# checkout only fetches the PR's base branch (e.g. squid/tentacle/umbrella).
# The parity check needs the main ref to walk the commit history graph and find the upstream
# merge commit for each cherry-pick, so fetch it explicitly as origin/main.
run: git fetch --no-tags origin +refs/heads/main:refs/remotes/origin/main
- name: Setup Python
if: steps.router.outputs.run_audit == 'true'
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
@ -269,23 +462,81 @@ jobs:
if: steps.router.outputs.run_audit == 'true'
run: pip install GitPython python-redmine requests
- name: Run PTL Audit
- id: ptl_audit
name: Run PTL Audit
if: steps.router.outputs.run_audit == 'true'
env:
PTL_TOOL_GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PTL_TOOL_REDMINE_API_KEY: ${{ secrets.REDMINE_API_KEY_BACKPORT_BOT }}
PTL_TOOL_BASE_PROJECT: ${{ github.repository_owner }}
PTL_TOOL_BASE_REPO: ${{ github.event.repository.name }}
PTL_TOOL_PR_NUMBER: ${{ github.event.pull_request.number || github.event.issue.number }}
run: |
PR_NUMBER="${{ github.event.pull_request.number || github.event.issue.number }}"
echo "Fetching latest labels for PR $PR_NUMBER to check for late-applied overrides..."
LABELS=$(curl -s -H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \
"https://api.github.com/repos/${{ github.repository }}/issues/$PR_NUMBER/labels" | jq -r '.[].name')
if echo "$LABELS" | grep -q "releng-audit-override"; then
echo "PR has releng-audit-override. Skipping audit execution to prevent reapplying releng-audit-fail."
exit 0
fi
python src/script/ptl-tool.py --debug --ci-mode --audit
python src/script/ptl-tool.py --debug --ci-mode --audit $PR_NUMBER
- name: Report Audit Status & Update State Labels
if: always() && steps.router.outputs.run_audit == 'true' && steps.router.outputs.pr_sha != ''
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const statusContext = '${{ steps.router.outputs.status_context }}' || 'Backport Audit';
if (statusContext !== 'Backport Audit') {
core.info(`[Reporter] Non-standard audit run ('${statusContext}') — skipping state labels and commit status reporting.`);
return;
}
const outcome = '${{ steps.ptl_audit.outcome }}';
const eventName = context.eventName;
core.info(`[Reporter] Evaluating final audit outcome: '${outcome}' for standard audit run (event: ${eventName})...`);
let state = 'failure';
let description = 'Backport audit failed. See review comments or workflow logs.';
if (outcome === 'success') {
state = 'success';
description = 'Backport audit completed successfully.';
core.info('[Reporter] Audit completed successfully. Applying releng-audit-pass label...');
try {
await github.rest.issues.addLabels({
owner: context.repo.owner, repo: context.repo.repo,
issue_number: context.issue.number || context.payload.pull_request.number,
labels: ['releng-audit-pass']
});
core.info('[Reporter] Successfully applied releng-audit-pass label.');
} catch (e) {
core.error(`[Reporter] Failed to add pass label: ${e.message}`);
}
} else if (outcome === 'cancelled') {
state = 'error';
description = 'Backport audit execution was cancelled.';
core.warning('[Reporter] Audit execution was cancelled.');
} else {
// outcome === 'failure'
core.info('[Reporter] Audit detected failures. Applying releng-audit-fail label...');
try {
await github.rest.issues.addLabels({
owner: context.repo.owner, repo: context.repo.repo,
issue_number: context.issue.number || context.payload.pull_request.number,
labels: ['releng-audit-fail']
});
core.info('[Reporter] Successfully applied releng-audit-fail label.');
} catch (e) {
core.error(`[Reporter] Failed to add fail label: ${e.message}`);
}
}
try {
core.info(`[Reporter] Creating final commit status '${state}' for context '${statusContext}' on SHA '${{ steps.router.outputs.pr_sha }}'...`);
await github.rest.repos.createCommitStatus({
owner: context.repo.owner,
repo: context.repo.repo,
sha: '${{ steps.router.outputs.pr_sha }}',
state: state,
context: statusContext,
description: description,
target_url: `https://github.com/${context.repo.owner}/${context.repo.repo}/actions/runs/${context.runId}`
});
core.info(`[Reporter] Set ${state} commit status (${statusContext}) on SHA: ${{ steps.router.outputs.pr_sha }}`);
} catch (e) {
core.error(`[Reporter] Failed to set final commit status: ${e.message}`);
}

View File

@ -8,6 +8,8 @@
#
#
a2batic Kanika Murarka <kmurarka@redhat.com>
aainscow Alex Ainscow <aainscow@uk.ibm.com>
aaryanporwal Aaryan Porwal <aaryanporwal2233@gmail.com>
aaSharma14 Aashish Sharma <aasharma@redhat.com>
abhidesai6 Abhishek Desai <abhidesai6.ad@gmail.com>
abhishek-kane Abhishek Kane <abhishek.kane@ibm.com> <abhishek.kane@gmail.com>
@ -22,25 +24,36 @@ alimaredia Ali Maredia <amaredia@redhat.com>
amathuria Aishwarya Mathuria <amathuri@redhat.com>
amitkumar50 Amit Kumar <amitkuma@redhat.com>
andrewschoen Andrew Schoen <aschoen@redhat.com>
anoopcs9 Anoop C S <anoopcs@cryptolab.net>
anthonyeleven Anthony D'Atri <anthony.datri@gmail.com>
anuradhagadge Anuradha Gadge <Anuradha.Gadge@ibm.com>
aaryanporwal Aaryan Porwal <aaryanporwal2233@gmail.com>
AnuragRaut08 Anurag Raut <anuragtraut08@ibm.com>
arbtfnf anuragbandhu <anuragbandhu007@gmail.com>
asettle Alexandra Settle <asettle@suse.com>
ashjosh1git Ashwin M Joshi <ashjosh1@in.ibm.com>
athanatos Samuel Just <sjust@redhat.com>
avanthakkar Avan Thakkar <athakkar@redhat.com>
b-ranto Boris Ranto <branto@redhat.com>
badone Brad Hubbard <bhubbard@redhat.com>
baergj Joshua Baergen <jbaergen@digitalocean.com>
baruza Barbora Ančincová <bara@redhat.com>
bassamtabbara Bassam Tabbara <bassam.tabbara@quantum.com>
batrick Patrick Donnelly <pdonnell@ibm.com>
baum Alexander Indenbaum <aindenba@redhat.com>
BBoozmen Oguzhan Ozmen <oozmen@bloomberg.net>
benhanokh Gabriel Benhanokh <gbenhano@redhat.com>
bigjust Justin Caratzas <jcaratza@redhat.com>
bill-scales Bill Scales <bill_scales@uk.ibm.com>
bk201 Kiefer Chang <kiefer.chang@suse.com>
BlaineEXE Blaine Gardner <bgardner@suse.com>
bluikko Ville Ojamo <git2233+ceph@ojamo.eu>
branch-predictor Piotr Dałek <piotr.dalek@corp.ovh.com>
b-ranto Boris Ranto <branto@redhat.com>
bryanmontalvan Bryan Montalvan <bmontalv@redhat.com>
callithea Laura Paduano <lpaduano@suse.com>
capri1989 Kai Wagner <kwagner@suse.com>
caroav Aviv Caro <Aviv.Caro@ibm.com>
cbodley Casey Bodley <cbodley@redhat.com>
cfsnyder Cory Snyder <csnyder@iland.com>
chardan Jesse Williamson <jwilliamson@suse.de>
chhabaramesh Ramesh Chander <Ramesh.Chander@sandisk.com>
chrisphoffman Christopher Hoffman <choffman@redhat.com>
@ -48,20 +61,24 @@ cloudbehl Ankush Behl <cloudbehl@gmail.com>
CourtneyCCaldwell Courtney Caldwell <ccaldwel@redhat.com>
Daniel-Pivonka Daniel Pivonka <dpivonka@redhat.com>
ddiss David Disseldorp <ddiss@suse.de>
devikab25 Devika Babrekar <devika.babrekar@ibm.com>
devikab25 Devika Babrekar <devika.babrekar@ibm.com>
Devp00l Stephan Müller <smueller@suse.com>
dillaman Jason Dillaman <dillaman@redhat.com>
djgalloway David Galloway <dgallowa@redhat.com>
dmick Dan Mick <dmick@redhat.com>
dnyanee1997 Dnyaneshwari talwekar <dtalweka@redhat.com>
dparmar18 Dhairya Parmar <dparmar@redhat.com>
dragonylffly Li Wang <laurence.liwang@gmail.com>
dsavineau Dimitri Savineau <dsavinea@redhat.com>
dubeyko Viacheslav Dubeyko <Slava.Dubeyko@ibm.com>
dvanders Dan van der Ster <dan.vanderster@clyso.com>
dzafman David Zafman <dzafman@redhat.com>
edwinzrodriguez Edwin Rodriguez <edwin.rodriguez1@ibm.com>
emmericp Paul Emmerich <paul.emmerich@croit.io>
epuertat Ernesto Puerta <epuertat@redhat.com>
ErwanAliasr1 Erwan Velu <erwan@redhat.com>
Exotelis Sebastian Krah <skrah@suse.com>
ffilz Frank S. Filz <ffilzlnx@mindspring.com>
fullerdj Douglas Fuller <dfuller@redhat.com>
GabrielBrascher Gabriel Brascher <gabriel@apache.org>
galsalomon66 Gal Salomon <gsalomon@redhat.com>
@ -74,21 +91,27 @@ idryomov Ilya Dryomov <idryomov@redhat.com>
ifed01 Igor Fedotov <ifedotov@suse.com>
ivancich J. Eric Ivancich <ivancich@redhat.com>
jan--f Jan Fajerski <jfajerski@suse.com>
Jayaprakash-ibm Jaya Prakash Madaka <jayaprakash@ibm.com>
jcsp John Spray <john.spray@redhat.com>
jdurgin Josh Durgin <jdurgin@redhat.com>
jecluis João Eduardo Luís <joao@suse.de>
jmolmo Juan Miguel Olmo <jolmomar@redhat.com>
jmundack Joseph Mundackal <jmundackal@bloomberg.net>
joscollin Jos Collin <jcollin@redhat.com>
josephsawaya Joseph Sawaya <jsawaya@redhat.com>
jschmid1 Joshua Schmid <jschmid@suse.de>
jtlayton Jeff Layton <jlayton@redhat.com>
kalaspuffar Daniel Persson <mailto.woden@gmail.com>
kamoltat Kamoltat Sirivadhna <ksirivad@redhat.com>
karthik-us1 Karthik U S <karthik.u.s1@ibm.com>
kchheda3 Krunal Chheda <kchheda3@bloomberg.net>
knrt10 Kautilya Tripathi <kautilya.tripathi@ibm.com>
kotreshhr Kotresh Hiremath Ravishankar <khiremat@redhat.com>
kshtsk Kyr Shatskyy <kyrylo.shatskyy@suse.com>
ktdreyer Ken Dreyer <kdreyer@redhat.com>
Kushal-deb Kushal Deb <Kushal.Deb@ibm.com>
LenzGr Lenz Grimmer <lgrimmer@suse.com>
leonid-s-usov Leonid Usov <leonid.usov@ibm.com>
leseb Sébastien Han <seb@redhat.com>
liewegas Sage Weil <sage@redhat.com>
liupan1111 Pan Liu <liupan1111@gmail.com>
@ -99,27 +122,35 @@ markhpc Mark Nelson <mnelson@redhat.com>
Matan-B Matan Breizman <mbreizma@redhat.com>
mattbenjamin Matt Benjamin <mbenjami@redhat.com>
mchangir Milind Changire <mchangir@redhat.com>
mctaggatart Sage McTaggart <sagemct@ibm.com>
melissa-kun-li Melissa Li <melissali@redhat.com>
mgfritch Michael Fritch <mfritch@suse.com>
mheler Matthew Heler <matthew.heler@hotmail.com>
mikechristie Mike Christie <mchristi@redhat.com>
mkogan1 Mark Kogan <mkogan@redhat.com>
mmgaggle Kyle Bader <kbader@ibm.com>
mogeb Mohamad Gebai <mgebai@suse.com>
MrFreezeex Arthur Outhenin-Chalandre <arthur.outhenin-chalandre@cern.ch>
myoungwon Myoungwon Oh <myoungwon.oh@samsung.com>
nmunet Naman Munet <nmunet@redhat.com>
Naveenaidu Naveen Naidu <naveen.naidu@ibm.com>
neesingh-rh Neeraj Pratap Singh <neesingh@redhat.com>
neha-ojha Neha Ojha <nojha@redhat.com>
NitzanMordhai Nitzan Mordechai <nmordech@redhat.com>
nizamial09 Nizamudeen A <nia@redhat.com>
nmshelke Nikhilkumar Shelke <nshelke@redhat.com>
nSedrickm Ngwa Sedrick Meh <nsedrick101@gmail.com>
nmunet Naman Munet <nmunet@redhat.com>
noahdesu Noah Watkins <nwatkins@redhat.com>
nSedrickm Ngwa Sedrick Meh <nsedrick101@gmail.com>
oritwas Orit Wasserman <owasserm@redhat.com>
p-na Patrick Nawracay <pnawracay@suse.com>
p-se Patrick Seidensal <pseidensal@suse.com>
parth-gr Parth Arora <paarora@redhat.com>
pcuzner Paul Cuzner <pcuzner@redhat.com>
Pegonzal Pedro Gonzalez Gomez <pegonzal@redhat.com>
pereman2 Pere Diaz Bou <pdiazbou@redhat.com>
petrutlucian94 Lucian Petrut <lpetrut@cloudbasesolutions.com>
phlogistonjohn John Mulligan <jmulligan@redhat.com>
p-na Patrick Nawracay <pnawracay@suse.com>
prgoel-code Prachi prgoel@redhat.com
p-se Patrick Seidensal <pseidensal@suse.com>
pujaoshahu Puja Shahu <pshahu@redhat.com>
rchagam Anjaneya Chagam <anjaneya.chagam@intel.com>
renhwztetecs huanwen ren <ren.huanwen@zte.com.cn>
@ -127,26 +158,33 @@ ricardoasmarques Ricardo Marques <rimarques@suse.com>
rishabh-d-dave Rishabh Dave <ridave@redhat.com>
rjfd Ricardo Dias <rdias@suse.com>
rkachach Redouane Kachach <rkachach@redhat.com>
robbat2 Robin H. Johnson <robbat2@orbis-terrarum.net>
ronen-fr Ronen Friedman <rfriedma@redhat.com>
runsisi luo runbing <luo.runbing@zte.com.cn>
rzarzynski Radoslaw Zarzynski <rzarzyns@redhat.com>
s0nea Tatjana Dehler <tdehler@suse.com>
sagargopale2233 Sagar Gopale <sagar.gopale@ibm.com>
salieri11 Igor Golikov <igolikov@ibm.com>
Sarthak0702 Sarthak Gupta <sarthak.dev.0702@gmail.com>
saschagrunert Sascha Grunert <sgrunert@suse.com>
sebastian-philipp Sebastian Wagner <sewagner@redhat.com>
shraddhaag Shraddha Agrawal <shraddhaag@ibm.com>
Kushal-deb Kushal Deb <Kushal.Deb@ibm.com>
ShwetaBhosale1 Shweta Bhosale <Shweta.Bhosale1@ibm.com>
Shwetha-Acharya Shwetha Acharya <sacharya@redhat.com>
ShyamsundarR Shyamsundar R <srangana@redhat.com>
sidharthanup Sidharth Anupkrishnan <sanupkri@redhat.com>
smanjara Shilpa Jagannath <smanjara@redhat.com>
smithfarm Nathan Cutler <ncutler@suse.com>
spuiuk Sachin Prabhu <sp@spui.uk>
sunilangadi2 Sunil Angadi <Sunil.Angadi@ibm.com>
sunnyku Sunny Kumar <sunkumar@redhat.com>
synarete Shachar Sharon <ssharon@redhat.com>
szuraski898 Steven Zuraski <steven.zuraski@ibm.com>
taodd dongdong tao <tdd21151186@gmail.com>
tchaikov Kefu Chai <k.chai@proxmox.com>
theanalyst Abhishek Lekshmanan <abhishek.lekshmanan@gmail.com>
Thingee Mike Perez <miperez@redhat.com>
ThomasLamprecht Thomas Lamprecht <t.lamprecht@proxmox.com>
toabctl Thomas Bechtold <tbechtold@suse.com>
travisn Travis Nielsen <tnielsen@redhat.com>
trociny Mykola Golub <mgolub@suse.com>
@ -166,12 +204,13 @@ vumrao Vikhyat Umrao <vikhyat@redhat.com>
Waadkh7 Waad Alkhoury <walkhour@redhat.com>
wido Wido den Hollander <wido@42on.com>
wjwithagen Willem Jan Withagen <wjw@digiware.nl>
xhernandez Xavi Hernandez <xhernandez@gmail.com>
xiaoxichen Xiaoxi Chen <superdebuger@gmail.com>
xiexingguo xie xingguo <xie.xingguo@zte.com.cn>
xxhdx1985126 Xuehan Xu <xuxuehan@360.cn>
yaarith Yaarit Hatuka <yaarithatuka@gmail.com>
Yan-waller yanjun <yan.jun8@zte.com.cn>
yangdongsheng Dongsheng Yang <dongsheng.yang@easystack.cn>
Yan-waller yanjun <yan.jun8@zte.com.cn>
yehudasa Yehuda Sadeh <yehuda@redhat.com>
yunfeiguan Yunfei Guan <yunfei.guan@xtaotech.com>
yuriw Yuri Weinstein <yweins@redhat.com>
@ -179,37 +218,3 @@ yuvalif Yuval Lifshitz <ylifshit@redhat.com>
yuyuyu101 Haomai Wang <haomai@xsky.com>
zdover23 Zac Dover <zac.dover@gmail.com>
zmc Zack Cerza <zack@redhat.com>
Thingee Mike Perez <miperez@redhat.com>
cfsnyder Cory Snyder <csnyder@iland.com>
benhanokh Gabriel Benhanokh <gbenhano@redhat.com>
kamoltat Kamoltat Sirivadhna <ksirivad@redhat.com>
anthonyeleven Anthony D Atri <anthony.datri@gmail.com>
petrutlucian94 Lucian Petrut <lpetrut@cloudbasesolutions.com>
dparmar18 Dhairya Parmar <dparmar@redhat.com>
nmshelke Nikhilkumar Shelke <nshelke@redhat.com>
neesingh-rh Neeraj Pratap Singh <neesingh@redhat.com>
parth-gr Parth Arora <paarora@redhat.com>
phlogistonjohn John Mulligan <jmulligan@redhat.com>
baergj Joshua Baergen <jbaergen@digitalocean.com>
zmc Zack Cerza <zack@redhat.com>
robbat2 Robin H. Johnson <robbat2@orbis-terrarum.net>
leonid-s-usov Leonid Usov <leonid.usov@ibm.com>
ffilz Frank S. Filz <ffilzlnx@mindspring.com>
Jayaprakash-ibm Jaya Prakash Madaka <jayaprakash@ibm.com>
spuiuk Sachin Prabhu <sp@spui.uk>
anoopcs9 Anoop C S <anoopcs@cryptolab.net>
dubeyko Viacheslav Dubeyko <Slava.Dubeyko@ibm.com>
bill-scales Bill Scales <bill_scales@uk.ibm.com>
kchheda3 Krunal Chheda <kchheda3@bloomberg.net>
Shwetha-Acharya Shwetha Acharya <sacharya@redhat.com>
xhernandez Xavi Hernandez <xhernandez@gmail.com>
ThomasLamprecht Thomas Lamprecht <t.lamprecht@proxmox.com>
jmundack Joseph Mundackal <jmundackal@bloomberg.net>
edwinzrodriguez Edwin Rodriguez <edwin.rodriguez1@ibm.com>
synarete Shachar Sharon <ssharon@redhat.com>
ashjosh1git Ashwin M Joshi <ashjosh1@in.ibm.com>
mkogan1 Mark Kogan <mkogan@redhat.com>
smanjara Shilpa Jagannath <smanjara@redhat.com>
arbtfnf anuragbandhu <anuragbandhu007@gmail.com>
aainscow Alex Ainscow <aainscow@uk.ibm.com>
salieri11 Igor Golikov <igolikov@ibm.com>

3
.gitmodules vendored
View File

@ -85,3 +85,6 @@
[submodule "src/lss"]
path = src/lss
url = https://chromium.googlesource.com/linux-syscall-support
[submodule "src/librdkafka"]
path = src/librdkafka
url = https://github.com/confluentinc/librdkafka.git

View File

@ -68,6 +68,7 @@ Anton Oks <anton.oks@gmx.de>
Anton Turetckii <tyrchenok@gmail.com> banuchka <tyrchenok@gmail.com>
Anuradha Gadge <anuradha.gadge@ibm.com> <anuradhagadge18@gmail.com>
Anurag Bandhu <abandhu@redhat.com> <anurag@localhost.localdomain>
Anurag Raut <anuragtraut08@ibm.com> <anuragtraut2003@gmail.com>
Aravind Ramesh <Aravind.Ramesh@wdc.com> Aravind <aravind.ramesh@wdc.com>
Aristoteles Neto <aristoteles.neto@webdrive.co.nz> <wdneto@users.noreply.github.com>
Aron Gunn <ritz_303@yahoo.com>

View File

@ -353,6 +353,7 @@ IBM <contact@IBM.com> Afreen Misbah <afreen@ibm.com>
IBM <contact@IBM.com> Aliaksei Makarau <aliaksei.makarau@ibm.com>
IBM <contact@IBM.com> Andrew Solomon <asolomon@us.ibm.com>
IBM <contact@IBM.com> Anuradha Gadge <Anuradha.Gadge@ibm.com>
IBM <contact@IBM.com> Anurag Raut <anuragtraut08@ibm.com>
IBM <contact@IBM.com> Devika Babrekar <Devika.Babrekar@ibm.com>
IBM <contact@IBM.com> Dnyaneshwari talwekar <Dnyaneshwari.Talwekar@ibm.com>
IBM <contact@IBM.com> Guillaume Abrioux <gabrioux@ibm.com>

View File

@ -75,6 +75,20 @@ if(MINGW)
link_directories(${MINGW_LINK_DIRECTORIES})
endif()
option(WITH_MOLD "Use the Mold linker" OFF)
if(WITH_MOLD)
find_program(_mold_path mold REQUIRED)
message(STATUS "Mold linker: ${_mold_path}")
set(CMAKE_LINKER ${_mold_path} CACHE FILEPATH "Linker" FORCE)
set(MOLD_FUSE_LD_FLAG "-fuse-ld=mold" CACHE INTERNAL "")
foreach(_flags CMAKE_EXE_LINKER_FLAGS CMAKE_SHARED_LINKER_FLAGS CMAKE_MODULE_LINKER_FLAGS)
if(NOT "${${_flags}}" MATCHES "fuse-ld=")
string(APPEND ${_flags} " ${MOLD_FUSE_LD_FLAG}")
endif()
endforeach()
set(USING_MOLD_LINKER TRUE CACHE INTERNAL "")
endif()
option(WITH_CCACHE "Build with ccache.")
if(WITH_CCACHE)
if(CMAKE_C_COMPILER_LAUNCHER OR CMAKE_CXX_COMPILER_LAUNCHER)
@ -103,13 +117,13 @@ if(WITH_SCCACHE)
ERROR_VARIABLE sccache_dist_status_error
GET "${sccache_dist_status}" SchedulerStatus 1 num_cpus
)
string(FIND "${sccache_dist_status}" "disabled" find_result)
if(find_result EQUAL -1)
if(sccache_dist_status_error)
message(WARNING "Using sccache, but it is not configured for distributed "
"compilation: ${sccache_dist_status_error}")
else()
message(STATUS "Using sccache with distributed compilation. Effective cores: ${sccache_cores}")
set(NINJA_MAX_COMPILE_JOBS ${sccache_cores})
set(NINJA_MAX_LINK_JOBS ${sccache_cores})
else()
message(WARNING "Using sccache, but it is not configured for distributed complilation")
endif()
else()
message(WARNING "Using sccache, but cannot determine maximum job value since cmake version is <3.19")
@ -209,7 +223,6 @@ if(LINUX)
elseif(FREEBSD)
set(HAVE_UDEV OFF)
set(HAVE_LIBAIO OFF)
set(HAVE_LIBDML OFF)
set(HAVE_BLKID OFF)
set(HAVE_KEYUTILS OFF)
else()
@ -290,13 +303,6 @@ if(WITH_LIBURING)
endif()
endif()
CMAKE_DEPENDENT_OPTION(WITH_BLUESTORE_PMEM "Enable PMDK libraries" OFF
"WITH_BLUESTORE" OFF)
if(WITH_BLUESTORE_PMEM)
find_package(dml)
set(HAVE_LIBDML ${DML_FOUND})
endif()
CMAKE_DEPENDENT_OPTION(WITH_RBD_MIRROR "Enable build for rbd-mirror daemon executable" OFF
"WITH_RBD" OFF)
@ -311,7 +317,7 @@ CMAKE_DEPENDENT_OPTION(WITH_RBD_SSD_CACHE "Enable librbd persistent write back c
"WITH_RBD" OFF)
CMAKE_DEPENDENT_OPTION(WITH_SYSTEM_PMDK "Require and build with system PMDK" OFF
"WITH_RBD_RWL OR WITH_BLUESTORE_PMEM" OFF)
"WITH_RBD_RWL" OFF)
CMAKE_DEPENDENT_OPTION(WITH_RBD_UBBD "Enable ubbd support for rbd device utility" OFF
"WITH_RBD" OFF)
@ -321,10 +327,6 @@ if(WITH_RBD_UBBD)
build_ubbd()
endif()
if(WITH_BLUESTORE_PMEM)
set(HAVE_BLUESTORE_PMEM ON)
endif()
CMAKE_DEPENDENT_OPTION(WITH_SPDK "Enable SPDK" OFF
"CMAKE_SYSTEM_PROCESSOR MATCHES i386|i686|amd64|x86_64|AMD64|aarch64|riscv64" OFF)
option(WITH_SYSTEM_SPDK "Require a system SPDK instead of building bundled src/spdk" OFF)
@ -342,10 +344,10 @@ if(WITH_SPDK)
endif(WITH_SPDK)
if(WITH_BLUESTORE)
if(NOT AIO_FOUND AND NOT HAVE_POSIXAIO AND NOT WITH_SPDK AND NOT WITH_BLUESTORE_PMEM)
if(NOT AIO_FOUND AND NOT HAVE_POSIXAIO AND NOT WITH_SPDK)
message(SEND_ERROR "WITH_BLUESTORE is ON, "
"but none of the bluestore backends is enabled. "
"Please install libaio, or enable WITH_SPDK or WITH_BLUESTORE_PMEM (experimental)")
"Please install libaio, or enable WITH_SPDK")
endif()
endif()
@ -460,6 +462,10 @@ if(ALLOCATOR)
elseif(NOT ALLOCATOR STREQUAL "libc")
message(FATAL_ERROR "Unsupported allocator selected: ${ALLOCATOR}")
endif()
elseif(WITH_ASAN)
# Force libc: tcmalloc/jemalloc still export operator new/delete even when the
# sanitizer shadows malloc, so sanitizer-allocated memory is freed via tcmalloc and SIGSEGVs.
set(ALLOCATOR "libc")
else(ALLOCATOR)
find_package(gperftools 2.6.2)
set(HAVE_LIBTCMALLOC ${gperftools_FOUND})
@ -569,12 +575,15 @@ option(WITH_RADOSGW "RADOS Gateway is enabled" ON)
option(WITH_RADOSGW_BEAST_OPENSSL "RADOS Gateway's Beast frontend uses OpenSSL" ON)
option(WITH_RADOSGW_AMQP_ENDPOINT "RADOS Gateway's pubsub support for AMQP push endpoint" ON)
option(WITH_RADOSGW_KAFKA_ENDPOINT "RADOS Gateway's pubsub support for Kafka push endpoint" ON)
option(WITH_SYSTEM_RDKAFKA "build against system librdkafka instead of the bundled submodule" OFF)
option(WITH_RADOSGW_LUA_PACKAGES "RADOS Gateway's support for dynamically adding lua packagess" ON)
option(WITH_RADOSGW_FDB "FoundationDB support for RADOS Gateway (experimental)" OFF)
option(WITH_RADOSGW_DBSTORE "DBStore backend for RADOS Gateway" ON)
option(WITH_RADOSGW_MOTR "CORTX-Motr backend for RADOS Gateway" OFF)
option(WITH_RADOSGW_DAOS "DAOS backend for RADOS Gateway" OFF)
option(WITH_RADOSGW_D4N "D4N wrapper for RADOS Gateway" ON)
cmake_dependent_option(WITH_RADOSGW_POSIX "POSIX backend for RADOS Gateway" ON WITH_RADOSGW_DBSTORE OFF) # posix depends on dbstore
option(WITH_RADOSGW_POSIX "POSIX backend for RADOS Gateway" ON)
cmake_dependent_option(WITH_RADOSGW_STANDALONE "Standalone RADOS Gateway" ON WITH_RADOSGW_POSIX OFF)
option(WITH_RADOSGW_RADOS "RADOS backend for Rados Gateway" ON)
option(WITH_RADOSGW_SELECT_PARQUET "Support for s3 select on parquet objects" ON)
option(WITH_RADOSGW_ARROW_FLIGHT "Build arrow flight when not using system-provided arrow" OFF)
@ -583,6 +592,9 @@ option(WITH_RADOSGW_BACKTRACE_LOGGING "Enable backtraces in rgw logs" OFF)
option(WITH_SYSTEM_ARROW "Use system-provided arrow" OFF)
option(WITH_SYSTEM_UTF8PROC "Use system-provided utf8proc" OFF)
# POSIX depends on DBStore:
cmake_dependent_option(WITH_RADOSGW_POSIX "POSIX backend for RADOS Gateway" ON WITH_RADOSGW_DBSTORE OFF)
if(WITH_RADOSGW)
find_package(EXPAT REQUIRED)
find_package(OATH REQUIRED)
@ -636,6 +648,23 @@ if(WITH_RADOSGW)
message(STATUS "crypto soname: ${LIBCRYPTO_SONAME}")
endif (WITH_RADOSGW)
if(WITH_RADOSGW_FDB)
include(${CMAKE_MODULE_PATH}/CPM.cmake)
CPMAddPackage("gh:eyalz800/zpp_bits@4.5.1")
message("-- enabled zpp_bits")
# JFW: sadly, I'm having trouble getting FoundationDB to build, but this is VERY close
# to "just working". But, how would we install the RPMs, etc.? This will need looking at
# before this can be non-experimental:
# CPMAddPackage("gh:apple/foundationdb@7.3.63")
message("FoundationDB support is EXPERIMENTAL and requires manual setup:")
message(" wget https://github.com/apple/foundationdb/releases/download/7.3.63/foundationdb-clients-7.3.63-1.el7.x86_64.rpm")
message(" wget https://github.com/apple/foundationdb/releases/download/7.3.63/foundationdb-server-7.3.63-1.el7.x86_64.rpm")
message("-- enabled FoundationDB (whether its there or not)")
endif()
#option for CephFS
option(WITH_CEPHFS "CephFS is enabled" ON)
@ -720,6 +749,13 @@ endif(LINUX)
option(WITH_ASAN "build with ASAN" OFF)
if(WITH_ASAN)
list(APPEND sanitizers "address")
# Shared ASan/LSan runtime options: the in-tree suppression files plus the
# flags our tests need. Consumed by add_ceph_test() and baked into bin/ceph
# so both ignore the same still-reachable third-party leaks.
set(CEPH_ASAN_OPTIONS "suppressions=${CMAKE_SOURCE_DIR}/qa/asan.supp,detect_odr_violation=0"
CACHE INTERNAL "ASAN_OPTIONS for ceph tests and the ceph CLI")
set(CEPH_LSAN_OPTIONS "suppressions=${CMAKE_SOURCE_DIR}/qa/lsan.supp,print_suppressions=0"
CACHE INTERNAL "LSAN_OPTIONS for ceph tests and the ceph CLI")
endif()
option(WITH_ASAN_LEAK "explicitly enable ASAN leak detection" OFF)
@ -817,6 +853,12 @@ else()
include(BuildBoost)
build_boost(1.87
COMPONENTS ${BOOST_COMPONENTS} ${BOOST_HEADER_COMPONENTS})
if(WITH_ASAN)
# Boost.Context is built ucontext-only here (context-impl=ucontext); define
# the matching backend tree-wide so every Boost.Context/Coroutine2 consumer
# agrees on it, instead of relying on per-target propagation that is easy to miss.
add_compile_definitions(BOOST_USE_ASAN BOOST_USE_UCONTEXT)
endif()
endif()
include_directories(BEFORE SYSTEM ${Boost_INCLUDE_DIRS})
add_library(Boost::asio INTERFACE IMPORTED)
@ -893,4 +935,3 @@ add_tags(ctags
EXCLUDE_OPTS ${CTAG_EXCLUDES}
EXCLUDES "*.js" "*.css" ".tox" "python-common/build")
add_custom_target(tags DEPENDS ctags)

View File

@ -36,7 +36,9 @@ RUN DISTRO=$DISTRO \
FOR_MAKE_CHECK=${FOR_MAKE_CHECK} \
bash -x ${CEPH_CTR_SRC}/buildcontainer-setup.sh
RUN \
if [ $(uname -m) != ppc64le ]; then \
if rpm --quiet --query sccache 2>/dev/null || { command -v dnf >/dev/null 2>&1 && dnf install -y sccache 2>/dev/null; }; then \
echo "sccache provided by distro packages"; \
elif [ $(uname -m) != ppc64le ]; then \
SCCACHE_ARCH="$(uname -m)"; \
if [ "$SCCACHE_ARCH" = riscv64 ]; then SCCACHE_ARCH=riscv64gc; fi; \
SCCACHE_URL="${SCCACHE_REPO}/releases/download/${SCCACHE_VERSION}/sccache-${SCCACHE_VERSION}-${SCCACHE_ARCH}-unknown-linux-musl.tar.gz"; \

View File

@ -1,3 +1,22 @@
* CephFS: The ``client_force_lazyio`` configuration option is now correctly marked
as not supporting runtime updates. Previously, the configuration schema indicated
this option could be changed at runtime, but changes had no effect on opened file
handlers in ceph-fuse or libcephfs clients because there is no logic to propagate
changes to open file handles.
* RADOS: PG autoscaler allows overlapping roots. Each root receives a PG target based
on its OSDs, with OSDs shared across multiple roots contributing proportionally
less to each root's allocation.
* CephFS: CephFS snapshots metadata is now mutable. It is now possible to add,
update and remove existing key-value pairs that are part of a snapshot
metadata via libcephfs API ceph_do_snap_md_op().
* CephFS: MDS dmclock support for subvolume QoS. With this feature, MDS assigns QoS
to throttle client metadata requests (e.g., create, mkdir, lookup, and so on) to
subvolumes, where each subvolume QoS is shared among multiple client sessions at
that time.
* librados/neorados: The C++ APIs for executing Ceph Class (CLS) methods
have undergone a breaking change to enforce compile-time type safety, replacing
legacy string-based parameters with strongly-typed ClsMethod structs. C++
@ -17,6 +36,8 @@
- The `rgw default data log backing` option is removed and it is no longer
possible to create clusters with an omap based datalog.
- `radosgw-admin datalog type` will only accept `--log_type=fifo`.
* RGW: iam:RemoveClientIdFromOIDCProvider is now a recognized action for
policy, corrected from a typo of iam:RemoveCientIdFromOIDCProvider
* MGR: The default values of ``mon_target_pg_per_osd`` and ``mon_max_pg_per_osd``
have been increased from 100 and 250 to 200 and 500, respectively. These values
@ -47,6 +68,12 @@
If the default provider is required when also using custom providers,
it must be explicitly loaded in the configuration file or code (see https://github.com/openssl/openssl/blob/master/README-PROVIDERS.md).
* RGW: Fixed bucket notification events so the 'x_amz_request_id' in NotificationEvent now matches the 'x_amz_request_id' returned by the corresponding S3 operation.
* RGW: The rgw_gc_max_deferred and rgw_gc_max_deferred_entries_size options have been removed, as they did not do anything.
- Updated the default for rgw_gc_max_queue_size to account for the extra space from removing rgw_gc_max_deferred_entries_size.
* RGW: The default value of ``rgw_thread_pool_size`` has been reduced from 512 to 128.
Benchmarks showed that the lower thread count improves throughput and reduces latency
on NVMe and HDD based environments. Users with specific workload requirements can
tune this value; see also ``rgw_max_concurrent_requests``.
* DASHBOARD: A new Overview landing page provides an at-a-glance cluster summary
@ -165,6 +192,35 @@
``ceph fs snapshot mirror daemon status`` now shows the remote cluster's
monitor addresses and cluster ID for each configured peer, making it easier
to verify peer connectivity and troubleshoot mirroring issues.
* CephFS Mirroring: The ``fs mirror peer status`` admin socket command reports
additional per-directory sync metrics (sync mode, throughput, crawl and
data-sync queue timing, bytes/files progress, and ETA). Output is grouped
under ``metrics/<mirrored-dir-path>/peer/<peer-uuid>``. For more information,
see https://tracker.ceph.com/issues/73453
* CephFS Mirroring: Per-directory snapshot sync progress is exposed as labeled perf
counters in the ``cephfs_mirror_directory`` group (``counter dump`` on the mirror
daemon admin socket, exportable via ``ceph-exporter``). Counters mirror
``fs mirror peer status`` fields (directory state, current sync, last sync, and
snap summary) and are labeled by peer UUID and mirrored directory path. See
https://tracker.ceph.com/issues/73457
* CephFS Mirroring: The mirroring module provides ``ceph fs snapshot mirror status``,
a Ceph CLI command similar to the ``fs mirror peer status`` admin socket interface
for viewing per-directory snapshot sync metrics. The output layout matches
``fs mirror peer status`` for core sync fields and additionally includes
``metrics_updated_at`` (time of the last omap write). Optional filters by
mirrored directory path and peer UUID are supported. For more information,
see https://tracker.ceph.com/issues/76686
* CephFS Mirroring: Introduced snapshot checkpoints feature that allows users to mark
specific snapshots as important milestones and track their replication status to remote
sites. Checkpoints automatically detect if a snapshot has already been synced and provide
visibility into disaster recovery readiness. The feature includes four new CLI commands:
``ceph fs snapshot mirror checkpoint add`` to mark a snapshot,
``ceph fs snapshot mirror checkpoint now`` to checkpoint the latest snapshot,
``ceph fs snapshot mirror checkpoint ls`` to list all checkpoints with their status
(created/complete/failed), and ``ceph fs snapshot mirror checkpoint remove`` to remove
a checkpoint. Checkpoint metadata is persistent across daemon restarts. This feature is
useful for compliance auditing, application consistency verification, and SLA tracking.
For more information, see https://tracker.ceph.com/issues/73454
* RBD: Mirror snapshot creation and trash purge schedules are now automatically
staggered when no explicit "start-time" is specified. This reduces scheduling
spikes and distributes work more evenly over time.
@ -175,6 +231,17 @@
than instantly aborting I/O and returning an error. It's possible to switch
from ``RBD_LOCK_MODE_EXCLUSIVE`` to ``RBD_LOCK_MODE_EXCLUSIVE_TRANSIENT``
policy and vice versa even if the lock is already held.
* RGW: In multisite deployments, zone endpoints configured in the zonegroup
(e.g. ``https://zone-a.example.com``) can now be resolved to all IP addresses
returned by DNS, with inter-zone traffic distributed across them using
round-robin and per-IP health tracking. This enables DNS-based service discovery
for inter-zone traffic without an external load balancer. This feature is disabled
by default: to opt in, set ``rgw_rest_conn_connect_to_resolved_ips = true``. The
per-IP retry-after-failure timeout is controlled by ``rgw_rest_conn_ip_fail_timeout_secs``
(default: 2 seconds). Deployments that work around libcurl's single-address behavior
by repeating the same endpoint multiple times in zone configuration should review that
setup before enabling, as round-robin distribution makes such duplication unnecessary.
* OSD: A health warning is reported when BlueFS usage exceeds the
configured ratio of the main OSD data device size. This warning is
informational and can be muted with:
@ -196,6 +263,28 @@
state and a subsequent redundancy loss occurs. Used in conjunction with
``last_clean``, the ``last_degraded`` timestamp enables the calculation of
data vulnerability and durability scores.
* RBD: It's possible to specify source cluster's ``mon_host`` and ``key`` for
``native`` format migration via the migration spec now. This eliminates the
dependency on ``<cluster-name>.conf`` file in a known location which is
rather rigid and also challenging to disseminate in some environments. The
key can be embedded in the migration spec or just referenced from there while
stored in the MON config-key store.
* RBD: `Group::list_images` python API will now correctly raise an ObjectNotFound
error when invoked on a non-existent group. The C APIs `rbd_group_image_list`,
`rbd_snap_list`, `rbd_group_snap_list` and C++ API `group_image_list` will now
return ENOENT instead of masking the error and returning 0.
* BlueStore: The experimental PMEM device backend has been removed, together
with the ``pmem`` value of the ``bdev_type`` option and the DML/DSA offload
path. The hardware it targeted, Intel Optane DC persistent memory, was
discontinued in 2022. The RBD persistent write-back cache, which also uses
PMDK, is unaffected.
Existing OSDs backed by persistent memory in fsdax mode (a block device such
as ``/dev/pmem0``) keep working on the default ``aio`` backend over the same
data. Remove any explicit ``bdev_type = pmem`` from their configuration. OSDs
on a devdax namespace (a character device such as ``/dev/dax0.0``) will no
longer start, because no remaining backend can open one; reconfigure the
namespace to fsdax with ``ndctl`` or re-provision them before upgrading.
>=20.0.0
@ -429,6 +518,9 @@
to upgrade all OSDs in the cluster. For more details see tracker:
https://tracker.ceph.com/issues/73031.
* NFS: cluster/export delete commands now throw a deprecation warning when
used and will be removed in V release.
>=19.2.1
* CephFS: The `fs subvolume create` command now allows tagging subvolumes through option
@ -996,3 +1088,13 @@ can be configured using the `rgw` option `rgw_bucket_persistent_notif_num_shards
https://docs.ceph.com/en/latest/radosgw/notifications/
Relevant tracker: https://tracker.ceph.com/issues/71677
* RGW: Multipart upload part heads are now tracked under a new
``rgw.multipart`` category in bucket stats, separate from completed
objects (``rgw.main``). This makes ``radosgw-admin bucket stats``
and the ``X-RGW-Object-Count`` header on HEAD Bucket more accurate:
in-progress or abandoned multipart parts no longer inflate the
object count. Existing in-progress uploads retain the old category
until they complete or abort.
Relevant tracker: https://tracker.ceph.com/issues/77509

View File

@ -1,2 +1 @@
plantweb
readthedocs-sphinx-search@git+https://github.com/readthedocs/readthedocs-sphinx-search@main

File diff suppressed because it is too large Load Diff

View File

@ -33,8 +33,8 @@ function(add_ceph_test test_name test_path)
set_property(TEST ${test_name}
APPEND
PROPERTY ENVIRONMENT
ASAN_OPTIONS=suppressions=${CMAKE_SOURCE_DIR}/qa/asan.supp,detect_odr_violation=0
LSAN_OPTIONS=suppressions=${CMAKE_SOURCE_DIR}/qa/lsan.supp,print_suppressions=0)
ASAN_OPTIONS=${CEPH_ASAN_OPTIONS}
LSAN_OPTIONS=${CEPH_LSAN_OPTIONS})
endif()
set_property(TEST ${test_name}
PROPERTY TIMEOUT ${CEPH_TEST_TIMEOUT})
@ -73,8 +73,11 @@ if(WITH_GTEST_PARALLEL)
BUILD_COMMAND ""
INSTALL_COMMAND "")
add_dependencies(tests gtest-parallel_ext)
# CACHE INTERNAL: the set() runs only in the first directory to create the
# target, so a plain variable would be invisible to PARALLEL tests elsewhere.
set(GTEST_PARALLEL_COMMAND
${Python3_EXECUTABLE} ${gtest_parallel_source_dir}/gtest-parallel)
${Python3_EXECUTABLE} ${gtest_parallel_source_dir}/gtest-parallel
CACHE INTERNAL "command to run a gtest binary through gtest-parallel")
endif()
endif()

View File

@ -144,15 +144,26 @@ function(do_build_boost root_dir version)
if(WITH_BOOST_VALGRIND)
list(APPEND b2 valgrind=on)
endif()
set(b2_targets headers stage)
set(b2_install_targets install)
if(WITH_ASAN)
list(APPEND b2 context-impl=ucontext)
# build the library with the BOOST_USE_ASAN consumers get from Boost::context,
# so fiber_activation_record has one layout (else heap-buffer-overflow)
list(APPEND b2 define=BOOST_USE_ASAN)
# `context-impl` is declared in libs/context/build/Jamfile.v2; the headers/stage
# and install targets never load it, so b2 aborts with `unknown feature
# "<context-impl>"`. Name the context project as a target so its Jamfile loads
# the feature first.
list(PREPEND b2_targets libs/context/build)
list(PREPEND b2_install_targets libs/context/build)
endif()
set(build_command
${b2} headers stage
${b2} ${b2_targets}
#"--buildid=ceph" # changes lib names--can omit for static
${boost_features})
set(install_command
${b2} install)
${b2} ${b2_install_targets})
if(EXISTS "${PROJECT_SOURCE_DIR}/src/boost/bootstrap.sh")
check_boost_version("${PROJECT_SOURCE_DIR}/src/boost" ${version})
set(source_dir
@ -254,10 +265,8 @@ macro(build_boost version)
set_target_properties(Boost::${c} PROPERTIES
INTERFACE_COMPILE_DEFINITIONS "BOOST_USE_VALGRIND")
endif()
if((c MATCHES "context") AND (WITH_ASAN))
set_target_properties(Boost::${c} PROPERTIES
INTERFACE_COMPILE_DEFINITIONS "BOOST_USE_ASAN;BOOST_USE_UCONTEXT")
endif()
# ASan's BOOST_USE_ASAN/BOOST_USE_UCONTEXT are defined tree-wide in the
# top-level CMakeLists.txt, not per-target.
list(APPEND Boost_LIBRARIES ${Boost_${upper_c}_LIBRARY})
endforeach()
foreach(c ${Boost_BUILD_COMPONENTS})

View File

@ -30,10 +30,10 @@ function(build_fio)
file(MAKE_DIRECTORY ${source_dir})
ExternalProject_Add(fio_ext
UPDATE_COMMAND "" # this disables rebuild on each run
GIT_REPOSITORY "https://github.com/ceph/fio.git"
GIT_REPOSITORY "https://github.com/axboe/fio.git"
GIT_CONFIG advice.detachedHead=false
GIT_SHALLOW 1
GIT_TAG "fio-3.27-cxx"
GIT_TAG "fio-3.42"
SOURCE_DIR ${source_dir}
BUILD_IN_SOURCE 1
CONFIGURE_COMMAND <SOURCE_DIR>/configure

View File

@ -1,4 +1,4 @@
function(build_pmdk enable_ndctl)
function(build_pmdk)
include(FindMake)
find_make("MAKE_EXECUTABLE" "make_cmd")
@ -15,12 +15,6 @@ function(build_pmdk enable_ndctl)
endif()
set(LIBPMEM_INTERFACE_LINK_LIBRARIES Threads::Threads)
if(${enable_ndctl})
set(ndctl "y")
list(APPEND LIBPMEM_INTERFACE_LINK_LIBRARIES ndctl::ndctl daxctl::daxctl)
else()
set(ndctl "n")
endif()
# Use debug PMDK libs in debug lib/rbd builds
if(CMAKE_BUILD_TYPE STREQUAL Debug)
@ -34,7 +28,7 @@ function(build_pmdk enable_ndctl)
ExternalProject_Add(pmdk_ext
${source_dir_args}
CONFIGURE_COMMAND ""
BUILD_COMMAND ${make_cmd} CC=${CMAKE_C_COMPILER} "EXTRA_CFLAGS=${pmdk_cflags}" NDCTL_ENABLE=${ndctl} BUILD_EXAMPLES=n BUILD_BENCHMARKS=n DOC=n
BUILD_COMMAND ${make_cmd} CC=${CMAKE_C_COMPILER} "EXTRA_CFLAGS=${pmdk_cflags}" NDCTL_ENABLE=n BUILD_EXAMPLES=n BUILD_BENCHMARKS=n DOC=n
BUILD_IN_SOURCE 1
BUILD_BYPRODUCTS "<SOURCE_DIR>/src/${PMDK_LIB_DIR}/libpmem.a" "<SOURCE_DIR>/src/${PMDK_LIB_DIR}/libpmemobj.a"
INSTALL_COMMAND "")

View File

@ -202,3 +202,12 @@ try_compile(HAVE_LINK_EXCLUDE_LIBS
${CMAKE_CURRENT_BINARY_DIR}
SOURCES ${CMAKE_CURRENT_LIST_DIR}/CephCheck_link.c
LINK_LIBRARIES "-Wl,--exclude-libs,ALL")
# Mold linker applies --exclude-libs after version script processing,
# which hides .symver-aliased symbols (e.g. rados_*) even when the
# version script lists them as global. Disable --exclude-libs for Mold;
# version scripts already control symbol visibility.
if(HAVE_LINK_EXCLUDE_LIBS AND USING_MOLD_LINKER)
message(STATUS "Mold linker -- disabling --exclude-libs (incompatible with .symver)")
set(HAVE_LINK_EXCLUDE_LIBS FALSE CACHE INTERNAL "" FORCE)
endif()

View File

@ -72,7 +72,11 @@ function(distutils_add_cython_module target name src)
list(APPEND PY_CPPFLAGS -D'__Pyx_check_single_interpreter\(ARG\)=ARG\#\#0')
set(PY_CC ${compiler_launcher} ${CMAKE_C_COMPILER} ${c_compiler_arg1})
set(PY_CXX ${compiler_launcher} ${CMAKE_CXX_COMPILER} ${cxx_compiler_arg1})
set(PY_LDSHARED ${link_launcher} ${CMAKE_C_COMPILER} ${c_compiler_arg1} "-shared")
if(USING_MOLD_LINKER)
set(PY_LDSHARED ${link_launcher} ${CMAKE_C_COMPILER} ${c_compiler_arg1} "-shared" "${MOLD_FUSE_LD_FLAG}")
else()
set(PY_LDSHARED ${link_launcher} ${CMAKE_C_COMPILER} ${c_compiler_arg1} "-shared")
endif()
string(REPLACE " " ";" PY_LDFLAGS "${CMAKE_SHARED_LINKER_FLAGS}")
list(APPEND PY_LDFLAGS -L${CMAKE_LIBRARY_OUTPUT_DIRECTORY})
@ -117,7 +121,11 @@ function(distutils_install_cython_module name)
get_property(compiler_launcher GLOBAL PROPERTY RULE_LAUNCH_COMPILE)
get_property(link_launcher GLOBAL PROPERTY RULE_LAUNCH_LINK)
set(PY_CC "${compiler_launcher} ${CMAKE_C_COMPILER}")
set(PY_LDSHARED "${link_launcher} ${CMAKE_C_COMPILER} -shared")
if(USING_MOLD_LINKER)
set(PY_LDSHARED "${link_launcher} ${CMAKE_C_COMPILER} -shared ${MOLD_FUSE_LD_FLAG}")
else()
set(PY_LDSHARED "${link_launcher} ${CMAKE_C_COMPILER} -shared")
endif()
set(PY_LDFLAGS "${CMAKE_SHARED_LINKER_FLAGS} -L${CMAKE_LIBRARY_OUTPUT_DIRECTORY}")
cmake_parse_arguments(DU "DISABLE_VTA" "" "" ${ARGN})
if(DU_DISABLE_VTA AND HAS_VTA)

View File

@ -1,42 +0,0 @@
# - Find libdaxctl
# Find the daxctl libraries and includes
#
# daxctl_INCLUDE_DIR - where to find libdaxctl.h etc.
# daxctl_LIBRARIES - List of libraries when using daxctl.
# daxctl_FOUND - True if daxctl found.
find_path(daxctl_INCLUDE_DIR daxctl/libdaxctl.h)
if(daxctl_INCLUDE_DIR AND EXISTS "${daxctl_INCLUDE_DIR}/libdaxctl.h")
foreach(ver "MAJOR" "MINOR" "RELEASE")
file(STRINGS "${daxctl_INCLUDE_DIR}/libdaxctl.h" daxctl_VER_${ver}_LINE
REGEX "^#define[ \t]+daxctl_VERSION_${ver}[ \t]+[0-9]+[ \t]+.*$")
string(REGEX REPLACE "^#define[ \t]+daxctl_VERSION_${ver}[ \t]+([0-9]+)[ \t]+.*$"
"\\1" daxctl_VERSION_${ver} "${daxctl_VER_${ver}_LINE}")
unset(${daxctl_VER_${ver}_LINE})
endforeach()
set(daxctl_VERSION_STRING
"${daxctl_VERSION_MAJOR}.${daxctl_VERSION_MINOR}.${daxctl_VERSION_RELEASE}")
endif()
find_library(daxctl_LIBRARY daxctl)
include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(daxctl
REQUIRED_VARS daxctl_LIBRARY daxctl_INCLUDE_DIR
VERSION_VAR daxctl_VERSION_STRING)
mark_as_advanced(daxctl_INCLUDE_DIR daxctl_LIBRARY)
if(daxctl_FOUND)
set(daxctl_INCLUDE_DIRS ${daxctl_INCLUDE_DIR})
set(daxctl_LIBRARIES ${daxctl_LIBRARY})
if(NOT (TARGET daxctl::daxctl))
add_library(daxctl::daxctl UNKNOWN IMPORTED)
set_target_properties(daxctl::daxctl PROPERTIES
INTERFACE_INCLUDE_DIRECTORIES "${daxctl_INCLUDE_DIRS}"
IMPORTED_LINK_INTERFACE_LANGUAGES "C"
IMPORTED_LOCATION "${daxctl_LIBRARIES}"
VERSION "${daxctl_VERSION_STRING}")
endif()
endif()

View File

@ -1,58 +0,0 @@
# - Find libdml
# Find the dml and dmlhl libraries and includes
#
# DML_INCLUDE_DIR - where to find dml.hpp etc.
# DML_LIBRARIES - List of libraries when using dml.
# DML_HL_LIBRARIES - List of libraries when using dmlhl.
# DML_FOUND - True if DML found.
find_path(DML_INCLUDE_DIR
dml/dml.hpp
PATHS
/usr/include
/usr/local/include)
find_library(DML_LIBRARIES NAMES dml libdml PATHS
/usr/local/
/usr/local/lib64
/usr/lib64
/usr/lib)
find_library(DML_HL_LIBRARIES NAMES dmlhl libdmlhl PATHS
/usr/local/
/usr/local/lib64
/usr/lib64
/usr/lib)
include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(dml DEFAULT_MSG
DML_LIBRARIES
DML_INCLUDE_DIR
DML_HL_LIBRARIES)
mark_as_advanced(
DML_LIBRARIES
DML_INCLUDE_DIR
DML_HL_LIBRARIES)
if(DML_FOUND)
if(NOT (TARGET dml::dml))
add_library(dml::dml UNKNOWN IMPORTED)
set_target_properties(dml::dml PROPERTIES
INTERFACE_INCLUDE_DIRECTORIES "${DML_INCLUDE_DIR}"
IMPORTED_LINK_INTERFACE_LANGUAGES "C"
IMPORTED_LOCATION "${DML_LIBRARIES}")
endif()
if(NOT (TARGET dml::dmlhl))
add_library(dml::dmlhl UNKNOWN IMPORTED)
set_target_properties(dml::dmlhl PROPERTIES
INTERFACE_INCLUDE_DIRECTORIES "${DML_INCLUDE_DIR}"
INTERFACE_LINK_LIBRARIES ${CMAKE_DL_LIBS}
INTERFACE_COMPILE_FEATURES cxx_std_17
INTERFACE_COMPILE_DEFINITIONS "DML_HW"
IMPORTED_LINK_INTERFACE_LANGUAGES "CXX"
IMPORTED_LOCATION "${DML_HL_LIBRARIES}")
endif()
endif()

View File

@ -1,42 +0,0 @@
# - Find libndctl
# Find the ndctl libraries and includes
#
# ndctl_INCLUDE_DIR - where to find libndctl.h etc.
# ndctl_LIBRARIES - List of libraries when using ndctl.
# ndctl_FOUND - True if ndctl found.
find_path(ndctl_INCLUDE_DIR ndctl/libndctl.h)
if(ndctl_INCLUDE_DIR AND EXISTS "${ndctl_INCLUDE_DIR}/libndctl.h")
foreach(ver "MAJOR" "MINOR" "RELEASE")
file(STRINGS "${ndctl_INCLUDE_DIR}/libndctl.h" ndctl_VER_${ver}_LINE
REGEX "^#define[ \t]+ndctl_VERSION_${ver}[ \t]+[0-9]+[ \t]+.*$")
string(REGEX REPLACE "^#define[ \t]+ndctl_VERSION_${ver}[ \t]+([0-9]+)[ \t]+.*$"
"\\1" ndctl_VERSION_${ver} "${ndctl_VER_${ver}_LINE}")
unset(${ndctl_VER_${ver}_LINE})
endforeach()
set(ndctl_VERSION_STRING
"${ndctl_VERSION_MAJOR}.${ndctl_VERSION_MINOR}.${ndctl_VERSION_RELEASE}")
endif()
find_library(ndctl_LIBRARY ndctl)
include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(ndctl
REQUIRED_VARS ndctl_LIBRARY ndctl_INCLUDE_DIR
VERSION_VAR ndctl_VERSION_STRING)
mark_as_advanced(ndctl_INCLUDE_DIR ndctl_LIBRARY)
if(ndctl_FOUND)
set(ndctl_INCLUDE_DIRS ${ndctl_INCLUDE_DIR})
set(ndctl_LIBRARIES ${ndctl_LIBRARY})
if(NOT (TARGET ndctl::ndctl))
add_library(ndctl::ndctl UNKNOWN IMPORTED)
set_target_properties(ndctl::ndctl PROPERTIES
INTERFACE_INCLUDE_DIRECTORIES "${ndctl_INCLUDE_DIRS}"
IMPORTED_LINK_INTERFACE_LANGUAGES "C"
IMPORTED_LOCATION "${ndctl_LIBRARIES}"
VERSION "${ndctl_VERSION_STRING}")
endif()
endif()

View File

@ -1,4 +1,4 @@
ARG FROM_IMAGE="quay.io/centos/centos:stream9"
ARG FROM_IMAGE="docker.io/rockylinux/rockylinux:10"
FROM $FROM_IMAGE
# allow FROM_IMAGE to be visible inside this stage
@ -124,7 +124,16 @@ RUN --mount=type=secret,id=prerelease_creds set -ex && \
source /run/secrets/prerelease_creds; \
REPO_URL="https://${PRERELEASE_USERNAME}:${PRERELEASE_PASSWORD}@download.ceph.com/prerelease/ceph/rpm-${CEPH_REF}/${EL_VER}/" ;\
fi && \
rpm -Uvh "$REPO_URL/noarch/ceph-release-1-${IS_RELEASE}.${EL_VER}.noarch.rpm" ; \
# Trim trailing slashes
REPO_URL="${REPO_URL%/}" && \
if [[ "$REPO_URL" == */pulp/content/* ]] ; then \
# Pulp nests packages under noarch/Packages/<first-letter>/
CEPH_RELEASE_RPM="$REPO_URL/noarch/Packages/c/ceph-release-1-${IS_RELEASE}.${EL_VER}.noarch.rpm" ; \
else \
# chacra serves packages flat under noarch/
CEPH_RELEASE_RPM="$REPO_URL/noarch/ceph-release-1-${IS_RELEASE}.${EL_VER}.noarch.rpm" ; \
fi && \
rpm -Uvh "$CEPH_RELEASE_RPM" ; \
if [[ "$IS_RELEASE" == 1 ]] ; then \
sed -i "s;http://download.ceph.com/;https://${PRERELEASE_USERNAME}:${PRERELEASE_PASSWORD}@download.ceph.com/prerelease/ceph/;" /etc/yum.repos.d/ceph.repo ; \
dnf clean expire-cache ; \

View File

@ -16,7 +16,7 @@ usage() {
$0 [containerfile] (defaults to 'Containerfile')
For a CI build (from ceph-ci.git, built and pushed to shaman):
CI_CONTAINER: must be 'true'
FROM_IMAGE: defaults to quay.io/centos/centos9:stream
FROM_IMAGE: defaults to docker.io/rockylinux/rockylinux:10
FLAVOR (OSD flavor, default or crimson)
BRANCH (of Ceph. <remote>/<ref>)
CEPH_SHA1 (of Ceph)
@ -125,7 +125,7 @@ CONTAINER_BUILD_ARGS=(
--squash
-f "$CFILE"
-t build.sh.output
--build-arg FROM_IMAGE="${FROM_IMAGE:-quay.io/centos/centos:stream9}"
--build-arg FROM_IMAGE="${FROM_IMAGE:-docker.io/rockylinux/rockylinux:10}"
--build-arg CEPH_SHA1="${CEPH_SHA1}"
--build-arg CEPH_GIT_REPO="${CEPH_GIT_REPO}"
--build-arg CEPH_REF="${BRANCH:-main}"

View File

@ -23,6 +23,7 @@ usr/bin/rgw-gap-list
usr/bin/rgw-gap-list-comparator
usr/bin/rgw-orphan-list
usr/bin/rgw-policy-check
usr/bin/rgw-policy-test
usr/bin/rgw-restore-bucket-index
usr/bin/rbd
usr/bin/rbdmap
@ -44,6 +45,7 @@ usr/share/man/man8/mount.ceph.8
usr/share/man/man8/rados.8
usr/share/man/man8/radosgw-admin.8
usr/share/man/man8/rgw-policy-check.8
usr/share/man/man8/rgw-policy-test.8
usr/share/man/man8/rgw-gap-list.8
usr/share/man/man8/rbd.8
usr/share/man/man8/rbdmap.8

View File

@ -1,5 +1,3 @@
natsort
CherryPy
packaging
requests
python-dateutil

6
debian/control vendored
View File

@ -76,13 +76,11 @@ Build-Depends: automake,
libgrpc++-dev,
protobuf-compiler-grpc,
libutf8proc-dev (>= 2.2.0),
librdkafka-dev (>= 1.1.0),
librdkafka-dev (>= 2.11) <pkg.ceph.system-rdkafka>,
libsasl2-dev <!pkg.ceph.system-rdkafka>,
libthrift-dev (>= 0.13.0),
libyaml-cpp-dev (>= 0.6),
libzstd-dev <pkg.ceph.check>,
libdaxctl-dev (>= 63) <pkg.ceph.pmdk>,
libndctl-dev (>= 63) <pkg.ceph.pmdk>,
libpmem-dev <pkg.ceph.pmdk>,
libpmemobj-dev (>= 1.8) <pkg.ceph.pmdk>,
libprotobuf-dev,
libxsimd-dev <!pkg.ceph.arrow>,

5
debian/copyright vendored
View File

@ -209,9 +209,8 @@ License: GPL-2
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
.
You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation, Inc.,
51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
.
On Debian systems, the complete text of the GNU General Public License
version 2 can be found in `/usr/share/common-licenses/GPL-2' file.

8
debian/rules vendored
View File

@ -18,6 +18,9 @@ endif
ifneq ($(filter pkg.ceph.crimson,$(DEB_BUILD_PROFILES)),)
extraopts += -DWITH_CRIMSON=ON
endif
ifneq ($(filter pkg.ceph.system-rdkafka,$(DEB_BUILD_PROFILES)),)
extraopts += -DWITH_SYSTEM_RDKAFKA=ON
endif
extraopts += -DWITH_JAEGER=ON
extraopts += -DWITH_SYSTEM_JERASURE=ON
@ -54,6 +57,11 @@ endif
ifeq ($(DWZ), false)
override_dh_dwz:
else
override_dh_dwz:
# Exclude ceph-osd-crimson, librgw, and radosgw due to excessive debug info (too many DIEs)
dh_dwz -Xceph-osd-crimson -Xlibrgw -Xradosgw
endif
# for python3-${pkg} packages

View File

@ -551,8 +551,28 @@ cephadm operations. Run a command of the following form:
ceph cephadm set-user <user>
Prior to running this, the cluster SSH key needs to be added to this user's
``authorized_keys`` file and non-root users must have passwordless sudo access.
The ``set-user`` command automatically configures the specified user on all cluster
hosts by calling ``cephadm setup-ssh-user`` on each host. This command includes the following:
- Setting up passwordless sudo access for non-root users
- Authorizing the cluster's SSH public key for the user
If you have already manually configured the user on all hosts, you can skip
the automatic setup by using the ``--skip-pre-steps`` flag:
.. prompt:: bash #
ceph cephadm set-user <user> --skip-pre-steps
For manual setup of SSH users on individual hosts, you can use the
``cephadm setup-ssh-user`` command directly:
.. prompt:: bash #
cephadm setup-ssh-user --ssh-user <user> --ssh-pub-key <public_key>
This command validates that the user exists, configures passwordless sudo access,
and authorizes the SSH public key.
Customizing the SSH Configuration
@ -668,6 +688,95 @@ when executing ``ceph * metadata``. This in turn means cephadm also
requires the bare host name when adding a host to the cluster:
``ceph orch host add <bare-name>``.
..
TODO: This chapter needs to provide way for users to configure
Grafana in the dashboard, as this is right now very hard to do.
Sudo Hardening
==============
Cephadm supports sudo hardening to enhance security by restricting sudo privilege
escalation for non-root SSH users. When sudo hardening is enabled, cephadm uses the
``cephadm_invoker.py`` script to securely execute cephadm commands with controlled
privilege escalation.
Enabling Sudo Hardening
-----------------------
To enable sudo hardening for the entire cluster, use the following command:
.. prompt:: bash #
ceph cephadm prepare-host-and-enable-sudo-hardening <user>
This command performs a comprehensive sudo hardening setup:
1. **Host Preparation**: Prepares all cluster hosts for sudo hardening by:
- Installing/upgrading cephadm RPM with the invoker script
- Configuring restricted sudo access for non-root users
- Setting up SSH key authorization
2. **SSH User Configuration**: Sets the specified user for cluster SSH operations
3. **Global Enablement**: Enables sudo hardening cluster-wide
The ``<user>`` parameter specifies which non-root user should be configured for SSH access.
This user will have restricted sudo access configured through the sudoers file.
You can manually prepare a host for sudo hardening using:
.. prompt:: bash #
cephadm prepare-host-sudo-hardening --ssh-user <user> --ssh-pub-key <pub_key>
.. note:: During initial host addition, the root user is used for setup. After
Sudo hardening is enabled, the specified non-root user with restricted sudo
access will be used for ongoing operations.
Sudo Hardening Workflow
-----------------------
When sudo hardening is enabled, the following workflow is used:
1. **Host Addition**: Before adding a new host with ``ceph orch host add``,
the cluster SSH key needs to be added to this user's ``authorized_keys``
file and non-root users must have passwordless sudo access.
2. **Command Execution**: Instead of executing cephadm directly, commands are
routed through ``cephadm_invoker.py``
3. **Binary Verification**: The invoker validates the cephadm binary's hash before execution
4. **Secure Execution**: Commands are executed with restricted permissions
The ``cephadm_invoker.py`` script provides the following subcommands:
- ``run``: Execute cephadm binary with hash verification
- ``deploy_cephadm_binary``: Deploy cephadm binary to final location
- ``check_existence``: Check if a file exists
Sudo Access Restrictions
-------------------------
Sudo hardening restricts sudo access for non-root users to enhance security. When a
host is prepared for sudo hardening, the sudoers configuration is modified to limit
the commands that can be executed with sudo. This prevents unauthorized command
execution while still allowing necessary cephadm operations.
The sudoers configuration restricts access to only the ``cephadm_invoker.py`` script
and essential system commands, providing a secure execution environment.
Security Benefits
-----------------
Sudo hardening provides the following security benefits:
1. The invoker validates the cephadm binary's hash
2. If validation fails, it signals for binary redeployment
3. Commands are executed with restricted sudo permissions
4. All operations are logged for security auditing
Disabling Sudo Hardening
------------------------
To disable sudo hardening:
.. prompt:: bash #
ceph config set mgr mgr/cephadm/sudo_hardening false
.. note:: Disabling sudo hardening does not automatically revert host configurations.
Hosts that were prepared for sudo hardening will retain the invoker setup.

View File

@ -149,6 +149,7 @@ packages that provide the ``cephadm`` command, run the following commands:
#. Add the repository:
.. prompt:: bash #
:substitutions:
./cephadm add-repo --release |stable-release|

View File

@ -393,6 +393,34 @@ For example:
``--log-dest`` options described in the bootstrap section above.
Setting Cephadm Log Verbosity
-----------------------------
The verbosity of cephadm's persistent logging (``/var/log/ceph/cephadm.log``
and syslog, when enabled) can be controlled separately from its terminal
output. Valid levels are ``info``, ``debug``, ``error``, and ``warning``.
The default is ``debug``.
When the cephadm orchestration module runs ``cephadm`` on cluster hosts, it
uses the :confval:`mgr/cephadm/cephadm_binary_logging_level` configuration
value:
.. prompt:: bash #
ceph config set mgr mgr/cephadm/cephadm_binary_logging_level info
For manual invocations, pass ``--logging-level`` on the command line:
.. prompt:: bash #
cephadm --logging-level info check-host
.. note:: The logging level applies only to persistent log destinations
(the log file and syslog). Console output during bootstrap and other
interactive use is unchanged. When you run ``cephadm`` directly, the
mgr configuration does not apply; use ``--logging-level`` as shown above.
Data Location
=============

View File

@ -806,6 +806,13 @@ For example:
ceph orch rm rgw.myrgw
The same command accepts ``--force`` and ``--force-delete-data``. Use
``ceph orch rm <service-name> --force --force-delete-data`` when you want
cephadm to remove on-disk data for supported daemon types instead of relocating
it under ``<fsid>/removed/``. The latter flag requires ``--force``. The same
pair of flags applies to ``ceph orch daemon rm <daemon-name>`` when removing
individual daemons.
.. _cephadm-spec-unmanaged:

View File

@ -8,10 +8,10 @@ Deploying mgmt-gateway
======================
In Ceph releases beginning with Tentacle, the ``mgmt-gateway`` service introduces a new design for Ceph applications
based on a modular, service-based architecture. This service, managed by cephadm and built on top of nginx
based on a modular, service-based architecture. This service, managed by cephadm and built on top of NGINX
(an open-source, high-performance web server), acts as the new front-end and single entry point to the
Ceph cluster. The ``mgmt-gateway`` provides unified access to all Ceph applications, including the Ceph dashboard
and monitoring stack. Employing nginx enhances security and simplifies access management due to its robust
and monitoring stack. Employing NGINX enhances security and simplifies access management due to its robust
community support and high-security standards. The ``mgmt-gateway`` service acts as a reverse proxy that routes
requests to the appropriate Ceph application instances.
@ -28,10 +28,10 @@ consolidated behind the new service endpoint: ``https://<node-ip>:<port>``.
Benefits of the mgmt-gateway service
====================================
* ``Unified Access``: Consolidated access through nginx improves security and provides a single entry point to services.
* ``Unified Access``: Consolidated access through NGINX improves security and provides a single entry point to services.
* ``Improved user experience``: Users no longer need to know where each application is running (IP/host).
* ``High Availability for dashboard``: nginx HA mechanisms are used to provide high availability for the Ceph dashboard.
* ``High Availability for monitoring``: nginx HA mechanisms are used to provide high availability for monitoring.
* ``High Availability for dashboard``: NGINX HA mechanisms are used to provide high availability for the Ceph dashboard.
* ``High Availability for monitoring``: NGINX HA mechanisms are used to provide high availability for monitoring.
Security enhancements
=====================
@ -42,7 +42,7 @@ Ceph dashboard.
High availability enhancements
==============================
nginx HA mechanisms are used to provide high availability for all the Ceph management applications including the Ceph dashboard
NGINX HA mechanisms are used to provide high availability for all the Ceph management applications including the Ceph dashboard
and monitoring stack. In case of the Ceph dashboard, users no longer need to know where the active manager is running.
``mgmt-gateway`` handles manager failover transparently and redirects the user to the active manager. In case of the
monitoring ``mgmt-gateway`` takes care of handling HA when several instances of Prometheus, Alertmanager or Grafana are
@ -58,7 +58,7 @@ even if certain core components for the service fail, including the ``mgmt-gatew
Multiple ``mgmt-gateway`` instances can be deployed in an active/standby configuration using keepalived
for seamless failover. The ``oauth2-proxy`` service can be deployed as multiple stateless instances,
with nginx acting as a load balancer across them using a round-robin strategy. This setup removes
with NGINX acting as a load balancer across them using a round-robin strategy. This setup removes
single points of failure and enhances the resilience of the entire system.
In this setup, the underlying internal services follow the same high availability mechanism. Instead of
@ -188,7 +188,7 @@ Limitations
Default images
~~~~~~~~~~~~~~
The ``mgmt-gateway`` service internally makes use of nginx reverse proxy. The following container image is used by default:
The ``mgmt-gateway`` service internally makes use of NGINX reverse proxy. The following container image is used by default:
::

View File

@ -463,11 +463,16 @@ To disable monitoring and remove the software that supports it, run the followin
.. prompt:: bash #
ceph orch rm grafana
ceph orch rm prometheus --force # this will delete metrics data collected so far
ceph orch rm prometheus
ceph orch rm node-exporter
ceph orch rm alertmanager
ceph mgr module disable prometheus
By default, cephadm moves Prometheus data aside under ``<fsid>/removed/`` on the
host. To delete that data instead, use
``ceph orch rm prometheus --force --force-delete-data`` in place of
``ceph orch rm prometheus`` (``--force-delete-data`` requires ``--force``).
See also :ref:`orch-rm`.
Setting up RBD-Image Monitoring

View File

@ -60,6 +60,7 @@ Alternatively, an NFS service can be applied using a YAML specification.
host2: 10.0.0.124
monitoring_networks:
- 192.168.124.0/24
enable_nfsv3: true
In this example, we run the server on the non-default ``port`` of
@ -78,6 +79,8 @@ IP address is assigned to the host, that IP address will be used. If the IP
address is not present and ``monitoring_networks`` is specified, an IP address
that matches one of the specified networks will be used. If neither condition
is met, the default binding will happen on all available network interfaces.
By default, only the NFSv4 protocol is enabled. NFSv3 can be enabled by setting
``enable_nfsv3`` to ``true`` in the service specification.
NFS over RDMA
-------------
@ -280,6 +283,27 @@ The following parameters can be used to configure TLS/SSL encryption for the NFS
If using ``inline`` certificates, all three certificate fields (``ssl_cert``,
``ssl_key``, ``ssl_ca_cert``) must be provided.
Cluster-level QoS can also be configured at deployment time by specifying
``cluster_qos_config`` and ``cluster_qos_port`` in the service spec.
See :ref:`mgr-nfs` (Cluster QoS management) for supported keys and
parameter constraints.
.. code-block:: yaml
service_type: nfs
service_id: mynfs
placement:
hosts:
- host1
spec:
port: 2049
cluster_qos_port: 31311
cluster_qos_config:
qos_type: PerShare
enable_bw_control: true
max_export_write_bw: 100MB
max_export_read_bw: 100MB
The specification can then be applied by running the following command:
.. prompt:: bash #

View File

@ -43,7 +43,7 @@ a secure and flexible authentication mechanism.
High availability
=================
In general, `oauth2-proxy` is used in conjunction with the `mgmt-gateway`. The `oauth2-proxy` service can be deployed as multiple
stateless instances, with the `mgmt-gateway` (nginx reverse-proxy) handling load balancing across these instances using a round-robin strategy.
stateless instances, with the `mgmt-gateway` (NGINX reverse-proxy) handling load balancing across these instances using a round-robin strategy.
Since oauth2-proxy integrates with an external identity provider (IDP), ensuring high availability for login is managed externally
and not the responsibility of this service.

View File

@ -53,13 +53,100 @@ An SMB service can be applied using a specification. An example in YAML follows:
include_ceph_users:
- client.smb.fs.cluster.tango
TLS/SSL Example
---------------
Here's an example SMB service specification with TLS/SSL configuration:
.. code-block:: yaml
service_id: smbcluster
service_type: smb
cluster_id: tango
config_uri: rados://smb/foxtrot/config.json
placement:
hosts:
- host0
spec:
ssl_certificates:
remote_control:
enabled: true
certificate_source: inline
ssl_cert: |
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
ssl_key: |
-----BEGIN PRIVATE KEY-----
...
-----END PRIVATE KEY-----
ssl_ca_cert: |
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
keybridge:
enabled: true
certificate_source: inline
ssl_cert: |
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
ssl_key: |
-----BEGIN PRIVATE KEY-----
...
-----END PRIVATE KEY-----
ssl_ca_cert: |
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----
This example configures an SMB service with TLS encryption enabled using
inline certificates.
TLS/SSL Parameters
~~~~~~~~~~~~~~~~~~
The following parameters can be used to configure TLS/SSL encryption per sidecar
for the SMB service:
* ``enabled`` (boolean): Enable or disable SSL/TLS encryption. Default is ``false``.
* ``certificate_source`` (string): Specifies the source of the TLS certificates.
Options include:
- ``cephadm-signed``: Use certificates signed by cephadm's internal CA
- ``inline``: Provide certificates directly in the specification using ``ssl_cert``,
``ssl_key`` and ``ssl_ca_cert`` fields
- ``reference``: Users can register their own certificate and key with certmgr and
set the ``certificate_source`` to ``reference`` in the spec.
* ``ssl_cert`` (string): The SSL certificate in PEM format. Required when using
``inline`` certificate source.
* ``ssl_key`` (string): The SSL private key in PEM format. Required when using
``inline`` certificate source.
* ``ssl_ca_cert`` (string): The SSL CA certificate in PEM format. Required when
using ``inline`` certificate source.
.. note::
``ssl_key``, ``ssl_cert`` and ``ssl_ca_cert`` can be set from the smb manager
module. If ``cert`` and ``key`` are specified in the resource_type
``ceph.smb.tls.credential`` and applied from the smb manager will be automatically
configured as ssl_certificate is enabled and update ``ssl_key``, ``ssl_cert`` to
the certificate manager. ``ssl_ca_cert`` will be set if it is specified in the
resource_type ``ceph.smb.tls.credential``
The specification can then be applied by running the following command:
.. prompt:: bash #
ceph orch apply -i smb.yaml
Service Spec Options
--------------------

View File

@ -136,7 +136,7 @@ Requirements:
(same shape as ``ceph_version_short`` in ``ceph osd metadata``).
* If the Monitors indicate to cephadm that no OSDs in the selected CRUSH bucket
are okay to upgrade, cephadm will log details and then retry the operation.
* If the bucket parameters for a ceph ``osd ok-to-upgrade`` upgrade are not provided,
* If the bucket parameters for a ceph ``osd ok-to-upgrade`` upgrade are not provided,
cephadm will fall back to the default ceph osd ok-to-stop gate for OSD upgrades.
* Bucket-scope upgrades apply only to OSDs. CRUSH buckets do not influence upgrades
of other daemon types, for example Monitors, Managers, and MDSes.

View File

@ -150,6 +150,8 @@ Filtering:
* ``--dname <string>`` only include events referring to this named dentry within a directory
fragment (may only be used in conjunction with ``--frag``)
* ``--client <int>`` only include events from this client session ID
* ``--max-rss <bytes>`` (works only with ``recover_dentries`` for now) limits the RSS of the cephfs-journal-tool
by scanning and flushing journal events in batches
Filters may be combined on an AND basis (i.e. only the intersection of events from each filter).

View File

@ -0,0 +1,360 @@
========================================
CephFS Snapshot Mirroring Checkpoints
========================================
.. contents::
:depth: 3
:local:
Introduction
============
CephFS Snapshot Mirroring Checkpoints allow you to mark specific snapshots as important
milestones and track whether they have been successfully replicated to your remote site.
This feature provides visibility into your disaster recovery readiness by showing which
critical data states have been safely mirrored.
**Use Cases:**
- **Compliance & Auditing**: Verify that end-of-day or end-of-month snapshots are replicated
- **Application Consistency**: Ensure application-consistent snapshots reach the DR site
- **Migration Tracking**: Monitor progress when migrating data between sites
- **SLA Verification**: Confirm that critical snapshots meet replication SLAs
What is a Checkpoint?
=====================
A checkpoint is a marker you place on a snapshot to track its replication status. When you
checkpoint a snapshot, the system monitors whether that snapshot (and all previous snapshots)
have been successfully synchronized to the remote filesystem.
**Key Concepts:**
- **Automatic Status Tracking**: Checkpoints automatically detect if a snapshot has already been synced
- **Persistent**: Checkpoint information survives daemon restarts
- **Per-Directory**: Each mirrored directory has its own set of checkpoints
Checkpoint Lifecycle
--------------------
Every checkpoint goes through these states:
.. code-block:: text
┌─────────┐
│ CREATED │ ← Checkpoint added, snapshot not yet synced
└────┬────┘
├──→ ┌──────────┐
│ │ COMPLETE │ ← Snapshot successfully synced to remote
│ └──────────┘
└──→ ┌────────┐
│ FAILED │ ← Sync failed (will be retried automatically)
└────────┘
- **created**: Waiting for synchronization or sync in progress
- **complete**: Successfully replicated to remote site
- **failed**: Sync encountered an error (check error message for details)
Prerequisites
=============
Before using checkpoints, ensure:
1. **CephFS Mirroring is Enabled**:
.. code-block:: bash
ceph fs snapshot mirror enable <fs_name>
2. **Remote Peer is Configured**:
.. code-block:: bash
ceph fs snapshot mirror peer_add <fs_name> <peer_spec>
3. **Directory is Being Mirrored**:
.. code-block:: bash
ceph fs snapshot mirror add <fs_name> <dir_path>
4. **Snapshots Exist**: You need snapshots to checkpoint
For detailed mirroring setup, see :doc:`cephfs-mirroring`.
Getting Started
===============
Basic Workflow
--------------
Here's a typical workflow for using checkpoints:
**Step 1: Create Snapshots**
.. code-block:: bash
# Create snapshots in your mirrored directory
mkdir /mnt/cephfs/data/.snap/daily_backup_2026_06_01
mkdir /mnt/cephfs/data/.snap/daily_backup_2026_06_02
mkdir /mnt/cephfs/data/.snap/daily_backup_2026_06_03
**Step 2: Add Checkpoints**
.. code-block:: bash
# Mark important snapshots as checkpoints
ceph fs snapshot mirror checkpoint add myfs /data daily_backup_2026_06_01
ceph fs snapshot mirror checkpoint add myfs /data daily_backup_2026_06_03
**Step 3: Monitor Status**
.. code-block:: bash
# Check checkpoint status
ceph fs snapshot mirror checkpoint ls myfs /data --format json-pretty
**Example Output:**
.. code-block:: json
{
"dir_root": "/data",
"checkpoints": [
{
"snap_id": 100,
"snap_name": "daily_backup_2026_06_01",
"status": "complete",
"created_at": "2026-06-01T23:00:00.000000+0000",
"updated_at": "2026-06-01T23:15:00.000000+0000"
},
{
"snap_id": 102,
"snap_name": "daily_backup_2026_06_03",
"status": "created",
"created_at": "2026-06-03T23:00:00.000000+0000",
"updated_at": "2026-06-03T23:00:00.000000+0000"
}
]
}
Command Reference
=================
checkpoint add
--------------
Add a checkpoint to a specific snapshot.
**Syntax:**
.. code-block:: bash
ceph fs snapshot mirror checkpoint add <fs_name> <dir_path> <snap_name>
**Parameters:**
- ``fs_name``: Name of the filesystem
- ``dir_path``: Path to the mirrored directory (e.g., ``/data``)
- ``snap_name``: Name of the snapshot to checkpoint
**Example:**
.. code-block:: bash
ceph fs snapshot mirror checkpoint add myfs /data backup_snapshot_1
**Output:**
.. code-block:: json
{
"status": "success",
"message": "checkpoint added for snapshot backup_snapshot_1",
"dir_root": "/data",
"snap_id": 123,
"snap_name": "backup_snapshot_1",
"checkpoint_status": "created"
}
**Notes:**
- If the snapshot has already been synced, status will be ``complete`` immediately
- You cannot checkpoint the same snapshot twice
- The snapshot must exist in the directory
checkpoint now
--------------
Add a checkpoint to the most recent snapshot in a directory.
**Syntax:**
.. code-block:: bash
ceph fs snapshot mirror checkpoint now <fs_name> <dir_path>
**Example:**
.. code-block:: bash
ceph fs snapshot mirror checkpoint now myfs /data
**Output:**
.. code-block:: json
{
"status": "success",
"message": "checkpoint created on latest snapshot",
"dir_root": "/data",
"snap_id": 125,
"snap_name": "daily_backup_2026_06_03",
"checkpoint_status": "created"
}
**Use Case:**
Useful when you want to checkpoint the latest state without knowing the exact snapshot name.
checkpoint ls
-------------
List all checkpoints for a directory.
**Syntax:**
.. code-block:: bash
ceph fs snapshot mirror checkpoint ls <fs_name> <dir_path> [--format <format>]
**Parameters:**
- ``format``: Output format (``json`` or ``json-pretty``), default is ``json``
**Example:**
.. code-block:: bash
ceph fs snapshot mirror checkpoint ls myfs /data --format json-pretty
**Output Fields:**
- ``snap_id``: Internal snapshot ID
- ``snap_name``: Snapshot name
- ``status``: Current status (created/complete/failed)
- ``created_at``: When checkpoint was created
- ``updated_at``: Last status update time
- ``error_msg``: Error description (only present if status is failed)
checkpoint remove
-----------------
Remove a checkpoint from a specific snapshot.
**Syntax:**
.. code-block:: bash
ceph fs snapshot mirror checkpoint remove <fs_name> <dir_path> <snap_name>
**Example:**
.. code-block:: bash
# Remove checkpoint from a specific snapshot
ceph fs snapshot mirror checkpoint remove myfs /data backup_snapshot_1
**Output:**
.. code-block:: json
{
"status": "success",
"message": "checkpoint removed for snapshot backup_snapshot_1",
"dir_root": "/data",
"snap_name": "backup_snapshot_1"
}
**Notes:**
- Removing a checkpoint does NOT delete the snapshot itself
- The snapshot and its data remain intact
- This operation only removes the checkpoint tracking metadata
- This operation cannot be undone
- To remove multiple checkpoints, run the command multiple times
Important Notes
===============
Snapshot Deletion Behavior
---------------------------
.. warning::
**If a snapshot with a checkpoint is deleted, the checkpoint metadata is permanently lost.**
When you delete a snapshot that has a checkpoint:
1. **Before Sync Completes**: The checkpoint is lost and will never reach "complete" status
2. **After Sync Completes**: The checkpoint history is lost, but the data was already replicated
**Impact on Commands:**
- ``checkpoint ls``: Will not show the deleted snapshot's checkpoint
- ``checkpoint remove``: Will fail with "snapshot not found" error
**Best Practice:**
- Do not delete snapshots with active checkpoints until they reach "complete" status
- Use ``checkpoint ls`` to verify checkpoint status before deleting snapshots
- Consider removing checkpoints explicitly before deleting important snapshots
Snapshot Rename Behavior
-------------------------
.. note::
**Checkpoints survive snapshot renames, but you must use the new snapshot name in commands.**
When you rename a snapshot that has a checkpoint:
1. **Checkpoint Persists**: The checkpoint metadata remains attached to the snapshot
2. **Name Updates Automatically**: ``checkpoint ls`` will show the new snapshot name
3. **Old Name No Longer Works**: You must use the new name for ``checkpoint remove``
**Example:**
.. code-block:: bash
# Original snapshot with checkpoint
ceph fs snapshot mirror checkpoint add myfs /data old_name
# Rename the snapshot (using CephFS snapshot rename)
# ... snapshot renamed from 'old_name' to 'new_name' ...
# List checkpoints - shows new name
ceph fs snapshot mirror checkpoint ls myfs /data
# Output shows: "snap_name": "new_name"
# Remove using NEW name (this works)
ceph fs snapshot mirror checkpoint remove myfs /data new_name
# Remove using OLD name (this fails)
ceph fs snapshot mirror checkpoint remove myfs /data old_name
# Error: snapshot 'old_name' not found
**Best Practice:**
- Always use ``checkpoint ls`` to verify current snapshot names before removing checkpoints
- Update any automation scripts if you rename snapshots
Additional Resources
====================
- **CephFS Mirroring Guide**: :doc:`cephfs-mirroring`
- **Snapshot Documentation**: :doc:`/cephfs/snapshots`
- **Tracker Issue**: https://tracker.ceph.com/issues/73454

File diff suppressed because it is too large Load Diff

View File

@ -15,6 +15,10 @@ Key features include:
* Configurable block sizes, file counts, and fsync intervals.
* Detailed statistical reporting (Mean, Std Dev, Min/Max) for throughput and IOPS.
* Support for specific CephFS user/group impersonation (UID/GID) via ``ceph_mount_perms_set``.
* Optional JSON output and libcephfs perf counter dumps.
* Optional progress reporting during benchmark phases.
* Optional asynchronous I/O benchmarking with configurable queue depth.
* Optional overrides for selected client configuration settings such as object cache and messenger worker threads.
Building
========
@ -60,7 +64,7 @@ General Options
Path to keyring file
.. option:: --filesystem <name>
.. option:: --filesystem, --fs <name>
CephFS filesystem name to mount
@ -72,6 +76,19 @@ General Options
Group ID to mount as (default: ``-1``)
.. option:: --client-oc <0|1>
Override the ``client_oc`` setting for the benchmark mount
.. option:: --client-oc-size <value>
Override the ``client_oc_size`` setting for the benchmark mount
.. option:: --msgr-workers <n>
Override ``ms_async_op_threads`` for the benchmark mount. Valid values are
``1`` to ``24``; ``0`` keeps the configured default.
Benchmark Options
-----------------
@ -121,6 +138,40 @@ These options are used with the ``bench`` command.
Disable cleanup of files
.. option:: --json <path>
Write structured benchmark results to a JSON file. Use ``-`` to output to stdout.
.. option:: --duration <seconds>
Limit each write and read phase to ``N`` seconds. A value of ``0`` means the
benchmark processes all files.
.. option:: --perf-dump <path>
Dump libcephfs performance counters to the specified file after the benchmark
.. option:: --progress
Show live progress, bandwidth, file rate, and ETA during benchmark phases
.. option:: --progress-interval <percent>
Minimum percentage increment between progress updates (default: ``10``;
valid range: ``1`` to ``100``)
.. option:: --async-io
Use asynchronous I/O via ``ceph_ll_nonblocking_readv_writev`` for read and
write phases
.. option:: --queue-depth <n>
Maximum number of outstanding async I/Os per worker thread (default:
``16``; valid range: ``1`` to ``1024``). This option is only meaningful with
``--async-io`` and ``--async-io`` cannot be combined with
``--per-thread-mount``.
Examples
========

View File

@ -65,6 +65,18 @@ InoTables of each ``in`` MDS rank, to indicate that any written inodes' numbers
are now in use. In simple cases, this will result in an entirely valid backing
store state.
.. warning::
Large journal sizes can cause the tool's Resident Set Size (RSS) to spike
significantly. Use the ``--max-rss <bytes>`` flag to limit memory usage, keeping the following
in mind:
* **Performance Impact:** Do not set this value too low. Restricting memory excessively will
severely degrade runtime performance and prolong the recovery process.
* **30% Buffer Rule:** Maintain a safety buffer of at least 30% relative to the tool's total
memory budget. For example, if 100 GiB can be allotted to the tool, cap its RSS limit at
70 GiB: ``--max-rss 75161927680``.
.. warning::
The resulting state of the backing store is not guaranteed to be

View File

@ -85,6 +85,18 @@ Using first-damage.py
cephfs-journal-tool --rank=<fs_name>:0 event recover_dentries summary
.. warning::
Large journal sizes can cause the tool's Resident Set Size (RSS) to
spike significantly. To prevent excessive memory consumption, use the
``--max-rss <bytes>`` flag to cap the RSS.
Avoid setting this value too low, as it will severely degrade runtime
performance and prolong recovery. It is best practice to maintain a memory
buffer of at least 30% of the tool's total budget. For example, if 100 GiB
is to be allotted to the tool, cap the RSS at 70 GiB
(``--max-rss 75161927680``).
#. Reset the journal:
.. prompt:: bash #

View File

@ -24,8 +24,8 @@ failures within it are unlikely to make non-inlined data inaccessible.
Inline data has always been off by default and requires setting
the ``inline_data`` flag.
Inline data has been declared deprecated for the Octopus release, and will
likely be removed altogether in a future release.
Inline data has been deprecated since the Octopus release. Enabling it triggers
a health warning, and the feature will be removed altogether in a future release.
Mantle: Programmable Metadata Load Balancer
-------------------------------------------

View File

@ -476,6 +476,15 @@ This command resizes the subvolume quota, using the size specified by
``new_size``. The ``--no_shrink`` flag prevents the subvolume from shrinking
below the current "used size" of the subvolume.
Resizing can also be done using human-friendly units::
ceph fs subvolume resize foo subvol1 100KiB
ceph fs subvolume resize foo subvol1 200.45KiB
ceph fs subvolume resize foo subvol1 300KB
.. note:: Values will be strictly cast to IEC units even when SI units
are input, i.e. 1{K|KB|Ki|KiB} all translate to 1024 bytes.
The subvolume can be resized to an unlimited (but sparse) logical size by
passing ``inf`` or ``infinite`` as ``<new_size>``.

View File

@ -93,6 +93,7 @@ Administration
cephfs-tool <cephfs-tool>
Scheduled Snapshots <snap-schedule>
CephFS Snapshot Mirroring <cephfs-mirroring>
CephFS Snapshot Mirroring Checkpoints <cephfs-mirroring-checkpoints>
Purge Queue <purge-queue>
.. raw:: html

View File

@ -8,9 +8,9 @@ in the kernel driver.
Inline data
-----------
Inline data was introduced by the Firefly release. This feature is being
deprecated in mainline CephFS, and may be removed from a future kernel
release.
Inline data was introduced by the Firefly release. This feature has been
deprecated since the Octopus release: enabling it triggers a health warning,
and it will be removed in a future release.
Linux kernel clients >= 3.19 can read inline data and convert existing
inline data to RADOS objects when file data is modified. At present,

View File

@ -14,7 +14,7 @@ Enable LazyIO
LazyIO can be enabled in the following ways.
- ``client_force_lazyio`` option enables LAZY_IO globally for libcephfs and
ceph-fuse mount.
ceph-fuse mount. This is strictly a startup flag.
- ``ceph_lazyio(...)`` and ``ceph_ll_lazyio(...)`` enable LAZY_IO for file handle
in libcephfs.

View File

@ -68,3 +68,7 @@
.. confval:: mds_symlink_recovery
.. confval:: mds_extraordinary_events_dump_interval
.. confval:: subv_metrics_window_interval
.. confval:: mds_dmclock_enable
.. confval:: mds_dmclock_reservation
.. confval:: mds_dmclock_weight
.. confval:: mds_dmclock_limit

View File

@ -54,37 +54,3 @@ command. Older versions of Ceph require you to stop these daemons manually.
ceph fs set <fs_name> max_mds <old_max_mds>
ceph fs set <fs_name> allow_standby_replay <old_allow_standby_replay>
Upgrading pre-Firefly file systems past Jewel
=============================================
.. tip::
This advice only applies to users with file systems
created using versions of Ceph older than *Firefly* (0.80).
Users creating new file systems may disregard this advice.
Pre-firefly versions of Ceph used a now-deprecated format
for storing CephFS directory objects, called TMAPs. Support
for reading these in RADOS will be removed after the Jewel
release of Ceph, so for upgrading CephFS users it is important
to ensure that any old directory objects have been converted.
After installing Jewel on all your MDS and OSD servers, and restarting
the services, run the following command:
::
cephfs-data-scan tmap_upgrade <metadata pool name>
This only needs to be run once, and it is not necessary to
stop any other services while it runs. The command may take some
time to execute, as it iterates over all objects in your metadata
pool. It is safe to continue using your file system as normal while
it executes. If the command aborts for any reason, it is safe
to simply run it again.
If you are upgrading a pre-Firefly CephFS file system to a newer Ceph version
than Jewel, you must first upgrade to Jewel and run the ``tmap_upgrade``
command before completing your upgrade to the latest version.

View File

@ -165,9 +165,6 @@ else:
'engine': 'ditaa'
}
if build_with_rtd:
extensions += ['sphinx_search.extension']
# sphinx.ext.todo options
todo_include_todos = True

View File

@ -0,0 +1,58 @@
Cephadm Invoker
===============
The ``cephadm_invoker.py`` script provides a wrapper intended for executing
cephadm commands with limited sudo priviliges. It is used when sudo hardening
is enabled.
Overview
--------
The cephadm invoker validates the cephadm binary hash before execution and
provides a secure way to run cephadm commands and deploy binaries. It is installed as
part of the cephadm RPM at ``/usr/libexec/cephadm_invoker.py``.
Commands
--------
The invoker supports the following subcommands:
``run``
Execute cephadm binary with arguments after hash verification::
cephadm_invoker.py run <binary> [args...]
The binary path must include a hash in the filename for verification.
``deploy_binary``
Deploy cephadm binary from a temporary file to the final location::
cephadm_invoker.py deploy_binary <temp_file> <final_path>
``check_binary``
Check if a cephadm binary exists::
cephadm_invoker.py check_binary <cephadm_binary_path>
Returns exit code 0 if the file exists, 2 if it does not exist.
Exit Codes
----------
- ``0``: Success
- ``1``: General error (file not found, permission issues, etc.)
- ``2``: Binary hash mismatch or file doesn't exist (triggers redeployment)
- ``126``: Permission denied during execution
Security Features
-----------------
The cephadm invoker provides the following security features:
- **Binary Hash Verification**: Validates the cephadm binary integrity before execution
- **Restricted Execution**: Only allows execution of verified cephadm binaries
- **Secure Deployment**: Safely deploys cephadm binaries with proper permissions
- **Logging**: Comprehensive logging to both console and syslog for audit trails

View File

@ -11,5 +11,6 @@ CEPHADM Developer Documentation
developing-cephadm
host-maintenance
compliance-check
cephadm-invoker
Storage devices and OSDs management <./design/storage_devices_and_osds>
scalability-notes

View File

@ -326,8 +326,48 @@ For example:
]
One entry per mirror-daemon instance is displayed, along with information
including configured peers and basic statistics. For more detailed statistics,
use the admin socket interface as detailed below.
including configured peers and basic statistics.
**Directory snapshot sync metrics (mgr)**
The mirroring module implements ``ceph fs snapshot mirror status``. The
``cephfs-mirror`` daemon persists per-directory sync statistics to the
``cephfs_mirror`` object omap in the metadata pool so the Manager can expose
them through the Ceph CLI without an admin socket on a mirror daemon host. The
``metrics_status`` handler reads that omap, applies stale detection and default
idle metrics for newly mirrored directories, and returns JSON in the same nested
``metrics/<dir>/peer/<uuid>`` format as ``fs mirror peer status``. When
``snapshot_mirror_metrics_cache_enabled`` is true (default TTL 15 seconds via
``snapshot_mirror_metrics_cache_ttl``), responses are served from a complete
cache (full file system snapshot) and, for single-directory queries, a partial
per-directory cache that avoids full omap scans on complete-cache miss. When
caching is disabled, each query reads omap directly. Omap entries are marked
``stale`` when the persisted ``_instance_id`` is not among live mirror instances
or does not match the directory's tracked instance while persisted state is not
``idle``.
Each omap value includes metadata fields written by ``cephfs-mirror``:
- ``_instance_id`` — RADOS client instance of the writer; used for stale
detection and stripped from CLI output.
- ``metrics_updated_at`` — wall-clock time of the last omap write. Exposed in
``ceph fs snapshot mirror status`` output only; omitted from admin socket
``fs mirror peer status``, which reads live in-memory state and does not
need a separate persist timestamp. Operators use this field to judge how
fresh omap-backed metrics are, given ``cephfs_mirror_tick_interval`` omap
persist cadence and Manager caching (``snapshot_mirror_metrics_cache_enabled``
and ``snapshot_mirror_metrics_cache_ttl``).
Omap entries are removed when a directory is removed from mirroring. All metric
fields are written to omap; on daemon restart only ``last_synced_snap`` metadata
is loaded back. Per-session counters (``snaps_synced``, ``snaps_deleted``,
``snaps_renamed``) are persisted but not loaded and therefore start at zero each
session.
See :ref:`Directory snapshot sync metrics<cephfs_mirroring_mgr_snapshot_status>`
and :ref:`Snapshot sync metric fields<cephfs_mirroring_sync_metric_fields>` in
:doc:`/cephfs/cephfs-mirroring` for command syntax, examples, and operator
guidance.
CephFS mirror daemons provide admin socket commands for querying mirror status.
To list the available commands for ``mirror status``, run the following
@ -393,20 +433,35 @@ status. Commands of this kind take the form ``filesystem-name@filesystem-id peer
::
{
"/d0": {
"state": "idle",
"last_synced_snap": {
"id": 120,
"name": "snap1",
"sync_duration": 0.079997898999999997,
"sync_time_stamp": "274900.558797s"
},
"snaps_synced": 2,
"snaps_deleted": 0,
"snaps_renamed": 0
"metrics": {
"/d0": {
"peer": {
"a2dc7784-e7a1-4723-b103-03ee8d8768f8": {
"state": "idle",
"last_synced_snap": {
"id": 120,
"name": "snap1",
"crawl_duration": "2s",
"datasync_queue_wait_duration": "1s",
"sync_duration": "33s",
"sync_time_stamp": "2026-07-15T12:00:00.558797+0530",
"sync_bytes": "149.94 MiB",
"sync_files": 5000
},
"snaps_synced": 2,
"snaps_deleted": 0,
"snaps_renamed": 0
}
}
}
}
}
Several fields in the status output are formatted for readability rather than reported as raw
numbers. See :ref:`Value formatting <cephfs_mirror_peer_status_formatting>` in
:doc:`/cephfs/cephfs-mirroring` for duration, data size, throughput, percentage, count, and
timestamp formats.
Synchronization stats such as ``snaps_synced``, ``snaps_deleted`` and
``snaps_renamed`` are reset when the daemon is restarted or (when multiple
mirror daemons are deployed), when a directory is reassigned to another mirror
@ -419,6 +474,49 @@ A directory can be in one of the following states::
- `syncing`: The directory is currently being synchronized
- `failed`: The directory has hit upper limit of consecutive failures
::
{
"metrics": {
"/d0": {
"peer": {
"a2dc7784-e7a1-4723-b103-03ee8d8768f8": {
"state": "syncing",
"current_syncing_snap": {
"id": 121,
"name": "snap2",
"sync-mode": "full",
"avg_read_throughput_bytes": "13.03 MiB/s",
"avg_write_throughput_bytes": "24.24 MiB/s",
"crawl": {
"state": "completed",
"duration": "2s"
},
"datasync_queue_wait": {
"state": "complete",
"duration": "1s"
},
"bytes": {
"sync_bytes": "60.40 MiB",
"total_bytes": "149.94 MiB",
"sync_percent": "40.29%"
},
"files": {
"sync_files": 2013,
"total_files": 5000,
"sync_percent": "40.26%"
},
"eta": "7s"
},
"snaps_synced": 2,
"snaps_deleted": 0,
"snaps_renamed": 0
}
}
}
}
}
When a directory hits a configured number of consecutive synchronization
failures, the mirror daemon marks it as ``failed``. Synchronization for these
directories is retried. By default, the number of consecutive failures before a
@ -439,23 +537,37 @@ status:
::
{
"/d0": {
"state": "idle",
"last_synced_snap": {
"id": 120,
"name": "snap1",
"sync_duration": 0.079997898999999997,
"sync_time_stamp": "274900.558797s"
"metrics": {
"/d0": {
"peer": {
"a2dc7784-e7a1-4723-b103-03ee8d8768f8": {
"state": "idle",
"last_synced_snap": {
"id": 120,
"name": "snap1",
"crawl_duration": "2s",
"datasync_queue_wait_duration": "1s",
"sync_duration": "33s",
"sync_time_stamp": "2026-07-15T12:00:00.558797+0530",
"sync_bytes": "149.94 MiB",
"sync_files": 5000
},
"snaps_synced": 2,
"snaps_deleted": 0,
"snaps_renamed": 0
}
}
},
"snaps_synced": 2,
"snaps_deleted": 0,
"snaps_renamed": 0
},
"/f0": {
"state": "failed",
"snaps_synced": 0,
"snaps_deleted": 0,
"snaps_renamed": 0
"/f0": {
"peer": {
"a2dc7784-e7a1-4723-b103-03ee8d8768f8": {
"state": "failed",
"snaps_synced": 0,
"snaps_deleted": 0,
"snaps_renamed": 0
}
}
}
}
}

View File

@ -94,6 +94,33 @@ Shaman
for its `Web UI`_. But please note, shaman does not build the
packages, it just offers information on the builds.
ceph-perf-pull-requests
-----------------------
``ceph-perf-pull-requests`` runs CBT performance regression checks on dedicated
``performance`` Jenkins agents. The job definition lives in `ceph-build`_ and
generates two jobs from a single template: ``ceph-perf-classic`` and
``ceph-perf-crimson``.
A pull request can trigger a run with::
jenkins test classic perf
jenkins test crimson perf
The ``performance`` label is also whitelisted for automatic runs. Each job:
#. checks out ``ceph-main`` (``origin/main``) and the PR merge ref
#. builds both trees (classic ``vstart-base`` or Crimson ``crimson-osd``)
#. runs the checked-in ``radosbench_4K_read.yaml`` workload via ``run-cbt.sh``
#. compares PR results against ``main`` with ``cbt/compare.py``
#. publishes a GitHub check named ``perf-test-{classic,crimson}``
The benchmark YAML is always taken from ``ceph-main`` so PR and baseline runs use
the same workload definition. Teuthology-to-CBT translation is handled by
``src/test/crimson/cbt/t2c.py`` in the Ceph tree (not patched at build time).
.. _ceph-build: https://github.com/ceph/ceph-build
As the following shows, `chacra`_ manages multiple projects whose metadata
are stored in a database. These metadata are exposed via Shaman as a web
service. `chacractl`_ is a utility to interact with the `chacra`_ service.
@ -179,9 +206,7 @@ libzbd
packages `libzbd`_ . The upstream libzbd includes debian packaging already.
libpmem
packages `pmdk`_ . Please note, ``ndctl`` is one of the build dependencies of
pmdk, for an updated debian packaging, please see
https://github.com/ceph/ceph-ndctl .
packages `pmdk`_ .
.. note::

View File

@ -101,7 +101,7 @@ run::
$ MDS=0 MON=1 OSD=1 MGR=1 taskset -ac '0-95' /ceph/src/vstart.sh --new -x \
--localhost --without-dashboard --redirect-output --seastore --osd-args \
"--seastore_max_concurrent_transactions=128 --seastore_cachepin_type=LRU \
--seastore_main_device_type=RANDOM_BLOCK_SSD" --seastore-devs /dev/nvme0n1 \
--seastore_hot_device_type=RANDOM_BLOCK_SSD" --seastore-devs /dev/nvme0n1 \
--crimson --crimson-smp 1 --no-restart
Another SeaStore example::
@ -253,7 +253,17 @@ In order to use ``fio`` to test ``crimson-store-nbd``, perform the below steps.
CBT
---
We can use `cbt`_ for performance tests::
We can use `cbt`_ for performance tests. Benchmark workloads are checked in under
``src/test/crimson/cbt/`` as teuthology-style YAML files. Before ``run-cbt.sh``
invokes CBT, ``t2c.py`` translates the teuthology ``tasks`` list into a
CBT-ready configuration: it extracts the ``cbt`` task, fills in cluster paths
(``ceph.conf``, ``ceph``/``rados`` binaries, PID directory), and writes the
result to a temporary YAML file consumed by ``cbt.py``.
Unit tests for the translator live in ``src/test/crimson/cbt/test_t2c.py`` and
are registered with ``make check`` via ``add_ceph_test``.
::
$ git checkout main
$ make crimson-osd
@ -410,3 +420,5 @@ Code Walkthroughs
BackfillMachine <backfillmachine>
SeaStore <seastore>
PoseidonStore <poseidonstore>
PG Merge Synchronization <pgmerging>
The Logical Address in SeaStore <seastore_laddr>

View File

@ -0,0 +1,106 @@
===========================================
PG Merge Synchronization in Crimson
===========================================
This document describes the infrastructure used to coordinate the merging
of Placement Groups (PGs) in Crimson.
Background
----------
Crimson utilizes a shared-nothing memory model where each PG is owned by
a specific CPU core. Merging requires cross-core coordination while
maintaining memory safety and epoch consistency.
Core Concepts
-------------
.. _migration_safety:
Migration Safety (Memory Ownership)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PGs are managed by their ``birth_shard``. To prevent cross-shard
deallocation crashes:
1. **Extraction**: ``ShardServices::extract_pg()`` removes the source PG
from its shard map so it stops receiving messages.
2. **Transportation**: The PG crosses cores in a ``seastar::foreign_ptr``.
3. **Storage**: The target shard stores each source in
``crimson::local_shared_foreign_ptr<Ref<PG>>`` inside the target PG's
rendezvous map. When the target drops the last reference after
``PG::merge_from()``, destruction is routed back to the source
``birth_shard``.
.. _synchronization:
Target and Source PG Synchronization
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Merging involves a race between source PGs checking in and the target PG
reaching the merge point in ``PGAdvanceMap``.
Synchronization state lives on the **target PG**, not in a per-shard
registry:
* **``merge_rendezvous_t``**: A map of arrived sources plus a
``seastar::semaphore`` counting first-time registrations.
* **``PG::add_merge_source()``**: Called on the target's shard when
``ShardServices::register_merge_source()`` delivers a source PG.
Duplicate source pgids are ignored (idempotent under replay).
* **``PG::collect_merge_sources(n)``**: Waits on the rendezvous semaphore
for ``n`` arrivals (one signal per first insert), asserts the map has
``n`` entries, then returns sources and clears the rendezvous. If
``reset_merge_rendezvous()`` breaks the wait, returns an empty map.
* **``PG::reset_merge_rendezvous()``**: Clears in-flight handoffs and
resets the semaphore. Called on ``PG::stop()`` or after a Seastore
cross-shard abort so a failed try cannot leave stale sources for a
later epoch.
**Cross-shard handoff**: ``register_merge_source()`` resolves the target
shard, extracts the source locally, and uses ``invoke_on`` to call
``add_merge_source()`` on the target PG.
.. _map_consistency:
Map Advancement and Pipeline Stalling
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To ensure the merge occurs between PGs at the same logical time:
* ``PGAdvanceMap::check_for_merges()`` detects ``pg_num`` shrink between
map epochs and returns a ``merge_result_t`` (source, target, or none).
* ``merge_pg()`` performs merge-specific work only (Seastore eligibility,
rendezvous collection, ``PG::merge_from()`` on the target).
* A single ``PGAdvanceMap`` operation may batch multiple map epochs
(``from`` through ``to``). Only a **merge source** stops the epoch
loop early: ``finish_merge_source()`` commits ``rctx``, stops the PG,
and calls ``register_merge_source()``. A **merge target** runs
``merge_from()`` at the merge epoch, then continues advancing through
the remaining epochs up to ``to`` (matching classic OSD behavior).
Normal completion at the end of ``start()`` activates the final map
and commits ``rctx`` (also covering a completed merge target).
* The target does not call ``merge_from()`` until sources frozen at the
merge epoch have arrived on its rendezvous.
.. _seastore_cross_shard:
Seastore Cross-Shard Merges
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Seastore does not support merging collections across reactor shards.
Before a source hands off or a target collects sources, ``merge_pg()``
calls ``ShardServices::seastore_merge_shards_ok()`` with the full sibling
set. If the target and any source map to different shards, the OSD aborts
the attempt: clears ready-to-merge state, sends
``MOSDPGReadyToMerge{ ready=false }`` for the source so the monitor does
not commit the unsafe ``pg_num`` decrement, resets the target rendezvous,
and sends ``MOSDPGStopMerge`` to the monitor (once per pool per OSD).
The monitor clamps ``pg_num_target`` to ``pg_num`` and clears
``FLAG_CRIMSON_ALLOW_PG_MERGE`` for that pool until operators re-enable
merging (``crimson_allow_pg_merge``).
Cleanup
-------
After ``complete_rctx()``, the target releases its ``local_shared_foreign_ptr``
references; source PG objects are destroyed on their ``birth_shard``.
When the OSD finishes consuming a new OSDMap in ``OSD::committed_osd_maps()``,
``OSDSingletonState::prune_sent_ready_to_merge()`` drops ``sent_ready_to_merge_source``
entries (and stale ``pools_merge_stopped_reported`` pool ids) for PGs that no
longer exist in the committed map, mirroring classic ``OSD::consume_map()``.

View File

@ -204,7 +204,7 @@ Metadata Structures
Device Types
------------
Configured via the ``seastore_main_device_type`` option, the device types are
Configured via the ``seastore_hot_device_type`` option, the device types are
separated into **Segmented** and **RBM** backend types as follows:

View File

@ -201,7 +201,8 @@ after the upgrade finishes.
TODO: fsck/tooling must support scanning and migrating all mappings
between upgrades.
//create tracker after PR merged/ready.
//create tracker after PR merged/ready.
Multi-push Recovery
-------------------

View File

@ -0,0 +1,85 @@
=====================================
The Logical Bucket Cache in SeaStore
=====================================
This document introduces the LogicalBucketCache machinery which is
mainly for using SSDs as non-volatile cache for the relatively slow
HDD devices. Specifically, it serves to:
#. Load frequently accessed data from the cold tier to the Logical
Bucket Cache.
#. Evict relatively cold data from the Logical Bucket Cache to the
cold tier based on the heat of both data read and write.
The Logical Bucket Cache is enabled when there is at least one cold
tier.
General Approach
================
#. Cache line: a range of continuous laddr address space, which we
call logical bucket. For now, that continuous laddr range is the
scope of a RADOS object.
#. Extents are loaded to the cache device when: 1) they are newly
written by clients and don't trigger the write_through machinery.
For now, the write_through machinery is triggered when the newly
written data is larger than a specific threshold; 2) they are on
the slow cold device and read to or evicted from the in-memory Cache.
#. Extents are evicted from the cache device in units of logical buckets,
that is, extents of the same logical bucket are evicted from the cache
device altogether.
#. The process that write extents evicted from Cache down to the cache
device is called promotion.
#. The process that evict extents from the cache device to the cold device
is called demotion.
#. Extents that are loaded from the cold tier to the Logical Bucket Cache
are NOT removed from the cold tier, which means they'd have two paddrs,
one in the Logical Bucket Cache, the other in the cold tier, and the
promoted data is stored twice. This is designed to make the demote process
cheap. We call paddrs corresponding to the cold tier shadow paddrs.
So lba mappings have two paddrs now, the primary paddr and the shadow
paddr.
Promotion
=========
Promotion is the process that load the extents in the cold tier to the
Logical Bucket Cache.
Extents that are read from the cold tier are first added to the in-memory
cache; when evicted from the in-memory cache, they are added to the Promoter.
The Promoter write its extents down to the Logical Bucket Cache when the size
of its extents grows larger than ``seastore_cache_promotion_size``.
Demotion
========
Demotion happens when:
#. The usage of the Logical Bucket Cache reaches ``seastore_multiple_tiers_fast_evict_ratio``.
#. The in-memory data index of the Logical Bucket Cache reaches ``seastore_logical_bucket_capacity``.
Demoting extents that are live in both the Logical Bucket Cache and the
cold tier only involves setting the shadow paddr to the primary paddr and
removing the shadow paddr itself.
Demoting extents that are only live in the Logical Bucket Cache would involve
rewriting the extents to the cold tier.
WriteThrough
============
When the newly written data is larger than ``seastore_write_through_size``,
it would be written directly to the cold tier.
Internal Test Workload
=======================
To test the correctness of the whole Logical Bucket Cache machinery, we also
implemented an internal stress test workload. When turned on, SeaStore would
aggressively load the extents from the cold tier to the Logical Bucket Cache,
and demote extents back to the cold tier. The newly written data may also be
written directly to the cold tier based on ``seastore_test_workload_write_through_probability``.
The internal test workload is turned on by ``crimson_test_workload``.

View File

@ -71,6 +71,31 @@ teuthology using the `cram task`_.
.. _`cram`: https://bitheap.org/cram/
.. _`cram task`: https://github.com/ceph/ceph/blob/master/qa/tasks/cram.py
Running CLI tools tests
-----------------------
A convenience script is provided to run CLI tests easily without having to manually set up the environment.
From the Ceph source tree root directory:
.. prompt:: bash $
# Run all CLI tests
./src/script/run-cli-tests.sh
# Run with custom build directory
./src/script/run-cli-tests.sh -b /path/to/build
Alternatively, you can run the tests directly from the build directory by setting up your PATH:
.. prompt:: bash $
cd build
PATH="$PWD/bin:$PATH" ../src/test/run-cli-tests
The test suite includes tests for: ``ceph-authtool``, ``ceph-conf``,
``ceph-kvstore-tool``, ``crushtool``, ``monmaptool``, ``osdmaptool``,
``radosgw-admin``, and ``rbd``.
Tox-based testing of Python modules
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Some of the Python modules in Ceph use `tox <https://tox.readthedocs.io/en/latest/>`_

View File

@ -466,7 +466,7 @@ These operations provide the same semantics as in replicated pools, ensuring
compatibility with existing applications.
Read Operation Flow
~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~
Read operations follow a simple flow:
@ -906,9 +906,10 @@ Required Settings
To enable the new features, the following OSDMap pool settings are required:
- ``allows_ec_overwrites = true`
- ``allows_ec_overwrites = true``
- ``allows_ec_optimizations = true``
- ``supports_omap = true``
These settings can be configured per-pool. The cluster will enforce that all
OSDs are at the Umbrella release before allowing omap support to be enabled.
OSDs are at the Umbrella release before allowing omap support to be enabled.
The `supports_omap` pool flag is automatically enabled in pools with EC optimizations enabled.

View File

@ -41,8 +41,7 @@ Make sure X (and, ideally, X+1) is defined:
Github Actions
~~~~~~~~~~~~~~
- [ ] .github/workflows/redmine-upkeep.yml add release branch to pull_request_target trigger
- [ ] .github/workflows/releng-audit.yml add release branch to pull_request_target trigger
- [ ] `.github/workflows/*.yml' add release branch to pull_request_target trigger
Scripts
~~~~~~~
@ -137,6 +136,8 @@ After dev freeze
- [ ] open the Branch Protection settings of the Ceph repo. Duplicate settings to the new release branch.
- [ ] create vX.3.0 annotated tag on ``main`` so upgrades from new release to main are not wrongly considered downgrades.
- [ ] remove ``doc/releases/*.rst``. This should leave behind ``doc/releases/releases.yml`` which is used for doc building purposes. See also commit 33d63c3 ("doc: remove release notes for release branch") for details.
- [ ] remove ``.github/workflows``. ``main`` is authoritative for workflows; avoid possible confusion/conflicts.
- [ ] remove ``.github/pull_request_template.md``. Backports do not use it.
- [ ] cherry-pick 8cf9ad62949516666ad0f2c0bb7726ef68e4d666 ("doc: add releases links to toc"). There will be trivial conflicts.
- [ ] add redirect for new major release at `RTD <https://readthedocs.org/dashboard/ceph/redirects/>`_.
- [ ] add release name to redmine (using https://tracker.ceph.com/custom_fields/16/edit)

View File

@ -87,8 +87,8 @@ Once QE has determined a stopping point in the working (e.g., ``squid``) branch,
Notify the "Build Lead" that the release branch is ready.
2. Starting the build
=====================
2a. Starting the build
======================
We'll use a stable/regular 19.2.2 release of Squid as an example throughout this document.
@ -104,25 +104,52 @@ We'll use a stable/regular 19.2.2 release of Squid as an example throughout this
NOTE: if for some reason the build has to be restarted (for example if one distro failed) then the ``TAG`` option has to be unchecked.
4. Use https://docs.ceph.com/en/latest/start/os-recommendations/?highlight=debian#platforms to determine the ``DISTROS`` parameter. For example,
4. Use :ref:`start-platforms` to determine the ``DISTROS`` parameter. For example,
+-------------------+--------------------------------------------------+
| Release | Distro Codemap |
+===================+==================================================+
| pacific (16.X.X) | ``focal bionic buster bullseye`` |
+-------------------+--------------------------------------------------+
| quincy (17.X.X) | ``jammy focal centos9 bullseye`` |
+-------------------+--------------------------------------------------+
| reef (18.X.X) | ``jammy focal centos9 windows bookworm`` |
+-------------------+--------------------------------------------------+
| squid (19.X.X) | ``jammy centos9 windows bookworm`` |
+-------------------+--------------------------------------------------+
| tentacle (20.X.X) | ``jammy centos9 noble windows bookworm rocky10`` |
+-------------------+--------------------------------------------------+
+-------------------+---------------------------------------------------------+
| Release | Distro Codemap |
+===================+=========================================================+
| squid (19.X.X) | ``jammy centos9 windows bookworm`` |
+-------------------+---------------------------------------------------------+
| tentacle (20.X.X) | ``jammy centos9 rocky10 windows bookworm`` |
+-------------------+---------------------------------------------------------+
| umbrella (21.X.X) | ``jammy noble centos9 rocky10 bookworm trixie windows`` |
+-------------------+---------------------------------------------------------+
5. Click ``Build``.
2b. What to do if your build fails
==================================
The ceph-release-pipeline parent job has three stages.
If your build fails during the "create ceph release tag" stage, troubleshoot the child ceph-tag job and re-run the parent ceph-release-pipeline job with the same parameters.
----
If your build fails during the "package build" stage, troubleshoot the child ceph-dev-pipeline job. If only one variant failed to build (e.g., centos9 arm64), start a new ceph-release-pipeline job specifying::
DISTROS=centos9
ARCHS=arm64
TAG=false <-- VERY IMPORTANT
This will leave the version commit and previously-created tag intact in ceph-releases.git. You will want the subsequent ceph-dev-pipeline job to reuse that SHA/tag.
Once all of your variants are successfully built, you will have to manually run the ceph-tag job. For example,::
BRANCH=tentacle
TAG=true
TAG_PHASE=push
VERSION=20.2.0
RELEASE_TYPE=STABLE
Then proceed with the normal release process.
----
If your build fails during the "push ceph release tag" stage, troubleshoot the child ceph-tag job and re-run **just** the ceph-tag job again manually. Do not re-run ceph-release-pipeline.
3. Release Notes
================
@ -178,18 +205,57 @@ See `the Ceph Tracker wiki page that explains how to write the release notes <ht
.. prompt:: bash
merfi gpg /opt/repos/ceph/squid-19.2.2/debian/
sign-debs ceph squid
Example::
+ project=ceph
+ releases=(squid)
+ distro_versions=(jessie)
+ for release in "${releases[@]}"
+ for distro_version in "${distro_versions[@]}"
+ for path in /opt/repos/ceph/squid*
+ '[' -d /opt/repos/ceph/squid-19.2.2/debian/jessie ']'
+ needs_signing=0
+ gpg --verify /opt/repos/ceph/squid-19.2.2/debian/jessie/dists/bookworm/Release.gpg /opt/repos/ceph/squid-19.2.2/debian/jessie/dists/bookworm/Release
+ needs_signing=1
+ break
+ '[' 1 -eq 0 ']'
+ echo 'Signing: /opt/repos/ceph/squid-19.2.2/debian/jessie'
Signing: /opt/repos/ceph/squid-19.2.2/debian/jessie
+ merfi gpg /opt/repos/ceph/squid-19.2.2/debian/jessie
--> Starting path collection, looking for files to sign
--> 1 repos found
--> 2 repos found
--> signing: /opt/repos/ceph/squid-19.2.2/debian/jessie/dists/bookworm/Release
--> Running command: gpg --batch --yes --armor --detach-sig --output Release.gpg Release
--> Running command: gpg --batch --yes --clearsign --output InRelease Release
--> signing: /opt/repos/ceph/squid-19.2.2/debian/jessie/dists/jammy/Release
--> Running command: gpg --batch --yes --armor --detach-sig --output Release.gpg Release
--> Running command: gpg --batch --yes --clearsign --output InRelease Release
+ gpg --verify /opt/repos/ceph/squid-19.2.2/debian/jessie/dists/bookworm/Release.gpg /opt/repos/ceph/squid-19.2.2/debian/jessie/dists/bookworm/Release
gpg: Signature made Thu May 14 12:00:00 2026 UTC
gpg: using RSA key 460F3994
gpg: Good signature from "Ceph Release Key <security@ceph.com>"
+ echo 'verifying: /opt/repos/ceph/squid-19.2.2/debian/jessie/dists/bookworm/Release.gpg'
verifying: /opt/repos/ceph/squid-19.2.2/debian/jessie/dists/bookworm/Release.gpg
+ gpg --verify /opt/repos/ceph/squid-19.2.2/debian/jessie/dists/bookworm/InRelease
gpg: Signature made Thu May 14 12:00:00 2026 UTC
gpg: using RSA key 460F3994
gpg: Good signature from "Ceph Release Key <security@ceph.com>"
+ echo 'verifying: /opt/repos/ceph/squid-19.2.2/debian/jessie/dists/bookworm/InRelease'
verifying: /opt/repos/ceph/squid-19.2.2/debian/jessie/dists/bookworm/InRelease
+ gpg --verify /opt/repos/ceph/squid-19.2.2/debian/jessie/dists/jammy/Release.gpg /opt/repos/ceph/squid-19.2.2/debian/jessie/dists/jammy/Release
gpg: Signature made Thu May 14 12:00:00 2026 UTC
gpg: using RSA key 460F3994
gpg: Good signature from "Ceph Release Key <security@ceph.com>"
+ echo 'verifying: /opt/repos/ceph/squid-19.2.2/debian/jessie/dists/jammy/Release.gpg'
verifying: /opt/repos/ceph/squid-19.2.2/debian/jessie/dists/jammy/Release.gpg
+ gpg --verify /opt/repos/ceph/squid-19.2.2/debian/jessie/dists/jammy/InRelease
gpg: Signature made Thu May 14 12:00:00 2026 UTC
gpg: using RSA key 460F3994
gpg: Good signature from "Ceph Release Key <security@ceph.com>"
+ echo 'verifying: /opt/repos/ceph/squid-19.2.2/debian/jessie/dists/jammy/InRelease'
verifying: /opt/repos/ceph/squid-19.2.2/debian/jessie/dists/jammy/InRelease
etc...

View File

@ -20,59 +20,11 @@ cost.
For more information, see `https://ceph.com/foundation
<https://ceph.com/foundation>`_.
Members
=======
Diamond
-------
Visit `https://ceph.io/en/foundation/members/ <https://ceph.io/en/foundation/members/>`_ to see the full list of Ceph Foundation Members
* `45Drives <https://45drives.com/>`_
* `Bloomberg <https://bloomberg.com>`_
* `IBM <https://ibm.com>`_
* `Clyso <https://www.clyso.com/en/>`_
Platinum
--------
* `Western Digital <https://westerndigital.com/>`_
Gold
----
* `42on <https://www.42on.com/>`_
* `OVH <https://www.ovh.com/>`_
* `Samsung Electronics <https://samsung.com/>`_
Silver
------
* `Canonical <https://www.canonical.com/>`_
* `CloudFerro <https://cloudferro.com/>`_
* `croit <http://www.croit.io/>`_
* `Digital Ocean <http://digitalocean.com/>`_
* `ISS <http://iss-integration.com/>`_
* `Intel <http://www.intel.com/>`_
* `OSNexus <https://osnexus.com/>`_
* `Sony Interactive <https://sonyinteractive.com/>`_
Associate
---------
* `Boston University <http://www.bu.com/>`_
* `Center for Research in Open Source Systems (CROSS) <http://cross.ucsc.edu/>`_
* `CERN <https://home.cern/>`_
* `FASRC <https://www.rc.fas.harvard.edu/>`_
* `grnet <https://grnet.gr/>`_
* `Monash University <http://www.monash.edu/>`_
* `NRF SARAO <http://www.ska.ac.za/about/sarao/>`_
* `Open Infrastructure Foundation <http://openinfra.dev>`_
* `Science & Technology Facilities Councel (STFC) <https://stfc.ukri.org/>`_
* `SWITCH <https://switch.ch/>`_
* `University of Michigan <http://www.osris.org/>`_
Governing Board
===============
@ -97,21 +49,6 @@ engineering activities are managed through traditional open source
processes and are overseen by the :ref:`csc`. For more
information see :ref:`governance`.
Members
-------
* Brett Kelly (45Drives)
* Enrico Bocchi (CERN) - Associate member representative
* Dan van der Ster (Clyso) - Ceph Council representative
* Joachim Kraftmayer (Clyso)
* Josh Durgin (IBM) - Ceph Council representative
* Matthew Leonard (Bloomberg)
* Anthony Middleton (Linux Foundation) - Ceph Community Manager
* Neha Ojha (IBM) - Ceph Council Representative
* Steven Umbehocker (OSNexus) - General member representative
* Pawel Sadowski (OVH)
* Sungmin Lee (Samsung Electronics)
* Vincent Hsu (IBM)
Joining
=======

View File

@ -353,9 +353,8 @@
reduce the time it takes to read data from and to write to the
Ceph cluster. RGW bucket indexes are stored as omaps.
Erasure-coded pools, by default, cannot store RADOS omap data
structures. This functionality can be enabled in erasure-coded
pools with optimizations enabled by setting the
``supports_omap`` flag to true.
structures. This functionality can be enabled in non-Crimson
erasure-coded pools by enabling EC optimizations.
Run the command ``ceph osd df`` to see the storage space used by
omaps on each OSD.

View File

@ -38,8 +38,7 @@ Membership
* Candidates self-nominate or are nominated by other members
* Discussion of how roles/responsibilities may be delegated
* Ranked-choice vote by the steering committee
* 2 year terms, with one member being elected in even years, and the
other two in odd years
* 1 year terms with all members elected yearly
* Members may resign at any time, and the steering committee may vote
to appoint a replacement for the rest of their term
* members must involve >1 employer
@ -86,6 +85,7 @@ Current Members
* Adam King <adking@redhat.com>
* Afreen Misbah <afreen@ibm.com>
* Anthony D'Atri <anthony.datri@gmail.com>
* Aviv Caro <Aviv.Caro@ibm.com>
* Casey Bodley <cbodley@redhat.com>
* Dan van der Ster <dan.vanderster@clyso.com>
* David Orman <ormandj@1111systems.com>
@ -99,15 +99,18 @@ Current Members
* Joseph Mundackal <jmundackal@bloomberg.net>
* Josh Durgin <jdurgin@redhat.com>
* João Eduardo Luis <joao@clyso.com>
* Laura Flores <lflores@redhat.com>
* Kyle Bader <kbader@ibm.com>
* Laura Flores <lflores@ibm.com>
* Mark Nelson <mark.nelson@clyso.com>
* Matan Breizman <mbreizma@redhat.com>
* Matt Benjamin <mbenjami@redhat.com>
* Mike Perez <miperez@redhat.com>
* Myoungwon Oh <ohmyoungwon@gmail.com>
* Sage McTaggart <sagemct@ibm.com>
* Neha Ojha <nojha@redhat.com>
* Patrick Donnelly <pdonnell@ibm.com>
* Radoslaw Zarzynski <rzarzyns@redhat.com>
* Redouane Kachach <rkachach@redhat.com>
* Venky Shankar <vshankar@redhat.com>
* Vikhyat Umrao <vikhyat@ibm.com>
* Xie Xingguo <xie.xingguo@zte.com.cn>

View File

@ -88,7 +88,8 @@ Supported categories are:
* network
* power
* fans
* firmwares
* temperatures
* firmware
* criticals
@ -143,7 +144,7 @@ ________________
.. prompt:: bash # auto
# ceph orch hardware status node-10 --category firmwares
# ceph orch hardware status node-10 --category firmware
+------------+----------------------------------------------------------------------------+--------------------------------------------------------------+----------------------+-------------+--------+
| HOST | COMPONENT | NAME | DATE | VERSION | STATUS |
+------------+----------------------------------------------------------------------------+--------------------------------------------------------------+----------------------+-------------+--------+
@ -187,6 +188,7 @@ For Developers
.. automethod:: NodeProxyEndpoint.power
.. automethod:: NodeProxyEndpoint.processors
.. automethod:: NodeProxyEndpoint.fans
.. automethod:: NodeProxyEndpoint.firmwares
.. automethod:: NodeProxyEndpoint.temperatures
.. automethod:: NodeProxyEndpoint.firmware
.. automethod:: NodeProxyEndpoint.led

View File

@ -42,13 +42,13 @@ repository and execute the following::
cd build
ninja
See `Installing a Build`_ to install a build in user space and `Ceph README.md`_
See :ref:`install-storage-cluster-build` to install a build in user space and `Ceph README.md`_
doc for more details on build.
Build Ceph Packages
===================
To build packages, you must clone the `Ceph`_ repository. You can create
To build packages, you must clone the :ref:`Ceph repository <install-clone-source>`. You can create
installation packages from the latest code using ``dpkg-buildpackage`` for
Debian/Ubuntu or ``rpmbuild`` for the RPM Package Manager.
@ -61,7 +61,7 @@ Advanced Package Tool (APT)
---------------------------
To create ``.deb`` packages for Debian/Ubuntu, ensure that you have cloned the
`Ceph`_ repository, installed the `Build Prerequisites`_ and installed
:ref:`Ceph repository <install-clone-source>`, installed the `Build Prerequisites`_ and installed
``debhelper``::
sudo apt-get install debhelper
@ -76,7 +76,7 @@ For multi-processor CPUs use the ``-j`` option to accelerate the build.
RPM Package Manager
-------------------
To create ``.rpm`` packages, ensure that you have cloned the `Ceph`_ repository,
To create ``.rpm`` packages, ensure that you have cloned the :ref:`Ceph repository <install-clone-source>`,
installed the `Build Prerequisites`_ and installed ``rpm-build`` and
``rpmdevtools``::
@ -104,6 +104,4 @@ Build the RPM packages::
For multi-processor CPUs use the ``-j`` option to accelerate the build.
.. _Ceph: ../clone-source
.. _Installing a Build: ../install-storage-cluster#installing-a-build
.. _Ceph README.md: https://github.com/ceph/ceph#building-ceph

View File

@ -1,3 +1,5 @@
.. _install-clone-source:
=========================================
Cloning the Ceph Source Code Repository
=========================================

View File

@ -123,7 +123,7 @@ For RPMs::
The major releases of Ceph are summarized at: :ref:`Releases <ceph-releases-index>`
.. tip:: For non-US users: There might be a mirror close to you where
to download Ceph from. For more information see: `Ceph Mirrors`_.
to download Ceph from. For more information see: :ref:`install-mirrors`.
Debian Packages
~~~~~~~~~~~~~~~
@ -168,7 +168,7 @@ of Debian and Ubuntu releases supported.
echo deb https://download.ceph.com/debian-testing/ $(lsb_release -sc) main | sudo tee /etc/apt/sources.list.d/ceph.list
.. tip:: For non-US users: There might be a mirror close to you where
to download Ceph from. For more information see: `Ceph Mirrors`_.
to download Ceph from. For more information see: :ref:`install-mirrors`.
RPM Packages
@ -234,7 +234,7 @@ You can download the RPMs directly from
https://download.ceph.com/rpm-testing
.. tip:: For non-US users: There might be a mirror close to you where
to download Ceph from. For more information see: `Ceph Mirrors`_.
to download Ceph from. For more information see: :ref:`install-mirrors`.
openSUSE Leap 15.1
^^^^^^^^^^^^^^^^^^
@ -389,4 +389,3 @@ line to get the short codename.
.. _the testing Debian repository: https://download.ceph.com/debian-testing/dists
.. _the shaman page: https://shaman.ceph.com
.. _Ceph Mirrors: ../mirrors

View File

@ -7,8 +7,7 @@ source code. You may download source code tarballs for Ceph releases here:
`Ceph Release Tarballs`_
.. tip:: For international users: There might be a mirror close to you where download Ceph from. For more information see: `Ceph Mirrors`_.
.. tip:: For international users: There might be a mirror close to you where download Ceph from. For more information see: :ref:`install-mirrors`.
.. _Ceph Release Tarballs: https://download.ceph.com/tarballs/
.. _Ceph Mirrors: ../mirrors

View File

@ -24,7 +24,6 @@ in Kubernetes, while also enabling management of storage resources and
provisioning via Kubernetes APIs. We recommend Rook as the way to run Ceph in
Kubernetes or to connect an existing Ceph storage cluster to Kubernetes.
* Rook supports only Nautilus and newer releases of Ceph.
* Rook is the preferred method for running Ceph on Kubernetes, or for
connecting a Kubernetes cluster to an existing (external) Ceph
cluster.
@ -34,15 +33,6 @@ Kubernetes or to connect an existing Ceph storage cluster to Kubernetes.
Other methods
~~~~~~~~~~~~~
`ceph-ansible <https://docs.ceph.com/ceph-ansible/>`_ deploys and manages
Ceph clusters using Ansible.
* ceph-ansible is widely deployed.
* ceph-ansible is not integrated with the orchestrator APIs that were
introduced in Nautilus and Octopus, which means that the management features
and dashboard integration introduced in Nautilus and Octopus are not
available in Ceph clusters deployed by means of ceph-ansible.
`ceph-salt <https://github.com/ceph/ceph-salt>`_ installs Ceph using Salt and cephadm.
`jaas.ai/ceph-mon <https://jaas.ai/ceph-mon>`_ installs Ceph using Juju.
@ -63,6 +53,4 @@ Windows
~~~~~~~
For Windows installations, consult this document:
`Windows installation guide`_.
.. _Windows installation guide: ./windows-install
:ref:`install-windows`.

View File

@ -1,3 +1,5 @@
.. _install_storage_cluster:
==============================
Install Ceph Storage Cluster
==============================
@ -79,6 +81,7 @@ Once you have added either release or development packages, or added a
sudo yum install ceph
.. _install-storage-cluster-build:
Installing a Build
==================

View File

@ -1,3 +1,5 @@
.. _install-mirrors:
=============
Ceph Mirrors
=============
@ -24,17 +26,18 @@ These mirrors are available on the following locations:
- **US-West: US West Coast**: http://us-west.ceph.com/
- **CN: China**: http://mirrors.ustc.edu.cn/ceph/
You can replace all download.ceph.com URLs with any of the mirrors, for example:
You can replace all ``download.ceph.com`` URLs with any of the mirrors,
for example:
- http://download.ceph.com/tarballs/
- http://download.ceph.com/debian-hammer/
- http://download.ceph.com/rpm-hammer/
- https://download.ceph.com/tarballs/
- https://download.ceph.com/debian-tentacle/
- https://download.ceph.com/rpm-tentacle/
Change this to:
- http://eu.ceph.com/tarballs/
- http://eu.ceph.com/debian-hammer/
- http://eu.ceph.com/rpm-hammer/
- https://eu.ceph.com/tarballs/
- https://eu.ceph.com/debian-tentacle/
- https://eu.ceph.com/rpm-tentacle/
Mirroring
@ -62,4 +65,4 @@ set for all mirrors. These can be found on `GitHub`_.
If you want to apply for an official mirror, please contact the ceph-users mailinglist.
.. _GitHub: https://github.com/ceph/ceph/tree/master/mirroring
.. _GitHub: https://github.com/ceph/ceph/tree/main/mirroring

View File

@ -1,5 +1,7 @@
:orphan:
.. _install-windows-basic-config:
===========================
Windows basic configuration
===========================

View File

@ -1,5 +1,7 @@
:orphan:
.. _install-windows:
==========================
Installing Ceph on Windows
==========================
@ -15,7 +17,7 @@ Supported platforms
-------------------
.. note::
Please see the `OS recommendations`_ regarding client package support.
Please see the :ref:`os-recommendations` regarding client package support.
Windows Server 2019 and Windows Server 2016 are supported. Previous Windows
Server versions, including Windows client versions such as Windows 10, might
@ -66,7 +68,7 @@ https://github.com/ceph/ceph/blob/master/README.windows.rst
Configuration
=============
Please check the `Windows configuration sample`_ to get started.
Please check the :ref:`Windows configuration sample <install-windows-basic-config>` to get started.
You'll also need a keyring file. The `General CephFS Prerequisites`_ page provides a
simple example, showing how a new CephX user can be created and how its secret
@ -80,12 +82,8 @@ Further reading
* `RBD Windows documentation`_
* :ref:`CephFS Windows documentation <ceph-dokan>`
* `Windows troubleshooting`_
* :ref:`install-windows-troubleshooting`
.. _Windows configuration sample: ../windows-basic-config
.. _RBD Windows documentation: ../../rbd/rbd-windows/
.. _Windows troubleshooting: ../windows-troubleshooting
.. _General CephFS Prerequisites: ../../cephfs/mount-prerequisites
.. _Client Authentication: ../../cephfs/client-auth
.. _Windows testing: ../dev/tests-windows
.. _OS recommendations: ../../start/os-recommendations

View File

@ -1,5 +1,7 @@
:orphan:
.. _install-windows-troubleshooting:
===============================
Troubleshooting Ceph on Windows
===============================

View File

@ -61,6 +61,7 @@ if(WITH_RADOSGW)
rgw-orphan-list.rst
rgw-gap-list.rst
rgw-policy-check.rst
rgw-policy-test.rst
ceph-diff-sorted.rst
rgw-restore-bucket-index.rst)
endif()

View File

@ -20,6 +20,7 @@ Synopsis
| **ceph-bluestore-tool** qfsck --path *osd path*
| **ceph-bluestore-tool** allocmap --path *osd path*
| **ceph-bluestore-tool** restore_cfb --path *osd path*
| **ceph-bluestore-tool** recovery-compare --path *osd path*
| **ceph-bluestore-tool** show-label --dev *device* ...
| **ceph-bluestore-tool** show-label-at --dev *device* --offset *lba* ...
| **ceph-bluestore-tool** prime-osd-dir --dev *device* --path *osd path*
@ -72,6 +73,9 @@ Commands
Reverses changes done by the new NCB code (either through ceph restart or when running allocmap command) and restores RocksDB B Column-Family (allocator-map).
:command:`recovery-compare`
Runs legacy onode recovery and multithread onode recovery. Prints timings and compares results.
:command:`bluefs-export`

View File

@ -632,11 +632,11 @@ rules and failure handling on all pools. For a given PG to successfully peer
and be marked active, ``min_size`` replicas will now need to be active under all
(currently two) CRUSH buckets of type <dividing_bucket>.
<tiebreaker_mon> is the tiebreaker mon to use if a network split happens.
<tiebreaker_mon> is the tiebreaker Monitor to use if a network split happens.
This parameter is optional. If not supplied, the system will automatically
select a monitor that is not in either data zone. If there are multiple
monitors outside the data zones, automatic selection will fail and you must
explicitly specify the tiebreaker monitor.
select a Monitor that is not in either data zone. If there are multiple
Monitors outside the data zones, automatic selection will fail and you must
explicitly specify the tiebreaker Monitor.
<dividing_bucket> is the bucket type across which to stretch.
This will typically be ``datacenter`` or other CRUSH hierarchy bucket type that
@ -1196,7 +1196,8 @@ Subcommand ``get`` gets pool parameter <var>.
Usage::
ceph osd pool get <poolname> size|min_size|pg_num|pgp_num|crush_rule|write_fadvise_dontneed
ceph osd pool get <poolname> size|min_size|pg_num|pgp_num|crush_rule|hashpspool|
nodelete|nopgchange|nosizechange|write_fadvise_dontneed
Only for tiered pools::
@ -1249,7 +1250,7 @@ Usage::
ceph osd pool set <poolname> size|min_size|pg_num|
pgp_num|crush_rule|hashpspool|nodelete|nopgchange|nosizechange|
hit_set_type|hit_set_period|hit_set_count|hit_set_fpp|debug_fake_ec_pool|
hit_set_type|hit_set_period|hit_set_count|hit_set_fpp|
target_max_bytes|target_max_objects|cache_target_dirty_ratio|
cache_target_dirty_high_ratio|
cache_target_full_ratio|cache_min_flush_age|cache_min_evict_age|

View File

@ -13,8 +13,8 @@ Synopsis
| [--log-dir LOG_DIR] [--logrotate-dir LOGROTATE_DIR]
| [--unit-dir UNIT_DIR] [--verbose] [--timeout TIMEOUT]
| [--retry RETRY] [--no-container-init]
| {version,pull,inspect-image,ls,list-networks,list-rdma,adopt,rm-daemon,rm-cluster,run,shell,enter,ceph-volume,unit,logs,bootstrap,deploy,check-host,prepare-host,add-repo,rm-repo,install,list-images,update-osd-service}
| ...
| {version,pull,inspect-image,ls,list-networks,list-rdma,adopt,rm-daemon,rm-cluster,remove-file,deploy-file,run,shell,enter,ceph-volume,unit,logs,bootstrap,deploy,check-host,prepare-host,prepare-host-sudo-hardening,setup-ssh-user,add-repo,rm-repo,install,list-images,update-osd-service}
| ...
| **cephadm** **pull**
@ -90,8 +90,19 @@ Synopsis
| **cephadm** **check-host** [-h] [--expect-hostname EXPECT_HOSTNAME]
| **cephadm** **remove-file** [-h] [--fsid FSID] --path PATH
| **cephadm** **deploy-file** [-h] [--fsid FSID] --path PATH [--mode MODE]
| [--uid UID] [--gid GID]
| **cephadm** **prepare-host**
| **cephadm** **prepare-host-sudo-hardening** [-h] [--ssh-user SSH_USER]
| [--ssh-pub-key SSH_PUB_KEY]
| [--cephadm-version VERSION]
| **cephadm** **setup-ssh-user** [-h] --ssh-user SSH_USER --ssh-pub-key SSH_PUB_KEY
| **cephadm** **add-repo** [-h] [--release RELEASE] [--version VERSION]
| [--dev DEV] [--dev-commit DEV_COMMIT]
| [--gpg-url GPG_URL] [--repo-url REPO_URL]
@ -289,6 +300,32 @@ Arguments:
* [--expect-hostname EXPECT_HOSTNAME] Check that hostname matches an expected value
remove-file
-----------
Remove a regular file on the local host. Missing paths are ignored.
Arguments:
* [--fsid FSID] cluster FSID
* --path PATH absolute path of the file to remove (required)
deploy-file
-----------
Write or replace a file on the local host. The **entire file body** is read from
**standard input** as raw bytes (no encoding or line-ending translation).
Arguments:
* [--fsid FSID] cluster FSID
* --path PATH absolute destination path for the file (required)
* [--mode MODE] octal file mode (for example ``644`` or ``0644``)
* [--uid UID] numeric owner user id (**must** be given together with ``--gid``)
* [--gid GID] numeric owner group id (**must** be given together with ``--uid``)
deploy
------
@ -420,6 +457,49 @@ Arguments:
* [--expect-hostname EXPECT_HOSTNAME] Set hostname
prepare-host-sudo-hardening
---------------------------
Prepare a host for sudo hardening by authorizing SSH keys, installing/upgrading
the cephadm package, and setting up restricted sudoers permissions::
cephadm prepare-host-sudo-hardening --ssh-user cephadm --ssh-pub-key <key>
This command performs the following steps:
1. Authorizes the provided SSH public key for the specified user
2. Installs or upgrades the cephadm package to match the cluster version (includes cephadm_invoker.py)
3. Sets up sudoers with restricted permissions for cephadm_invoker.py
Arguments:
* [--ssh-user SSH_USER] SSH user for key authorization (default: root)
* [--ssh-pub-key SSH_PUB_KEY] SSH public key to authorize
* [--cephadm-version VERSION] Specific cephadm version to install
setup-ssh-user
--------------
Setup SSH user with passwordless sudo and SSH key authorization::
cephadm setup-ssh-user --ssh-user cephadm --ssh-pub-key <public_key>
This command configures an SSH user for cephadm operations by:
1. Validating that the user exists on the system
2. Setting up passwordless sudo for the user (skipped for root)
3. Authorizing the SSH public key for the user
This command is automatically called by ``ceph cephadm set-user`` to configure
SSH users across all cluster hosts.
Arguments:
* [--ssh-user SSH_USER] SSH user to setup (required)
* [--ssh-pub-key SSH_PUB_KEY] SSH public key to add to user's authorized_keys (required)
pull
----

View File

@ -151,15 +151,35 @@ pools; it only runs simulations by mapping values in the range
Displays how many attempts were needed to find a device mapping.
For instance::
0: 95224
1: 3745
2: 2225
tries 0: 95224
tries 1: 3745
tries 2: 2225
..
shows that **95224** mappings succeeded without retries, **3745**
mappings succeeded with one attempts, etc. There are as many rows
as the value of the **--set-choose-total-tries** option.
.. option:: --show-retry-exhaustion
Checks whether any PG mappings hit the maximum retry limit
(typically 50 tries) and displays a warning if retry exhaustion
is detected. This is useful for validating CRUSH rules, especially
in stretch cluster configurations with exactly
2 datacenters and 4 replicas can potentially fail to map some PGs due to CRUSH's
pseudorandom selection repeatedly choosing the same datacenter.
Although, the probability is extremely low, we still want to detect this.
Outputs either a warning message if exhaustion is detected::
WARNING: Retry exhaustion detected!
5 PG(s) hit the maximum retry limit of 50
This indicates CRUSH failed to find optimal placement for some PGs.
or a success message showing the maximum tries actually needed::
No retry exhaustion detected (maximum tries needed: 7 / 50)
.. option:: --output-csv
Creates CSV files (in the current directory) containing information

View File

@ -639,7 +639,7 @@ Options
.. option:: --shard-id=<shard-id>
Optional for mdlog list, bi list, data sync status. Required for ``mdlog trim``.
Optional for mdlog list, bi list, data sync status, gc list, gc process. Required for ``mdlog trim``.
.. option:: --max-entries=<entries>

View File

@ -0,0 +1,65 @@
:orphan:
===================================================
rgw-policy-test -- test evaluation of bucket policy
===================================================
.. program:: rgw-policy-test
Synopsis
========
| **rgw-policy-test**
[-e *key*=*value*]... [-t *tenant*] [-R resource] [-I identity] *action* *filename*
Description
===========
This utility reads a policy from a file or stdin and evaluates it
against the supplied identity, resource, action, and request context
(the key/value pairs evaluated by conditions.) The utility will print
a trace of evaluating the policy and the effect of its evaluation.
On error it will print a message and return non-zero.
Options
=======
.. option:: -t *tenant*, --tenant *tenant*
Specify *tenant* as the tenant. If not provided, the default
(empty) tenant is used.
.. option:: -e *key*=*value*, --environment *key*=*value*
Add (*key*, *value*) to the environment for evaluation. May be
specified multiple times.
.. option:: -I *SCHEMA:STRING*, --identity *SCHEMA:STRING*
Specify *identity* as the identity whose access is to be checked.
.. option:: -R *ARN*, --resource *ARN*
Specify *ARN* as the resource for which to check access.
.. option:: *action*
Use *action* as the action to check.
.. option:: *filename*
Read the policy from *filename*, or - for standard input.
Availability
============
**rgw-policy-test** is part of Ceph, a massively scalable, open-source,
distributed storage system. Please refer to the Ceph documentation at
https://docs.ceph.com/ for more information.
See also
========
:doc:`radosgw <radosgw>`\(8)

View File

@ -48,4 +48,5 @@
man/8/ceph-immutable-object-cache
man/8/ceph-diff-sorted
man/8/rgw-policy-check
man/8/rgw-policy-test
man/8/rgw-restore-bucket-index

View File

@ -161,9 +161,9 @@ The Manager automatically adjusts :confval:`mgr_stats_period` based on message q
depth to prevent overload during high cluster activity. This feature is enabled by
default and can be controlled with the following settings:
- :confval:`mgr_stats_period_autotune` (boolean, default: true): Enable or disable
- :confval:`mgr_stats_period_autotune` : Enable or disable
automatic tuning of the stats period.
- :confval:`mgr_stats_period_autotune_queue_threshold` (integer, default: 100):
- :confval:`mgr_stats_period_autotune_queue_threshold` :
The message queue depth threshold that triggers an increase in the stats period.
When the queue depth exceeds this threshold, the stats period is increased to

434
doc/mgr/ceph_secrets.rst Normal file
View File

@ -0,0 +1,434 @@
.. _mgr-ceph-secrets:
===================
Ceph Secrets Module
===================
The ``ceph_secrets`` manager module provides centralised secret storage for
Ceph operators and Ceph manager modules. Instead of embedding plaintext
credentials in service specifications or configuration objects, secrets are
stored once and referenced by URI. Ceph manager modules such as cephadm
and rook can store secret payloads centrally, reference them by URI, and
resolve them to plaintext only when needed at deploy time.
For example, a cephadm-managed service may need an API token. Instead of
embedding that token directly in the service specification, an operator can
store it once:
.. prompt:: bash $
ceph secret set cephadm/service/my-service/api_token -i /tmp/api-token
and then use the URI
``secret:/cephadm/service/my-service/api_token`` in the
service spec instead of the plaintext token. A cephadm integration can then
resolve the URI at deploy time and write the token only into the daemon files
that need it.
Secrets are stored in the Monitor KV store under the ``secret_store/v1/`` prefix
and are organised by namespace, scope, and name. Each secret is versioned and
carries ``created``/``updated`` timestamps. A per-namespace epoch counter
is incremented on every ``set`` and on any ``rm`` that actually removes a
secret, allowing consumers to detect changes without fetching the full secret
list.
.. note::
The ``mon`` backend stores secrets in the Monitor KV store, which is not an
external KMS or vault. Users and MGR modules with sufficient Ceph
permissions can still reveal or resolve stored secret values. Namespaces
provide logical and storage isolation, not an authorisation boundary.
If the module is not already enabled, run::
ceph mgr module enable ceph_secrets
.. contents::
:local:
:depth: 2
Concepts
========
Namespace
---------
A namespace is a logical storage boundary, typically matching the name of
the consuming MGR module (e.g. ``cephadm``, ``rook``). Secrets in different
namespaces are stored independently and have independent epoch counters.
Namespaces are not an authorisation boundary; any caller with access to the
``ceph_secrets`` module can read secrets from any namespace.
Scope
-----
Within a namespace, every secret has a scope that encodes what the secret
belongs to:
.. list-table::
:header-rows: 1
:widths: 15 40 45
* - Scope
- Meaning
- CLI path form
* - ``global``
- Cluster-wide secret, not tied to any specific target
- ``<namespace>/global/<name>``
* - ``service``
- Secret for a specific named service
- ``<namespace>/service/<service-name>/<name>``
* - ``host``
- Secret for a specific host
- ``<namespace>/host/<hostname>/<name>``
* - ``custom``
- Arbitrary slash-delimited path under the namespace
- ``<namespace>/custom/<path>``
Path grammar
------------
All path segments (namespace, scope target, name, and each segment of a
custom path) must match ``[A-Za-z0-9._-]+`` and must not end with ``'.'``.
Empty segments, percent-encoding, and leading/trailing whitespace are
rejected.
Custom scope paths may contain multiple slash-separated segments (e.g.
``cephadm/custom/app/db/password``). Two custom paths that share a common
prefix are independent secrets; ``cephadm/custom/a/b`` and
``cephadm/custom/a/b/c`` do not conflict.
Secret URIs
-----------
Internally, secrets are identified by URIs of the form::
secret:/<namespace>/global/<name>
secret:/<namespace>/service/<target>/<name>
secret:/<namespace>/host/<target>/<name>
secret:/<namespace>/custom/<path>
These URIs appear in the output of :ref:`scan_refs <mgr-ceph-secrets-scan>`
and may be embedded in configuration objects as opaque references to be
resolved at deploy time.
CLI Reference
=============
All CLI commands use the path form described above. Responses are JSON by
default; pass ``--format yaml`` for YAML output.
.. _mgr-ceph-secrets-set:
secret set
----------
Create or update a secret. The input file content is stored as an opaque string::
ceph secret set <path> -i <file>
Secret data must not be empty. Other contents, including leading or trailing
whitespace and final newlines, are preserved exactly. Callers that need to
store structured data should encode it themselves, for example as JSON, and
decode it after retrieval or resolution.
If the secret already exists its data is replaced and its version
incremented, unless the existing secret policy marks it non-editable (set via
the Python API with ``editable=False``), in which case the update is rejected.
The ``created`` timestamp is set on the first write and is never changed
thereafter; ``updated`` is refreshed on every write.
Example:
.. prompt:: bash $
ceph secret set cephadm/service/my-service/api_token -i /tmp/api-token
{"metadata": {"version": 1, "created": "2025-06-01T12:00:00Z", "updated": "2025-06-01T12:00:00Z"}}
.. _mgr-ceph-secrets-get:
secret get
----------
Retrieve the metadata (and, optionally, the data) for a secret::
ceph secret get <path> [--reveal] [--format {json|yaml}]
Without ``--reveal``, the ``data`` field is omitted from the response to
avoid accidental exposure in terminal output or logs. With ``--reveal``,
``data`` contains the stored opaque string.
Example:
.. prompt:: bash $
ceph secret get cephadm/service/my-service/api_token
{
"metadata": {
"version": 1,
"created": "2025-06-01T12:00:00Z",
"updated": "2025-06-01T12:00:00Z"
}
}
ceph secret get cephadm/service/my-service/api_token --reveal
{
"metadata": {
"version": 1,
"created": "2025-06-01T12:00:00Z",
"updated": "2025-06-01T12:00:00Z"
},
"data": "api-token-value"
}
.. _mgr-ceph-secrets-get-value:
secret get-value
----------------
Return the raw secret data string directly, with no JSON envelope::
ceph secret get-value <path>
Unlike ``secret get --reveal``, the output is the stored string itself,
making it suitable for use in shell scripts and pipelines. Returns an
error if the secret does not exist.
Example:
.. prompt:: bash $
ceph secret get-value cephadm/service/my-service/api_token
api-token-value
.. _mgr-ceph-secrets-ls:
secret ls
---------
List secrets, optionally filtered by namespace, scope, and/or target::
ceph secret ls [--namespace <ns>] [--scope <scope>]
[--sec_target <target>] [--reveal] [--show_internals]
[--format {json|yaml}]
Without filters, all secrets across all namespaces are listed.
``--reveal`` includes the stored opaque data string in each output record.
``--show_internals`` additionally includes the ``policy`` object (``user_made``
and ``editable`` flags) in each record.
Example:
.. prompt:: bash $
ceph secret ls --namespace cephadm --scope host --sec_target node1
{
"cephadm/host/node1/ssh_key": {
"metadata": {
"version": 3,
"created": "2025-05-10T08:00:00Z",
"updated": "2025-06-01T09:00:00Z"
},
"ref": {
"namespace": "cephadm",
"scope": "host",
"target": "node1",
"name": "ssh_key"
}
}
}
.. _mgr-ceph-secrets-rm:
secret rm
---------
Remove a secret::
ceph secret rm <path>
The operation is idempotent: removing a secret that does not exist succeeds
and reports ``"status": "not_found"`` rather than an error.
Example:
.. prompt:: bash $
ceph secret rm cephadm/service/my-service/api_token
{"status": "removed"}
ceph secret rm cephadm/service/my-service/api_token
{"status": "not_found"}
.. _mgr-ceph-secrets-get-epoch:
Epoch
-----
The epoch for a namespace is accessible via the Python API only (see
:ref:`Epoch-based change detection <mgr-ceph-secrets-epoch>`). There is
no CLI command for it.
Module API
==========
Other MGR modules should consume ``ceph_secrets`` via ``CephSecretsClient``
(``src/pybind/mgr/ceph_secrets_client.py``) rather than calling
``mgr.remote()`` directly. The client file, along with
``ceph_secrets_types.py``, lives at the top level of ``src/pybind/mgr/`` so
any MGR module can import it without depending on the ``ceph_secrets`` package
internals.
.. code-block:: python
from ceph_secrets_client import CephSecretsClient
from ceph_secrets_types import SecretScope
client = CephSecretsClient(self) # self is a MgrModule instance
# Store a secret as an opaque string
client.secret_set(
namespace="cephadm",
scope=SecretScope.HOST,
target="node1",
name="ssh_key",
data="AQB...==",
)
# Retrieve metadata (no data unless reveal=True)
rec = client.secret_get("cephadm", SecretScope.HOST, "node1", "ssh_key")
if rec:
version = rec["metadata"]["version"]
# Remove
client.secret_rm("cephadm", SecretScope.HOST, "node1", "ssh_key")
CephSecretsClient methods
--------------------------
.. list-table::
:header-rows: 1
:widths: 35 65
* - Method
- Description
* - ``secret_set(namespace, scope, target, name, data, ...)``
- Create or update a secret. ``data`` must be a non-empty opaque
string. Increments the version and refreshes ``updated`` on each
call; ``created`` is set only on the first write. Optional
``user_made`` and ``editable`` flags control the stored policy;
``editable=False`` blocks future updates but does not prevent removal.
* - ``secret_get(namespace, scope, target, name, reveal=False)``
- Return the secret's metadata dict, plus the opaque ``data`` string if
*reveal* is True. Returns ``None`` if the secret does not exist.
* - ``secret_get_value(namespace, scope, target, name)``
- Return the stored opaque string directly, or ``None`` if the secret
does not exist. Use this when only the value is needed and the
metadata envelope is not required.
* - ``secret_get_version(namespace, scope, target, name)``
- Return the current version integer, or ``None`` if not found. A
cheaper alternative to ``secret_get`` when only change-detection is needed.
* - ``secret_get_versions(uris)``
- Batch-fetch version numbers for a list of canonical ``secret:/...``
URIs. Returns a dict keyed by URI; missing secrets map to ``None``.
Malformed URIs are skipped entirely, so a missing key indicates
malformed input rather than an absent secret.
* - ``secret_rm(namespace, scope, target, name)``
- Remove a secret. Idempotent: returns ``False`` if not found.
* - ``secret_get_epoch(namespace)``
- Return the current epoch for the namespace. Use as a cheap
change-detector before a full refresh.
* - ``scan_refs(obj, namespace)``
- Walk a JSON-like object and return secret-like reference strings found
within it. This includes valid whole-value secret URIs and malformed
or embedded secret-like references that should be reported to the
caller. Only canonical ``secret:/...`` URIs should be passed to
``secret_get_versions``.
* - ``scan_unresolved_refs(obj, namespace)``
- Like ``scan_refs`` but returns only references that are missing,
malformed, embedded inside a larger string, or cannot currently be
read successfully. Useful for pre-flight validation.
* - ``resolve_object(obj)``
- Walk a JSON-like object and replace every whole-value ``secret:/...``
URI string with the stored opaque string for the referenced secret.
.. _mgr-ceph-secrets-scan:
Secret URI embedding and resolution
-----------------------------------
Configuration objects that need to reference secrets without embedding
plaintext can store ``secret:/...`` URIs as string values. The module
resolves them on demand:
.. code-block:: python
spec = {
"service_type": "my-service",
"spec": {
"api_token": " secret:/cephadm/service/my-service/api_token ",
},
}
# Check for missing, malformed, embedded, or unresolvable secret references
unresolved = client.scan_unresolved_refs(spec, namespace="cephadm")
if unresolved:
raise RuntimeError(f"Unresolved secret references: {unresolved}")
# Replace URI references with plaintext values
resolved = client.resolve_object(spec)
# resolved["spec"]["api_token"] now holds the stored API token
.. note::
Resolution replaces the entire string value of a field. Surrounding
whitespace around a URI reference is ignored, so values such as
``" secret:/cephadm/global/foo "`` are accepted. Embedding a secret URI
inside a larger string (for example ``"Bearer secret:/..."``) is not
supported; after trimming surrounding whitespace, the field value must be
the URI.
The resolved value is the stored opaque string. The module does not parse
stored data as JSON and does not support field-level URI selection. If a
caller stores structured data, it must encode and decode that structure
itself.
.. _mgr-ceph-secrets-epoch:
Epoch-based change detection
----------------------------
Consumers that maintain a local cache of secrets can use the epoch to avoid
unnecessary full refreshes:
.. code-block:: python
def maybe_refresh(client, namespace, last_epoch):
current_epoch = client.secret_get_epoch(namespace)
if current_epoch == last_epoch:
return last_epoch # nothing changed
# ... do a full refresh ...
return current_epoch
last_epoch = 0
last_epoch = maybe_refresh(client, "cephadm", last_epoch)
The epoch is incremented by every successful ``set`` operation and by
``rm`` only when an existing secret is actually removed (an idempotent
``rm`` returning ``"not_found"`` does not bump the epoch). It starts at 0
for a namespace that has never been written to, and mutations in one namespace
never affect another namespace's epoch.
Configuration
=============
.. mgr_module:: ceph_secrets
.. confval:: secrets_backend
The storage backend used for secrets. Currently only the Monitor KV store
(``mon``) is supported. This option is reserved for future backends
(e.g. HashiCorp Vault).

View File

@ -1176,7 +1176,8 @@ The list of system roles are:
- **cluster-manager**: allows full permissions for the *hosts*, *osd*,
*monitor*, *manager*, and *config-opt* scopes.
- **pool-manager**: allows full permissions for the *pool* scope.
- **cephfs-manager**: allows full permissions for the *cephfs* scope.
- **cephfs-manager**: allows full permissions for the *cephfs* scope, and
*read* permission for the *hosts* and *pool* scopes.
The list of available roles can be retrieved with the following command:

View File

@ -47,5 +47,6 @@ sensible.
MDS Autoscaler module <mds_autoscaler>
NFS module <nfs>
SMB module <smb>
Secrets module <ceph_secrets>
Progress Module <progress>
CLI API Commands module <cli_api>

View File

@ -31,7 +31,7 @@ Create NFS Ganesha Cluster
.. prompt:: bash #
ceph nfs cluster create <cluster_id> [<placement>] [--ingress] [--virtual_ip <value>] [--ingress-mode {default|keepalive-only|haproxy-standard|haproxy-protocol}] [--port <int>] [--enable-rdma] [--rdma_port <int>] [-i <spec_file>]
ceph nfs cluster create <cluster_id> [<placement>] [--ingress] [--virtual_ip <value>] [--ingress-mode {default|keepalive-only|haproxy-standard|haproxy-protocol}] [--ingress-placement <placement>] [--port <int>] [--enable-rdma] [--rdma_port <int>] [--enable-nfsv3] [-i <spec_file>]
This creates a common recovery pool for all NFS Ganesha daemons, new user based on
``cluster_id``, and a common NFS Ganesha config RADOS object.
@ -62,6 +62,9 @@ cluster)::
NFS can be deployed on a port other than 2049 (the default) with ``--port <port>``.
By default, only NFS v4 protocol is enabled. To enable both NFS v3 and v4 protocols,
add the ``--enable-nfsv3`` flag.
To deploy NFS with a high-availability front-end (virtual IP and load balancer), add the
``--ingress`` flag and specify a virtual IP address. This will deploy a combination
of keepalived and haproxy to provide an high-availability NFS frontend for the NFS
@ -108,6 +111,18 @@ of the details of NFS redirecting traffic on the virtual IP to the
appropriate backend NFS servers, and redeploying NFS servers when they
fail.
By default, the *ingress* service follows the same placement as the NFS
Ganesha daemons (the optional ``<placement>`` argument). To schedule
keepalived and HAProxy on a different set of hosts, pass
``--ingress-placement`` with a separate placement string. For example,
to run three NFS daemons on ``host1`` and ``host2`` while colocating
ingress on three dedicated nodes::
ceph nfs cluster create mynfs "2 host1 host2" --ingress --virtual_ip 192.168.1.100/24 --ingress-placement "3 host3 host4 host5"
If ``--ingress-placement`` is omitted, both services share the NFS
placement.
An optional ``--ingress-mode`` parameter can be provided to choose
how the *ingress* service is configured:
@ -277,6 +292,275 @@ This removes the user defined configuration.
for the new config blocks to be effective.
Cluster QoS management
======================
NFS Ganesha supports cluster-wide and per-export Quality of Service (QoS) for
bandwidth and IOPS (operations per second).
.. note:: Cluster-level QoS changes take effect after the NFS service is
restarted. The ``ceph nfs cluster qos enable`` and ``ceph nfs cluster qos
disable`` commands (for both bandwidth control and IOPS control), as well as
``ceph nfs cluster qos set``, restart all NFS Ganesha daemons in the
cluster automatically via ``ceph orch restart nfs.<cluster_id>``. Plan for
a brief service interruption when running these commands on a live cluster.
``qos_type`` values
-------------------
All ``qos enable`` commands require a ``qos_type`` argument. Valid values are:
* ``PerShare`` — limits apply per NFS export (share).
* ``PerClient`` — limits apply per NFS client.
* ``PerShare_PerClient`` — limits apply per export and per client.
Parameter constraints
---------------------
Bandwidth parameters (``max_*_bw``) accept human-readable values with the
following units: ``KiB``, ``MiB``, ``GiB``, ``KB``, ``MB`` and ``GB``
(for example, ``100MB`` or ``128KiB``). The valid range is
128 KiB/s to 100 GiB/s.
IOPS parameters (``max_*_iops``) must be integers in the range 10 to 1638400.
The cluster QoS message interval (``cqos_msg_interval``) can be set with
``ceph nfs cluster qos set``. The valid range is 100 to 300 milliseconds.
Deploy-time QoS via service spec
--------------------------------
When deploying an NFS cluster with cephadm, cluster-level QoS can be configured
at creation time using the ``cluster_qos_config`` field in the NFS service
spec, or via a command of the form ceph nfs cluster create -i foo.yml with a YAML input file.
``cluster_qos_config`` is a dictionary. At least one of
``enable_bw_control`` or ``enable_iops_control`` must be ``true``. Supported
keys:
* ``enable_qos`` (bool, default ``true``)
* ``qos_type`` (required when QoS is enabled): ``PerShare``, ``PerClient``, or
``PerShare_PerClient``
* ``enable_bw_control`` (bool)
* ``combined_rw_bw_control`` (bool)
* ``enable_iops_control`` (bool)
* ``max_export_write_bw``, ``max_export_read_bw``, ``max_client_write_bw``,
``max_client_read_bw`` (string bandwidth values)
* ``max_export_combined_bw``, ``max_client_combined_bw`` (string bandwidth
values)
* ``max_export_iops``, ``max_client_iops`` (integer)
* ``cqos_msg_interval`` (integer, 100300 milliseconds)
The NFS service spec also accepts ``cluster_qos_port`` (default ``31311``) to specify
the cluster QoS messaging port used by Ganesha daemons.
Example service spec with cluster QoS:
.. code-block:: yaml
service_type: nfs
service_id: mynfs
placement:
hosts:
- host1
spec:
port: 2049
cluster_qos_port: 31311
cluster_qos_config:
qos_type: PerShare
enable_bw_control: true
combined_rw_bw_control: false
max_export_write_bw: 100MB
max_export_read_bw: 100MB
Example ``ceph nfs cluster create`` input file:
.. code-block:: yaml
cluster_qos_config:
qos_type: PerShare
enable_bw_control: true
max_export_write_bw: 100MB
max_export_read_bw: 100MB
.. code:: bash
$ ceph nfs cluster create mynfs "host1" -i cluster_qos.yaml
Enable QoS bandwidth control for an NFS Ganesha cluster
-------------------------------------------------------
.. code:: bash
$ ceph nfs cluster qos enable bandwidth_control <cluster_id> <qos_type:PerShare|PerClient|PerShare_PerClient> [--combined-rw-bw-ctrl] [--max_export_write_bw <value>] [--max_export_read_bw <value>] [--max_client_write_bw <value>] [--max_client_read_bw <value>] [--max_export_combined_bw <value>] [--max_client_combined_bw <value>]
This command enables or updates Quality of Service (QoS) bandwidth control for
an NFS Ganesha cluster, where
``<cluster_id>`` is the NFS Ganesha cluster ID.
``<qos_type>`` is the type of bandwidth control: ``PerShare``, ``PerClient``, or
``PerShare_PerClient``.
If ``PerShare`` ``qos_type`` is selected, then the cluster-level QoS config is
applicable to all exports on that NFS Ganesha cluster. It requires
``max_export_write_bw`` and ``max_export_read_bw`` parameters if
``--combined-rw-bw-ctrl`` is not set, otherwise ``max_export_combined_bw`` is
required.
If ``PerClient`` ``qos_type`` is selected, then the cluster-level QoS config is
applicable to all clients accessing exports on that cluster. It requires
``max_client_write_bw`` and ``max_client_read_bw`` parameters if
``--combined-rw-bw-ctrl`` is not set, otherwise ``max_client_combined_bw`` is
required.
If ``PerShare_PerClient`` ``qos_type`` is selected, then the cluster-level
config applies to all exports and all clients on that NFS Ganesha cluster.
It requires ``max_export_write_bw``, ``max_export_read_bw``,
``max_client_write_bw``, and ``max_client_read_bw`` parameters if
``--combined-rw-bw-ctrl`` is not set, otherwise ``max_export_combined_bw`` and
``max_client_combined_bw`` are required.
``--combined-rw-bw-ctrl`` enables combined read and write bandwidth. When set,
only the combined bandwidth parameters allowed for the selected ``qos_type``
may be specified.
``--max_export_write_bw`` is the maximum write bandwidth for each export.
``--max_export_read_bw`` is the maximum read bandwidth for each export.
``--max_client_write_bw`` is the maximum write bandwidth for each client.
``--max_client_read_bw`` is the maximum read bandwidth for each client.
``--max_export_combined_bw`` is the maximum combined read/write bandwidth for each export.
``--max_client_combined_bw`` is the maximum combined read/write bandwidth for each client.
The bandwidth value can be specified using any supported bandwidth unit: bytes/s,
KB/s, KiB/s, MB/s, MiB/s, GB/s, or GiB/s.
For example::
$ ceph nfs cluster qos enable bandwidth_control nfs_clust PerShare --max_export_write_bw 100MB --max_export_read_bw 100MB
.. note:: If this command is used to update ``qos_type``, update all exports
with the required parameters as well.
By default, exports inherit the cluster-level QoS setting when no
``QOS_BLOCK`` is present in the export block.
Disable QoS bandwidth control for NFS Ganesha cluster
-----------------------------------------------------
.. code:: bash
$ ceph nfs cluster qos disable bandwidth_control <cluster_id>
This command disables bandwidth control QoS at the cluster level. If
cluster-level bandwidth control is disabled, export-level bandwidth control
has no effect even when enabled on an export.
For example::
$ ceph nfs cluster qos disable bandwidth_control nfs_clust
Enable QoS IOPS control for NFS Ganesha cluster
-----------------------------------------------
.. code:: bash
$ ceph nfs cluster qos enable ops_control <cluster_id> <qos_type:PerShare|PerClient|PerShare_PerClient> [--max_export_iops <int>] [--max_client_iops <int>]
This command enables or updates IOPS control for an NFS cluster.
``<cluster_id>`` is the NFS Ganesha cluster ID.
``<qos_type>`` is the type of ops control: ``PerShare``, ``PerClient``, or
``PerShare_PerClient``.
If ``PerShare`` ``qos_type`` is selected, the cluster-level QoS config applies
to all exports on that NFS Ganesha cluster and requires ``--max_export_iops``.
If ``PerClient`` ``qos_type`` is selected, the cluster-level QoS config
applies to all clients accessing exports on that cluster and requires
``--max_client_iops``.
If ``PerShare_PerClient`` ``qos_type`` is selected, the cluster-level config
applies to all exports and all clients on that NFS Ganesha cluster and
requires both ``--max_export_iops`` and ``--max_client_iops``.
``--max_export_iops`` is the IOPS limit per export.
``--max_client_iops`` is the IOPS limit per client.
For example::
$ ceph nfs cluster qos enable ops_control nfs_clust PerShare --max_export_iops 1000
.. note:: If this command is used to update ``qos_type``, update all exports
with the required parameters as well.
Disable QoS IOPS control for NFS Ganesha cluster
------------------------------------------------
.. code:: bash
$ ceph nfs cluster qos disable ops_control <cluster_id>
This command disables IOPS control for an NFS Ganesha cluster. After disabling
ops control at the cluster level, export-level IOPS control has no effect even
when enabled on an export.
For example::
$ ceph nfs cluster qos disable ops_control nfs_clust
Set cluster QoS message interval
--------------------------------
.. code:: bash
$ ceph nfs cluster qos set <cluster_id> <msg_interval>
This command sets ``cqos_msg_interval``, the message interval (in
milliseconds) used for cluster QoS synchronization among NFS Ganesha hosts.
The valid range is 100 to 300. Cluster-level QoS must already be configured
(using ``enable bandwidth_control`` or ``enable ops_control``) before this
command can be used.
For example::
$ ceph nfs cluster qos set nfs_clust 200
Get QoS configuration for NFS Ganesha cluster
---------------------------------------------
.. code:: bash
$ ceph nfs cluster qos get <cluster_id>
This command displays the cluster-level QoS configuration. Use ``ceph -f
json`` (or ``json-pretty``, ``yaml``, etc.) to control output formatting.
For example::
$ ceph nfs cluster qos get nfs_clust
{
"combined_rw_bw_control": false,
"enable_bw_control": true,
"enable_iops_control": true,
"enable_qos": true,
"max_export_iops": 1000,
"max_export_read_bw": "100.0MB",
"max_export_write_bw": "100.0MB",
"qos_type": "PerShare"
}
When ``cqos_msg_interval`` has been set, it also appears in the output.
Export Management
=================
@ -475,6 +759,147 @@ This displays export block for a cluster based on pseudo root name, where:
``<pseudo_path>`` is the pseudo root path (must be an absolute path).
Enable QoS bandwidth control for a specific export
--------------------------------------------------
.. code:: bash
$ ceph nfs export qos enable bandwidth_control <cluster_id> <pseudo_path> [--combined-rw-bw-ctrl] [--max_export_write_bw <value>] [--max_export_read_bw <value>] [--max_client_write_bw <value>] [--max_client_read_bw <value>] [--max_export_combined_bw <value>] [--max_client_combined_bw <value>] [--skip-notify-nfs-server]
This command enables or updates QoS bandwidth control for an export. Enable
cluster-level bandwidth control with ``qos_type`` ``PerShare`` or
``PerShare_PerClient`` before enabling export-level bandwidth control. This
creates a ``QOS_BLOCK`` in the export block, where
``<cluster_id>`` is the NFS Ganesha cluster ID.
``<pseudo_path>`` is the pseudo-root path (must be an absolute path).
``--combined-rw-bw-ctrl`` enables combined read and write bandwidth. When set,
only the combined bandwidth parameters allowed for the cluster ``qos_type`` may
be specified.
``--max_export_write_bw`` is the maximum write bandwidth for each export.
``--max_export_read_bw`` is the maximum read bandwidth for each export.
``--max_client_write_bw`` is the maximum write bandwidth for each client.
``--max_client_read_bw`` is the maximum read bandwidth for each client.
``--max_export_combined_bw`` is the maximum combined read/write bandwidth for each export.
``--max_client_combined_bw`` is the maximum combined read/write bandwidth for each client.
``--skip-notify-nfs-server`` skips notifying running NFS Ganesha daemons after
the change (useful for batch updates).
The bandwidth value can be specified using any supported bandwidth unit: bytes/s,
KB/s, KiB/s, MB/s, MiB/s, GB/s, or GiB/s.
For example::
$ ceph nfs export qos enable bandwidth_control nfs_clust /export1 --combined-rw-bw-ctrl --max_export_combined_bw 200MB
.. note:: Export-level bandwidth control cannot be enabled if the cluster-level
``qos_type`` is ``PerClient``.
Disable QoS bandwidth control for a specific export
---------------------------------------------------
.. code:: bash
$ ceph nfs export qos disable bandwidth_control <cluster_id> <pseudo_path> [--skip-notify-nfs-server]
This disables export-level bandwidth control. After disabling, the export no
longer has export-specific bandwidth limits. If the export has no
``QOS_BLOCK``, it inherits cluster-level bandwidth control again.
For example::
$ ceph nfs export qos disable bandwidth_control nfs_clust /export1
.. note:: To restore cluster-level QoS defaults for an export, use
``ceph nfs export apply <cluster_id>`` with an export definition that does
not include a ``QOS_BLOCK``.
Enable QoS IOPS control for a specific export
---------------------------------------------
.. code:: bash
$ ceph nfs export qos enable ops_control <cluster_id> <pseudo_path> [--max_export_iops <int>] [--max_client_iops <int>] [--skip-notify-nfs-server]
This enables IOPS control for a specified export. The same command can be used
to update an existing IOPS limit. Enable cluster-level IOPS control with ``qos_type``
``PerShare`` or ``PerShare_PerClient`` before enabling export-level IOPS control.
``<cluster_id>`` is the NFS Ganesha cluster ID.
``<pseudo_path>`` is the pseudo-root path (must be an absolute path).
``--max_export_iops`` is the IOPS limit for the export.
``--max_client_iops`` is the IOPS limit per client of the export.
``--skip-notify-nfs-server`` skips notifying running NFS Ganesha daemons after
the change.
For example::
$ ceph nfs export qos enable ops_control nfs_clust /export1 --max_export_iops 2000
.. note:: Export-level IOPS control cannot be enabled if the cluster-level
``qos_type`` is ``PerClient``.
Disable QoS IOPS control for a specific export
----------------------------------------------
.. code:: bash
$ ceph nfs export qos disable ops_control <cluster_id> <pseudo_path> [--skip-notify-nfs-server]
This command disables export-level IOPS control. After disabling, the export no
longer has export-specific IOPS limits.
For example::
$ ceph nfs export qos disable ops_control nfs_clust /export1
Get QoS configuration for export
--------------------------------
.. code:: bash
$ ceph nfs export qos get <cluster_id> <pseudo_path>
This command displays the export-level QoS configuration. Use ``ceph -f
json`` (or ``json-pretty``, ``yaml``, ``xml``, ``xml-pretty``) to control output formatting.
When cluster-level QoS is configured, fields prefixed with ``global_`` show
the cluster defaults (``global_enable_qos``, ``global_enable_bw_control``,
``global_enable_iops_control``). Export-specific fields (without the
``global_`` prefix) are present only when a ``QOS_BLOCK`` exists on the
export. If the export has no ``QOS_BLOCK``, an empty object ``{}`` is
returned.
For example::
$ ceph nfs export qos get nfs_clust /export1
{
"global_enable_bw_control": true,
"global_enable_iops_control": true,
"global_enable_qos": true,
"combined_rw_bw_control": true,
"enable_bw_control": true,
"enable_iops_control": true,
"enable_qos": true,
"max_export_combined_bw": "200.0MB",
"max_export_iops": 2000
}
Export QoS can also be set via ``ceph nfs export apply`` by including a
``qos_block`` key in the export JSON, or a ``QOS_BLOCK`` section in a Ganesha
EXPORT config fragment.
Create or update export via JSON specification
----------------------------------------------

View File

@ -113,10 +113,19 @@ Creating/growing/shrinking/removing services:
ceph orch apply mds <fs_name> [--placement=<placement>] [--dry-run]
ceph orch apply rgw <name> [--realm=<realm>] [--zone=<zone>] [--port=<port>] [--ssl] [--placement=<placement>] [--dry-run]
ceph orch apply nfs <name> <pool> [--namespace=<namespace>] [--placement=<placement>] [--dry-run]
ceph orch rm <service_name> [--force]
ceph orch rm <service_name> [--force] [--force-delete-data]
where ``placement`` is a :ref:`orchestrator-cli-placement-spec`.
For ``ceph orch rm``, ``--force`` may be required in some cases (for example when
removing an OSD service that would leave OSDs behind).
The ``--force-delete-data`` flag requires that ``--force`` is also passed.
This directs cephadm to delete on-disk data for certain daemon types instead
of moving it to ``<fsid>/removed/`` on the host. These daemon types include
``mon``, ``osd``, and ``prometheus``. This helps avoid filling up the
underlying filesystem over time.
e.g., ``ceph orch apply mds myfs --placement="3 host1 host2 host3"``
Service Commands:

View File

@ -24,6 +24,19 @@ Enable the ``prometheus`` module by running the below command :
ceph mgr module enable prometheus
ceph-exporter
=============
``ceph-exporter`` works alongside the Prometheus manager module. The two
components have distinct responsibilities:
- The **Prometheus manager module** exposes all cluster-level metrics by
default other than Ceph daemon performance counters. However, these metrics
may be exported by the Prometheus manager module by setting the module option
:confval:`mgr/prometheus/exclude_perf_counter` to `false`.
- The **ceph-exporter daemon** exposes only Ceph daemon performance counters
as Prometheus metrics, running on each host in the cluster.
Configuration
-------------

View File

@ -55,7 +55,7 @@ Create Cluster
.. prompt:: bash #
ceph smb cluster create <cluster_id> {user|active-directory} [--domain-realm=<domain_realm>] [--domain-join-user-pass=<domain_join_user_pass>] [--define-user-pass=<define_user_pass>] [--custom-dns=<custom_dns>] [--placement=<placement>] [--clustering=<clustering>] [--password-filter=<password_filter>] [--password-filter-out=<password_filter_out>]
ceph smb cluster create <cluster_id> {user|active-directory} [--domain-realm=<domain_realm>] [--domain-join-user-pass=<domain_join_user_pass>] [--define-user-pass=<define_user_pass>] [--custom-dns=<custom_dns>] [--placement=<placement>] [--clustering=<clustering>] [--client-compat={default|macos}] [--password-filter=<password_filter>] [--password-filter-out=<password_filter_out>]
Create a new logical cluster, identified by the cluster ID value. The cluster
create command must specify the authentication mode the cluster will use. This
@ -100,6 +100,11 @@ clustering
enables clustering regardless of the placement count. A value of ``never``
disables clustering regardless of the placement count. If unspecified,
``default`` is assumed.
client_compat
Optional. One of ``default`` or ``macos``. Controls client-specific SMB
features and optimizations. The ``default`` mode provides standard SMB
behavior with broad compatibility. The ``macos`` mode enables macOS-specific
optimized settings for macOS clients. If unspecified, ``default`` is assumed.
public_addrs
Optional. A string in the form of <ipaddress/prefixlength>[%<destination address>].
Supported only when using Samba's clustering. Assign "virtual" IP addresses
@ -165,6 +170,15 @@ previous example. Set three CTDB public address values and a custom placement:
--public-address=192.168.76.112/24 \
--placement="3 label:smb"
Create a cluster for macOS clients with user authentication:
.. prompt:: bash #
ceph smb cluster create mac_cluster user \
--define-user-pass=macuser%Passw0rd1 \
--client-compat=macos \
--placement="label:smb"
Update Cluster QoS
++++++++++++++++++
@ -216,6 +230,53 @@ Disable QoS for all shares in a cluster:
--write-bw-limit=0
Update Cluster Client Compatibility
++++++++++++++++++++++++++++++++++++
.. prompt:: bash #
ceph smb cluster update client-compat {default|macos} <cluster_id>
Update the client compatibility mode for an SMB cluster. This setting controls whether client-specific SMB features and optimizations are enabled.
The client compatibility mode determines how the Samba server is configured to optimize for specific client types:
- ``default``: Standard SMB behavior without client-specific optimizations. This is the default mode and provides broad compatibility with all SMB clients.
- ``macos``: Enable macOS-specific features including the Samba's fruit VFS module for proper handling of macOS metadata and optimized settings for macOS clients.
When ``macos`` mode is enabled, the following features are automatically configured:
- Fruit VFS module for proper handling of macOS-specific file attributes
- Streams_xattr VFS module for extended attribute support
Options:
client_compat
One of ``default`` or ``macos``
cluster_id
A short string uniquely identifying the cluster
Examples
~~~~~~~~
Enable macOS compatibility mode for a cluster:
.. prompt:: bash #
ceph smb cluster update client-compat macos mycluster
Revert to default (standard) compatibility mode:
.. prompt:: bash #
ceph smb cluster update client-compat default mycluster
.. note::
The ``macos`` compatibility mode is recommended when the primary clients accessing the SMB shares are macOS systems. Otherwise default mode is recommended.
Remove Cluster
++++++++++++++
@ -321,6 +382,51 @@ Create a read-only share at a custom path in the CephFS volume:
--path=/qbranch/top/secret/plans --readonly
Create RGW Share
++++++++++++++++
.. prompt:: bash #
ceph smb share create rgw <cluster_id> <share_id> <bucket> [--share-name=<share_name>] [--user-id=<user_id>] [--readonly]
Create a new SMB share, hosted by the named cluster, that maps to a RADOS
Gateway (RGW) bucket. This allows S3-compatible object storage to be accessed
via the SMB protocol.
Options:
cluster_id
A short string uniquely identifying the cluster
share_id
A short string uniquely identifying the share
bucket
The name of the RGW bucket to be shared
share_name
Optional. The public name of the share, visible to clients. If not provided
the ``share_id`` will be used automatically
user_id
Optional. The RGW user ID that owns the bucket. If not provided, the system
will attempt to determine the bucket owner automatically
readonly
Creates a read-only share
Examples
~~~~~~~~
Create a share for an RGW bucket:
.. prompt:: bash #
ceph smb share create rgw test1 photos my-photos-bucket
Create a share with a custom name and specific user:
.. prompt:: bash #
ceph smb share create rgw test1 photos my-photos-bucket \
--share-name="Photo Archive" --user-id=s3user
.. _qos-parameters:
QoS Parameters
@ -879,6 +985,12 @@ custom_smb_global_options
indicator that the user is aware that using this option can easily break
things in ways that the Ceph team can not help with. This special key will
automatically be removed from the list of options passed to Samba.
client_compat
Optional. One of ``default`` or ``macos``. Controls client-specific SMB
features and optimizations. The ``default`` mode provides standard SMB
behavior with broad compatibility. The ``macos`` mode enables macOS-specific
features including Samba's fruit VFS module for proper handling of macOS and
optimized settings for macOS clients. If unspecified, ``default`` is assumed.
.. warning::
Setting the ``clustering`` option allows an administrator to choose exactly
@ -977,6 +1089,20 @@ The following is an example of a cluster configured for standalone operation:
hosts:
- node6.mycluster.sink.test
The following is an example of a cluster optimized for macOS clients:
.. code-block:: yaml
resource_type: ceph.smb.cluster
cluster_id: macshare
auth_mode: user
client_compat: macos
user_group_settings:
- source_type: resource
ref: ug1
placement:
count: 1
An example cluster resource with intent to remove:
.. code-block:: yaml
@ -1018,7 +1144,8 @@ max_connections
connections to a specific share. The default value is 0 and it indicates
that there is no limit on the number of connections
cephfs
Required object. Fields:
Object. Configures CephFS-backed storage for the share. Either a ``cephfs``
or ``rgw`` object must be specified, but not both. Fields:
volume
Required string. Name of the cephfs volume to use
@ -1088,6 +1215,22 @@ cephfs
name
String. A value indicating what FSCrypt key to fetch. The specific
value of the name depends on the scope being used.
rgw
Object. Configures RADOS Gateway (RGW) backed storage for the share. Either
a ``cephfs`` or ``rgw`` object must be specified, but not both. This allows
S3-compatible object storage to be accessed via the SMB protocol. Fields:
bucket
Required string. The name of the RGW bucket to be shared.
user_id
Optional string. The RGW user ID that owns the bucket. If not provided,
the system will automatically determine the bucket owner and fetch the
necessary credentials.
credential_ref
Optional string. The ``rgw_credential_id`` value of a
``ceph.smb.rgw.credential`` resource that contains RGW access and
secret key values needed to use the given bucket.
restrict_access
Optional boolean, defaulting to false. If true the share will only permit
access by users explicitly listed in ``login_control``.
@ -1135,7 +1278,32 @@ custom_smb_share_options
things in ways that the Ceph team can not help with. This special key will
automatically be removed from the list of options passed to Samba.
The following is an example of a share with QoS settings including burst
The following is an example of an RGW-backed share with minimal configuration
(credentials auto-fetched):
.. code-block:: yaml
resource_type: ceph.smb.share
cluster_id: tango
share_id: s3share
name: "S3 Storage"
rgw:
bucket: my-bucket
Another example of an RGW-backed share with explicit user_id:
.. code-block:: yaml
resource_type: ceph.smb.share
cluster_id: tango
share_id: s3share
name: "S3 Storage"
rgw:
bucket: my-bucket
user_id: s3user
The following is an example of a CephFS share with QoS settings including burst
multipliers and human-readable bandwidth limits:
.. code-block:: yaml
@ -1285,6 +1453,43 @@ Example:
groups: []
RGW Credential Resource
------------------------
An RGW credential resource stores RADOS Gateway (RGW) access credentials that can be used by RGW-backed shares to authenticate with the object storage system.
A RGW credential resource supports the following fields:
resource_type
A literal string ``ceph.smb.rgw.credential``
rgw_credential_id
A short string identifying the RGW credential resource.
intent
One of ``present`` or ``removed``. If not provided, ``present`` is assumed.
If ``removed`` all following fields are optional
user_id
Required string. The RGW user ID that owns the credentials
access_key_id
Required string. The RGW access key for authentication
secret_access_key
Required string. The RGW secret key for authentication
linked_to_cluster:
Optional. A string containing a cluster ID. If set, the resource may only
be used with the linked cluster and will automatically be removed when the
linked cluster is removed.
Example:
.. code-block:: yaml
resource_type: ceph.smb.rgw.credential
rgw_credential_id: s3user
user_id: s3user
access_key: AKIAIOSFODNN7EXAMPLE
secret_key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
TLS Credential Resource
------------------------
@ -1362,14 +1567,122 @@ fsid
mon_host
String. The ``mon_host`` string (as sourced from a ceph.conf file)
cephfs_user
Object. Fields:
Optional object. Required if the cluster will host CephFS-backed shares.
Fields:
name
String. A ceph user name indicating the cephx user that will
String. A Ceph user name indicating the CephX user that will
access the CephFS volume(s) on the external cluster
key
String. The Base64 encoded key value corresponding to the cephx
String. The Base64 encoded key value corresponding to the CephX
user name provided
rgw_user
Optional object. Required if the cluster will host RGW-backed shares.
Fields:
name
String. A Ceph user name indicating the CephX user that will
access the RGW bucket(s) on the external cluster
key
String. The Base64 encoded key value corresponding to the CephX
user name provided
.. note::
At least one of ``cephfs_user`` or ``rgw_user`` must be configured.
Configure only the user(s) needed for your share types:
- CephFS-only clusters: Configure only ``cephfs_user``
- RGW-only clusters: Configure only ``rgw_user``
- Mixed clusters: Configure both ``cephfs_user`` and ``rgw_user``
Examples
~~~~~~~~
External cluster with CephFS support only:
.. code-block:: yaml
resource_type: ceph.smb.ext.cluster
external_ceph_cluster_id: external-cephfs
cluster:
fsid: "12345678-1234-1234-1234-123456789abc"
mon_host: "10.0.1.10:6789,10.0.1.11:6789"
cephfs_user:
name: "client.external-cephfs"
key: "AQC1234567890abcdefghijklmnopqrstuvwxyz=="
External cluster with RGW support only:
.. code-block:: yaml
resource_type: ceph.smb.ext.cluster
external_ceph_cluster_id: external-rgw
cluster:
fsid: "12345678-1234-1234-1234-123456789abc"
mon_host: "10.0.1.10:6789,10.0.1.11:6789"
rgw_user:
name: "client.external-rgw"
key: "AQD9876543210zyxwvutsrqponmlkjihgfedcba=="
External cluster with both CephFS and RGW support:
.. code-block:: yaml
resource_type: ceph.smb.ext.cluster
external_ceph_cluster_id: external-mixed
cluster:
fsid: "12345678-1234-1234-1234-123456789abc"
mon_host: "10.0.1.10:6789,10.0.1.11:6789"
cephfs_user:
name: "client.external-cephfs"
key: "AQC1234567890abcdefghijklmnopqrstuvwxyz=="
rgw_user:
name: "client.external-rgw"
key: "AQD9876543210zyxwvutsrqponmlkjihgfedcba=="
.. important::
**RGW Shares on External Clusters Require Manual Credentials**
When creating RGW shares on external clusters, you **must** manually define
RGW credentials using the ``ceph.smb.rgw.credential`` resource. Unlike local
clusters where credentials can be auto-fetched, external clusters cannot
automatically retrieve S3 access keys.
Example configuration with manual RGW credentials:
.. code-block:: yaml
resources:
# Define the external cluster with rgw_user
- resource_type: ceph.smb.ext.cluster
external_ceph_cluster_id: external-rgw
cluster:
fsid: "12345678-1234-1234-1234-123456789abc"
mon_host: "10.0.1.10:6789,10.0.1.11:6789"
rgw_user:
name: "client.external-rgw"
key: "AQD9876543210zyxwvutsrqponmlkjihgfedcba=="
# Define RGW credentials manually (REQUIRED for external clusters)
- resource_type: ceph.smb.rgw.credential
rgw_credential_id: my-s3-creds
user_id: s3-user
access_key_id: AKIAIOSFODNN7EXAMPLE
secret_access_key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
# RGW share referencing the manual credentials
- resource_type: ceph.smb.share
cluster_id: my-cluster
share_id: s3-share
name: "S3 Share"
rgw:
bucket: my-bucket
credential_ref: my-s3-creds
The ``rgw_user`` in the external cluster definition provides the CephX
credentials for accessing the RGW service, while the ``ceph.smb.rgw.credential``
resource provides the S3 access key and secret key for bucket operations.
A Declarative Configuration Example
@ -1483,10 +1796,11 @@ cluster has been configured for at least one share. The ``placement`` field of
the cluster resource is passed onto the orchestration layer and is used to
determine on what nodes of the Ceph cluster Samba containers will be run.
At this time Samba services can only listen on port 445. Due to this
restriction only one Samba server, as part of one cluster, may run on a single
Ceph node at a time. Ensure that the placement specs on each cluster do not
overlap.
The Samba containers may run on the same hosts if, and only if, the services
use different IP addresses and/or ports. If the placement spec allows more than
one container to run on the same host, use the ``bind_addrs`` field, the
``custom_ports`` field, or some combination, in the cluster resource to ensure
that the Samba server instances do not conflict.
The ``smb`` clusters are fully isolated from each other. This means that, as
long as you have sufficient resources in your Ceph cluster, you can run multiple

Some files were not shown because too many files have changed in this diff Show More