crimson/osd: fix backfill deadlock in BackfillState::Waiting
BackfillState::Enqueuing unconditionally posts RequestWaiting{}
when budget available return false. Waiting can exit only via
ObjectPushed which requires an in-flight push. If the budget
is exhausted with no pushes in flight no objectPush will ever
arrive and the PG stalls permanently.
Solution: Introduce a dedicated BudgetBlocked state. When
budget_available() is false and tracked_objects_completed()==true
post RequestBudgetBlocked{} instead of RequestWaiting{}.
BudgetBlocked calls get_backfill_throttle() to suspend until a slots
free up, then fires BudgetAvailable{} which transitions directly back
to Enqueuing to dispatch the next batch of pushes.
Fixes: https://tracker.ceph.com/issues/77804
librdkafka submodule
Vendors librdkafka v2.12.1 as a submodule (default) and teaches the
notification teuthology task to set up Kafka 4.x KRaft brokers. The
distro-shipped librdkafka 1.6.1 cannot SCRAM-auth against Kafka 4.x
brokers (#75900); the bundled version fixes that. Build with
WITH_SYSTEM_RDKAFKA=ON to opt back into the distro library (floor 2.6.1).
follwinf changes needed with teh submodule:
* build kafka_stub only in case of system librdkafka
* use the same header location in submodule and system
* supress compilation warning to avoid -WError issues
2.12.1 matches Fedora(44)'s current librdkafka-devel, so the vendored copy
tracks a version that distro packagers have already vetted rather than
whichever release happens to be latest upstream.
The teuthology version matrix picks one of {3.9.2, 4.1, 4.2, 4.3} per run.
3.9.2 covers the last Zookeeper-based release for the migration cohort.
4.1/4.2/4.3 track Apache's current three supported minor lines and are
resolved to the latest patch at test time via dlcdn.apache.org/kafka/, so
new patches within a supported minor need no PR update. A new minor line
(4.4) requires renaming one YAML.
* new java version (17) is needed for kafka 4.2 and up (we wil use it on
all version)
* kerberos task need to run before the kafka install task
After pulling this branch, fetch the new submodule:
git submodule update --init src/librdkafka
Fixes: https://tracker.ceph.com/issues/77336
Signed-off-by: ShreeJejurikar <shreemj8@gmail.com>
Co-authored-by: Yuval Lifshitz <ylifshit@ibm.com>
mgr/dashboard: restricting admin user to revoke its own admin permissions
Reviewed-by: Afreen Misbah <afreen@ibm.com>
Reviewed-by: Sagar Gopale <sagar.gopale@ibm.com>
I checked a couple of gherkin linters as an alternative but all of them
are not really well maintained. There are couple of new projeects but
even they are simply maintained by a single guy so i don't think it'll
get enough contributions later on. So instead, I am moving it to
prettier. Although we miss some functional validation, it'll take care
of some basic formatting.
Fixes: https://tracker.ceph.com/issues/78528
Signed-off-by: Nizamudeen A <nia@redhat.com>
Extract the Redmine auth check added in the previous commit into a
standalone verify_redmine_auth(R) function so it can be unit tested
in isolation, and add src/script/test_ptl_tool.py covering:
- verify_redmine_auth(): invalid key (AuthError), a key that
authenticates but lacks permission (ForbiddenError), a valid key,
and confirming unrelated Redmine errors (ServerError) still
propagate normally rather than being misreported as a credentials
problem.
- AuditReport.post_consolidated_review(): logs success on 2xx, logs
an error (rather than failing silently) on a non-2xx GitHub
response, never calls out to GitHub under --dry-run, and is a
no-op when the report has no issues to post.
No behavior change to verify_redmine_auth() itself -- this is a pure
extraction plus tests. ptl-tool.py has no existing test suite, so
these are self-contained (mocked Redmine/requests, no network access,
no real git checkout needed) and run with plain pytest; see the
module docstring for the exact invocation.
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
Invalid credentials were handled inconsistently:
- A missing GitHub/Redmine token was caught with a clear error at
startup, but an invalid or expired one was not: the Redmine
client is constructed lazily by python-redmine, so a bad
~/.redmine_key was never actually exercised until
manage_qa_tracker() called R.project.get() -- which runs *after*
PRs have been merged and the integration branch/tag has already
been pushed to ceph-ci. The result was an unhandled traceback
well after real side effects had already happened, with no
actionable message.
- post_consolidated_review() posted the consolidated audit review
to GitHub without checking the response status at all, so a bad
or under-scoped token would fail silently there while every other
GitHub write call in the file already checks status_code.
Fix both:
- Immediately after constructing the Redmine client, make a cheap
R.user.get('current') call and catch AuthError/ForbiddenError,
exiting with a clear message before any merging or pushing
happens.
- Check the response status in post_consolidated_review() and log
an error on failure, matching the pattern already used by the
other GitHub POST call sites in this file.
Verified with py_compile plus live --dry-run runs against
tracker.ceph.com: a deliberately invalid PTL_TOOL_REDMINE_API_KEY
now exits 1 immediately with a clear message and no git side
effects, while a valid key proceeds exactly as before.
Signed-off-by: Yuri Weinstein <yweinste@redhat.com>
rgw_gc: don't fail when processing entry with nonexisting pool
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Pritha Srivastava <prsrivas@redhat.com>
mgr/smb: add support for RGW shares with external Ceph clusters
Reviewed-by: Avan Thakkar <athakkar@redhat.com>
Reviewed-by: Anoop C S <anoopcs@cryptolab.net>
Reviewed-by: John Mulligan <jmulligan@redhat.com>
Block-manager users were denied access to the NVMe/TCP page because
host listing APIs require hosts read permission.
Fixes: https://tracker.ceph.com/issues/77952
Signed-off-by: Abhishek Desai <adesai@redhat.com>
