This commit removes centos9 from crimson's supported distros. This is in
line with the wider ceph moving on to rocky10 from centos9. We have
established that crimson is compatible with rocky10. More details can be
found in this tracker: https://tracker.ceph.com/issues/75823.
Signed-off-by: Shraddha Agrawal <shraddha.agrawal000@gmail.com>
Add --logging-type flag to run the Python bucket logging test suite
in either Standard or Journal mode. The same tests run against both
logging types with no changes to test logic or assertions.
- Add --logging-type pytest CLI option (Standard default, Journal opt-in)
- Detect boto3 LoggingType extension availability at session startup
- Thread logging_type through helpers and test functions
- Add teuthology task YAML for Journal mode suite runs
- Install service-2.sdk-extras.json in the teuthology task when
logging_type is Journal (s3tests cleans it up after its own run,
so the file isn't available by the time our Journal job runs)
- Document Journal mode local usage in the test suite README
Signed-off-by: Shree Jejurikar <shree.jejurikar@gmail.com>
Add cloud_target_by_bucket and cloud_target_by_bucket_prefix options
to rgw_cloudtier.py and s3tests.py. Create new test suite to run
target_by_bucket-specific s3-tests.
Signed-off-by: Matthew N. Heler <matthew.heler@hotmail.com>
used in testing. This translates into more segments, which helps
in preventing test failures due to insufficient free segments for mounting.
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
The objectstore tool tests restart the OSDs without allowing enough
time for GC to run, which can lead to no-OOL-segments conditions on restart. This
adds a gc_before_restart option to the test config, which when set
to true will run crimson-objectstore-tool --op gc on each OSD
before restarting them.
Fixes: https://tracker.ceph.com/issues/73101
Signed-off-by: Ronen Friedman <rfriedma@redhat.com>
Add a crimson workunit that validates pg query and list_unfound
including offset behavior, and wire it into the singleton suite.
Signed-off-by: Kautilya Tripathi <kautilya.tripathi@ibm.com>
Add a dedicated crimson-rados singleton suite yaml to run the
osd/pg-subcommands.sh standalone test.
Signed-off-by: Kautilya Tripathi <kautilya.tripathi@ibm.com>
This adds test script for pg subcommands like query, log, scrub,
deep-scrub, list-unfound. It compares the output between classic and
crimson.
Signed-off-by: Kautilya Tripathi <kautilya.tripathi@ibm.com>
When building with ASan, it reports leaks in five tests that exercise
CPython 3.10 either as an external process (bin/ceph) or embedded in
unittests:
#36 smoke.sh (timeout, via bin/ceph)
#205 unittest_mgr_pyformatter (libpython3.10.so embedded)
#206 unittest_mgr_pyutil (libpython3.10.so embedded)
#210 mgr-dashboard-smoke.sh (via bin/ceph)
#225 safe-to-destroy.sh (via bin/ceph)
A representative stack from #210:
Direct leak of 25846 byte(s) in 28 object(s) allocated from:
#0 malloc
#1 /usr/bin/python3.10+0x16d7be
Direct leak of 20456 byte(s) in 29 object(s):
#1 PyDict_Copy /usr/bin/python3.10+0x16ae06
Direct leak of 8456 byte(s) in 14 object(s):
#1 _PyObject_GC_NewVar /usr/bin/python3.10+0x14fc57
The unittests show the same shape against /lib/.../libpython3.10.so.
These are CPython 3.10 runtime artefacts, not Ceph bugs. In CPython
3.10, Py_FinalizeEx() leaves a set of interpreter-internal allocations
(module namespace dict copies, GC-tracked variable-size containers,
type-method caches, interned strings) heap-allocated until process
exit; the OS reclaims them. PEP 683 (Immortal Objects, accepted for
Python 3.12) extends pylifecycle.c::finalize_modules() to deallocate
these remaining objects during runtime shutdown:
"During runtime shutdown, the strategy will be to first let the
runtime try to do its best effort of deallocating these instances
normally. Most of the module deallocation will now be handled by
pylifecycle.c:finalize_modules() where we clean up the remaining
modules as best as we can."
-- PEP 683, https://peps.python.org/pep-0683/
The qa/lsan.supp file has carried the empirical note "python3.12
doesn't leak anything" since the file was introduced (commit
8c099a5340, "asan: add qa/lsan.supp for leak sanitizer
suppressions", Mar 2023). An empirical reproduction with a minimal
Py_Initialize()/Py_FinalizeEx() program built against Python 3.13
under -fsanitize=address reports zero leaks; the same minimal test
against Python 3.10 (CI environment) reports the leaks shown above.
Add three suppressions to qa/lsan.supp's existing 3.9-3.11 block:
- leak:^PyDict_Copy and leak:^_PyObject_GC_NewVar match the two
exported CPython functions visible in the leak stacks.
- leak:python3.10 substring-matches the module path in the stack
frame, catching unsymbolised offsets in both the /usr/bin/python3.10
binary and the libpython3.10.so.1.0 shared object. Mirrors the
existing leak:libsqlite3.so pattern earlier in the file.
The whole block (including the existing PyMem_Malloc entry above) can
be removed once CI runs on Python >= 3.12.
Signed-off-by: Kefu Chai <k.chai@proxmox.com>
follow up on commit f3c938f503
```
2026-04-22T10:16:21.350 DEBUG:teuthology.orchestra.run.trial118:> sudo ceph-bluestore-tool zap-device --dev /dev/nvme3n1 --yes-i-really-really-mean-it
2026-04-22T10:16:21.358 INFO:teuthology.orchestra.run.trial118.stderr:sudo: ceph-bluestore-tool: command not found
```
ceph-bluestore-tool is not available on the host and must be run from within a container.
Instead, use `ceph-volume lvm zap` which internally calls `ceph-bluestore-tool zap-device`.
Fixes: https://tracker.ceph.com/issues/76238
Signed-off-by: Guillaume Abrioux <gabrioux@ibm.com>
some commands during setup expect the zone to exist already, so run
'radosgw-admin user list' to make sure a default zone/zonegroup are
created. avoid duplicating this in several subtasks by moving this to
its own subtask that runs when a realm is not configured
Signed-off-by: Casey Bodley <cbodley@redhat.com>
This type of warning is typical during tests where we are thrashing the
OSDs.
Fixes: https://tracker.ceph.com/issues/73147
Signed-off-by: Laura Flores <lflores@ibm.com>
Currently it's set on the intermediary clone instead of the parent.
As a result the setting is effective only for reads that terminate at
the intermediary clone -- reads that go all the way to the parent may
end up being handled as not sparse depending on their size.
Fixes: https://tracker.ceph.com/issues/76101
Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
Fixes: https://tracker.ceph.com/issues/72938
Signed-off-by: szuraski898 <steven.zuraski@ibm.com>
Conflicts:
qa/valgrind.supp
The qa/valgrind.supp file had a conflict with an extra curly brace at the EOF as a result of changes adding CPython suppressions in the unittest-mgr branch, as well as additions coming form main.
Resolved by keeping both: the CPython suppressions for the mgr unit tests and all the rocky10 suppressions from main (rocky10_gcc14_mismatch_delete_map_erase
and rocky10 still reachable rocksdb leak), and removing the extra curly brace after merging.
qa/valgrind: generalize suppressions for gcc-14 MismatchedFree
Reviewed-by: Adam C. Emerson <aemerson@redhat.com>
Reviewed-by: Ilya Dryomov <idryomov@gmail.com>
